CN110850711A - Auxiliary driving control system and method based on cloud - Google Patents
Auxiliary driving control system and method based on cloud Download PDFInfo
- Publication number
- CN110850711A CN110850711A CN201911238364.2A CN201911238364A CN110850711A CN 110850711 A CN110850711 A CN 110850711A CN 201911238364 A CN201911238364 A CN 201911238364A CN 110850711 A CN110850711 A CN 110850711A
- Authority
- CN
- China
- Prior art keywords
- vehicle
- data
- control
- cloud
- driving
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 47
- 230000006854 communication Effects 0.000 claims abstract description 19
- 238000004891 communication Methods 0.000 claims abstract description 18
- 230000006399 behavior Effects 0.000 claims description 35
- 238000012549 training Methods 0.000 claims description 35
- 238000013500 data storage Methods 0.000 claims description 26
- 238000013527 convolutional neural network Methods 0.000 claims description 12
- 238000004364 calculation method Methods 0.000 claims description 11
- 238000012544 monitoring process Methods 0.000 claims description 11
- 238000003032 molecular docking Methods 0.000 claims 1
- 230000005540 biological transmission Effects 0.000 description 15
- 230000001360 synchronised effect Effects 0.000 description 6
- 230000002159 abnormal effect Effects 0.000 description 5
- 238000010586 diagram Methods 0.000 description 4
- 238000007781 pre-processing Methods 0.000 description 3
- 238000010276 construction Methods 0.000 description 2
- 238000013480 data collection Methods 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 238000006467 substitution reaction Methods 0.000 description 2
- 238000004458 analytical method Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000011217 control strategy Methods 0.000 description 1
- 238000013136 deep learning model Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 230000009977 dual effect Effects 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 238000003672 processing method Methods 0.000 description 1
- 239000013589 supplement Substances 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05B—CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
- G05B13/00—Adaptive control systems, i.e. systems automatically adjusting themselves to have a performance which is optimum according to some preassigned criterion
- G05B13/02—Adaptive control systems, i.e. systems automatically adjusting themselves to have a performance which is optimum according to some preassigned criterion electric
- G05B13/0265—Adaptive control systems, i.e. systems automatically adjusting themselves to have a performance which is optimum according to some preassigned criterion electric the criterion being a learning criterion
- G05B13/027—Adaptive control systems, i.e. systems automatically adjusting themselves to have a performance which is optimum according to some preassigned criterion electric the criterion being a learning criterion using neural networks only
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05B—CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
- G05B13/00—Adaptive control systems, i.e. systems automatically adjusting themselves to have a performance which is optimum according to some preassigned criterion
- G05B13/02—Adaptive control systems, i.e. systems automatically adjusting themselves to have a performance which is optimum according to some preassigned criterion electric
- G05B13/04—Adaptive control systems, i.e. systems automatically adjusting themselves to have a performance which is optimum according to some preassigned criterion electric involving the use of models or simulators
- G05B13/042—Adaptive control systems, i.e. systems automatically adjusting themselves to have a performance which is optimum according to some preassigned criterion electric involving the use of models or simulators in which a parameter or coefficient is automatically adjusted to optimise the performance
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/044—Recurrent networks, e.g. Hopfield networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/12—Protocols specially adapted for proprietary or special-purpose networking environments, e.g. medical networks, sensor networks, networks in vehicles or remote metering networks
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04W—WIRELESS COMMUNICATION NETWORKS
- H04W4/00—Services specially adapted for wireless communication networks; Facilities therefor
- H04W4/30—Services specially adapted for particular environments, situations or purposes
- H04W4/38—Services specially adapted for particular environments, situations or purposes for collecting sensor information
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04W—WIRELESS COMMUNICATION NETWORKS
- H04W4/00—Services specially adapted for wireless communication networks; Facilities therefor
- H04W4/30—Services specially adapted for particular environments, situations or purposes
- H04W4/40—Services specially adapted for particular environments, situations or purposes for vehicles, e.g. vehicle-to-pedestrians [V2P]
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Artificial Intelligence (AREA)
- Theoretical Computer Science (AREA)
- Evolutionary Computation (AREA)
- Health & Medical Sciences (AREA)
- General Physics & Mathematics (AREA)
- Software Systems (AREA)
- Computing Systems (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Medical Informatics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Computer Networks & Wireless Communication (AREA)
- Mathematical Physics (AREA)
- Signal Processing (AREA)
- General Engineering & Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Automation & Control Theory (AREA)
- Traffic Control Systems (AREA)
Abstract
The invention belongs to the technical field of automatic driving, and particularly relates to a cloud-based auxiliary driving control system and method, aiming at solving the problem that the control and scheduling of the take-over of one-to-many vehicles cannot be realized by a driver operating a driving simulator to carry out remote auxiliary driving control. The system comprises a cloud end arranged on a remote server and a vehicle end arranged on a controlled vehicle; the vehicle end is connected with the cloud end through a wireless communication link; the vehicle end is configured to acquire vehicle running environment data and vehicle driving behavior data and send the vehicle running environment data and the vehicle driving behavior data to the cloud end when switching to a remote control state based on the control state switching instruction, and acquire vehicle control data sent by the cloud end to control the vehicle; and the cloud end is configured to receive vehicle running environment data and vehicle driving behavior data, acquire vehicle control data corresponding to the controlled vehicle based on a preset automatic driving control model, and send the vehicle control data to the corresponding vehicle end. The invention realizes the control and dispatching of the take-over of one-to-many vehicles through the cloud.
Description
Technical Field
The invention belongs to the technical field of automatic driving, and particularly relates to a cloud-based auxiliary driving control system and method.
Background
In recent years, the development of an automatic driving technique has been accelerated, and automatic driving in a simple environment can be realized by modifying a conventional vehicle and providing sensors such as a camera, a laser radar, and a high-precision positioning device. However, due to the limitations of the computing resources and power consumption of the vehicle, the single vehicle is difficult to process the sensor data in real time and efficiently to perform the automatic driving decision planning calculation, so that the development of the single automatic driving function is limited. In addition, in some automatic driving vehicles in specific scenes, such as mines, ports, long-distance logistics transportation and the like, the situations of complex road conditions are often encountered, so that the single vehicle cannot have the decision-making capability of autonomous driving, and the problems of low operation efficiency and high safety risk are caused. Due to the limitation of the above situations, many single vehicles cannot use their own processors to process sensor data and/or cannot make automatic driving decisions under complex road conditions, and thus, there is a need for remote emergency take-over (or remote auxiliary driving control).
Furthermore, the 5G communication technology has the characteristics of high reliability, low time delay, large broadband and the like, so that a high-speed channel is provided for the automatic driving Internet of vehicles, and the remote auxiliary driving control becomes possible.
The existing remote auxiliary driving control process mostly adopts a remote driving control mode that a tester supervises a driving simulator, the operation aiming at the driving simulator is usually artificial, generally, only one automatic driving vehicle can be subjected to auxiliary control at the same time, and the taking over control and dispatching of a pair of multiple vehicles are difficult to perform. Aiming at the condition that a plurality of vehicles need to be remotely assisted to take over or control at the same time, the expandability of one-to-one control by manually operating the driving simulator is poor, and the cost is higher. In addition, the remote driving control method can realize the automatic upgrade of the driving control strategy only through external equipment or manual instructions, and cannot realize the flexible upgrade operation of the remote driving control method.
Disclosure of Invention
In order to solve the above problems in the prior art, that is, to solve the problem that the control of taking over and dispatching of one-to-many vehicles cannot be realized by a driver operating a driving simulator to perform remote assistant driving control, a first aspect of the present invention provides a cloud-based assistant driving control system, which includes a cloud end arranged on a remote server and a vehicle end arranged on a controlled vehicle; the vehicle end is connected with the cloud end through a wireless communication link;
the vehicle end is configured to acquire first data and send the first data to the cloud end when switching to a remote control state based on a control state switching instruction, and acquire second data sent by the cloud end to control a vehicle; the first data comprises vehicle running environment data and vehicle driving behavior data; the second data comprises vehicle control data;
and the cloud end is configured to receive the first data, calculate vehicle control data corresponding to the controlled vehicle as the second data based on a preset automatic driving control model, and send the second data to the corresponding vehicle end.
In some preferred embodiments, the driving assistance control system further includes an emergency interface end; the cloud end also comprises an emergency takeover monitoring module; the emergency connection pipe end is a manual remote control end;
the emergency take-over monitoring module is configured to establish an information channel between the vehicle end and the emergency take-over end when vehicle control data calculation fails;
the emergency receiving end is configured to receive first data of a controlled vehicle, acquire manual control information as vehicle control data and send the manual control information to a vehicle end corresponding to the controlled vehicle;
and when the vehicle control data is failed to be calculated, the time interval for sending the vehicle control data to the vehicle end of the corresponding controlled vehicle by the automatic driving control model at the cloud end is greater than a preset time threshold value.
In some preferred embodiments, the vehicle end comprises a first acquisition module, a second acquisition module:
the first acquisition module is configured to acquire vehicle running environment video data of a vehicle in a running process in real time and mark a timestamp in real time to form vehicle running environment data;
the second acquisition module is configured to acquire control information of vehicle driving in the running process of the vehicle in real time and mark a timestamp in real time to form vehicle driving behavior data.
In some preferred embodiments, the vehicle end further comprises a vehicle controller module;
and the vehicle controller module is configured to acquire and analyze the second data to obtain control instructions for various driving control components and send the control instructions to controllers of corresponding control components so as to drive the components to act according to the corresponding control instructions.
In some preferred embodiments, the autopilot control model is constructed based on a convolutional neural network CNN and a long-short term memory network LSTM.
In some preferred embodiments, the automatic driving model is disposed in an automatic driving control module in the cloud, and the automatic driving module includes a first model unit and a second model unit;
the first model unit is used for carrying out model training based on a training sample, acquiring new model parameters and updating the automatic driving control model in the second model unit;
the second model unit is used for calculating the second data based on an automatic driving control model.
In some preferred embodiments, the cloud comprises a data storage module for storing training samples of the automatic driving control model; the data storage module comprises a first data storage module, a second data storage module and a third data storage module;
the first data storage module is configured to acquire first data which are not subjected to auxiliary driving control at the vehicle end and store the first data as a first type of training sample;
the second data storage module is configured to acquire first data acquired by the manual remote control end in a real or simulated environment and store the first data as a second type of training sample;
and the third data storage module is configured to acquire the first data acquired by the acquisition vehicle in the real-time operation process and store the first data as a third type of training sample.
In some preferred embodiments, the wireless communication link is a 5G wireless communication link.
In a second aspect of the present invention, a cloud-based assistant driving control method is provided, where the cloud-based assistant driving control system includes the following steps:
step S100, when the vehicle end is switched to a remote control state based on a control state switching instruction, acquiring first data of a corresponding controlled vehicle in real time and sending the first data to the cloud end; the first data comprises vehicle running environment data and vehicle driving behavior data;
step S200, the cloud acquires vehicle control data corresponding to a controlled vehicle as second data through a preset automatic driving control model based on the acquired first data of the controlled vehicle, and sends the second data to a corresponding vehicle end;
and step S300, the vehicle side controls the vehicle based on the acquired second data.
In some preferred embodiments, if the time interval for acquiring the vehicle control data corresponding to the controlled vehicle at the cloud is greater than the preset time threshold, the manual remote control end arranged in the remote control center acquires the first data of the controlled vehicle, and acquires manual control information as the vehicle control data and sends the vehicle control information to the vehicle end corresponding to the controlled vehicle.
The invention has the beneficial effects that:
the invention realizes the control and dispatching of the take-over of one-to-many vehicles through the cloud. According to the invention, the end-to-end automatic driving control model is constructed at the cloud end, and the cloud end resources are fully utilized to plan the automatic driving decision, so that the cost of the vehicle end sensor and the requirement of the computing resources are greatly reduced, and the limit of the power consumption and the computing resources of the vehicle end is broken through. Compared with a manual driver for operating a remote control device (driving simulator) to realize remote driving, the end-to-end automatic driving control model has the advantages that the automatic driving process is easy to expand through training and regular updating, convenience of parallel computing can be fully utilized, remote control on multiple vehicles is achieved, and remote taking over and dispatching of the multiple vehicles are achieved.
Meanwhile, the invention monitors the end-to-end automatic driving process, if the error occurs in the take-over of the end-to-end automatic driving process, the manually operated driving simulator can still be selected for take-over, the dual take-over guarantee is realized, and the safety is higher.
Drawings
Other features, objects and advantages of the present application will become more apparent upon reading of the following detailed description of non-limiting embodiments thereof, made with reference to the accompanying drawings.
Fig. 1 is a schematic diagram of an overall structure of a cloud-based assistant driving control system according to an embodiment of the present invention;
fig. 2 is a schematic diagram illustrating a detailed structure of a cloud-based assistant driving control system according to an embodiment of the present invention;
FIG. 3 is an exemplary diagram of training of an autopilot control model and control data acquisition using the model in accordance with one embodiment of the present invention;
fig. 4 is a flowchart illustrating a cloud-based assistant driving control method according to an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings, and it is apparent that the described embodiments are some, but not all embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The present application will be described in further detail with reference to the following drawings and examples. It is to be understood that the specific embodiments described herein are merely illustrative of the relevant invention and not restrictive of the invention. It should be noted that, for convenience of description, only the portions related to the related invention are shown in the drawings.
It should be noted that the embodiments and features of the embodiments in the present application may be combined with each other without conflict.
The invention discloses a cloud-based auxiliary driving control system, which comprises a cloud end arranged on a remote server and a vehicle end arranged on a controlled vehicle, as shown in fig. 1; the vehicle end is connected with the cloud end through a wireless communication link;
the vehicle end is configured to acquire first data and send the first data to the cloud end when switching to a remote control state based on a control state switching instruction, and acquire second data sent by the cloud end to control a vehicle; the first data comprises vehicle running environment data and vehicle driving behavior data; the second data comprises vehicle control data;
and the cloud end is configured to receive the first data, calculate vehicle control data corresponding to the controlled vehicle as the second data based on a preset automatic driving control model, and send the second data to the corresponding vehicle end.
In order to more clearly describe the cloud-based assistant driving control system of the present invention, details of various systems in the embodiments of the system of the present invention are described below with reference to the drawings.
The auxiliary driving control system based on the cloud end comprises the cloud end arranged on a remote server and a vehicle end arranged on a controlled vehicle; the vehicle end is connected with the cloud end through a wireless communication link. The number of the unmanned vehicles may be one or more, and the present invention is not limited to this. The cloud end in the embodiment of the invention can communicate with different unmanned vehicles, and the communication method, the communication process, the data processing method and the process of each vehicle end are the same.
Fig. 2 is a specific structural example diagram of a cloud-based assistant driving control system according to an embodiment of the present application, where a vehicle end includes a vehicle wireless transmission module 210, a first acquisition module (video acquisition module) 220, a second acquisition module (driving behavior acquisition module) 230, and a vehicle controller module 240; the cloud comprises a cloud wireless transmission module 110, an automatic driving control module 120, a video acquisition module 130, a data storage module 140 and an emergency takeover monitoring module 150; the manual remote control terminal includes a driving simulator 310. Each module of the vehicle end, the cloud end and the manual remote control end is detailed below.
1. Vehicle end
The vehicle wireless transmission module 210 is connected with the cloud end through a wireless communication link. The vehicle wireless transmission module 210 is configured to forward the received vehicle control data to the vehicle controller module 240 when the control state switching instruction is switched to the remote control state in the system assisted driving control task operation stage. And the system is also used for forwarding the vehicle running environment data received in real time and the driving behavior data at the moment of taking over to the cloud terminal in the operation stage of the system auxiliary driving control task when the control state switching instruction is switched to the remote control state, only sending the vehicle running environment data received in real time to the cloud terminal after the taking over is completed, namely starting timing from the moment of taking over, sending the vehicle running environment data and the driving behavior data at the moment of taking over when t is 1, and only sending the vehicle running environment data when t > is 2.
In addition, the vehicle wireless transmission module 210 is further configured to forward the remote control state instruction received in real time to the cloud end, so as to notify the cloud end that the current unmanned vehicle needs to perform a remote auxiliary driving control task. The request can be sent when the real-time vehicle running environment data of the current unmanned vehicle and the real-time vehicle driving behavior data corresponding to the current unmanned vehicle in synchronization are sent to the cloud end, and can also be sent before. In the embodiment of the invention, the communication between the vehicle end and the cloud end adopts a 5G wireless communication link, and the 5G communication technology has the characteristics of high reliability, low time delay, large bandwidth and the like, so that the high real-time performance of the cloud driving system can be ensured. Thus, the vehicle wireless transmission module 210 in the present invention is preferably a 5G communication module.
The first acquisition module 220, namely a video acquisition module, is configured to acquire video data of a surrounding environment of the vehicle in the operation process in real time, obtain video frame (sequence) data with timing information including a front side, a left rear side and a right rear side of the vehicle in the operation process, and then transmit the current video frame (sequence) data to the cloud end in real time through the vehicle wireless transmission module 210. In addition, the real-time vehicle driving environment data also comprises a vehicle identification code (such as a vehicle VIN code) of the current unmanned vehicle. Wherein, the first collecting module 220 includes: a front view information acquisition unit 221, a left rear view information acquisition unit 222, and a right rear view information acquisition unit 223.
As shown in fig. 2, the front view information collecting unit 221 is installed at a middle position of a roof of the vehicle, and is used for viewing video information of a main viewing angle in front of the vehicle, and front video frame (sequence) data with timing information and a vehicle identification code attached thereto, which characterize a running environment in front of the vehicle during running of the vehicle.
And a left rear-view information collecting unit 222, installed at a position of the left side mirror of the vehicle, for viewing the environment video information at the viewing angle of the left side mirror of the vehicle, and representing the left rear video frame (sequence) data with timing information and a vehicle identification code attached to the left rear operating environment of the vehicle during the operation of the vehicle.
And a right rear-view information collecting unit 223 installed at a position of the right side mirror of the vehicle for viewing the environment video information at the viewing angle of the right rear-view mirror of the vehicle, and right rear video frame (sequence) data attached with the timing information and the vehicle identification code for representing the running environment of the right rear side of the vehicle during running of the vehicle.
The vehicle controller module 240 includes controllers for various vehicle operation-related manipulating components, specifically, an instruction resolver 241, a steering wheel controller 242, an accelerator pedal controller 243, and a brake pedal controller 244. Further, the vehicle controller module 240 is configured to receive and analyze the vehicle control data from the cloud end forwarded by the vehicle wireless transmission module 210 by using the instruction analyzer 241, obtain control instructions (a steering wheel angle instruction, an accelerator pedal stroke instruction, and a brake pedal stroke instruction) for various driving operation components, and send the control instructions to the controllers (the steering wheel controller 242, the accelerator pedal controller 243, and the brake pedal controller 244) of corresponding operation components, so as to drive the corresponding driving operation components by using the various controllers, and operate the current vehicle according to the corresponding control instructions.
The second collecting module 230, i.e. the driving behavior collecting module, is connected to the vehicle controller module 240 via a vehicle bus, and specifically, is connected to various control component controllers in the vehicle controller module 240 via the vehicle bus. The second collecting module 230 is configured to collect, through the steering wheel controller 242, the accelerator pedal controller 243 and the brake pedal controller 244, position state data of various driving control components collected in real time during a running process of the vehicle (by controllers of various vehicle control components in the vehicle controller module 240), and mark a timestamp and a vehicle identification code in real time to obtain real-time driving behavior data (sequence) synchronized with video frame (sequence) data collected in real time by the video collecting module 220. The vehicle bus of the embodiment of the invention preferably adopts a CAN bus.
In addition, the second collection module 230 is also connected to a vehicle control unit (not shown in fig. 2) in the current unmanned vehicle through a vehicle bus. When the vehicle control unit detects that the current unmanned vehicle has a remote auxiliary driving control demand, a remote control state instruction containing a vehicle identification code is generated, and the instruction is forwarded to the cloud end through the vehicle wireless transmission module 210. The above-mentioned remote auxiliary driving control demand is preferably the abnormal condition of various sensors or parts in the current vehicle, and the vehicle real-time fault detection is carried out by the vehicle control unit to learn, for example: abnormal conditions such as vehicle positioning failure, abnormal decision making and the like cause that the vehicle cannot finish the automatic driving capability by itself.
2. Cloud
The cloud wireless transmission module 110 is connected with each vehicle end through a wireless communication link. The cloud wireless transmission module 110 is configured to forward the received vehicle control data (output by the autopilot control module 120) for each controlled vehicle to the vehicle end of the corresponding controlled vehicle in the system assisted driving control task operation stage.
In addition, the cloud wireless transmission module 110 is further configured to start timing from the moment of taking over in the system driving assistance control task operation stage, receive the vehicle driving environment data and the vehicle driving behavior data at the moment of taking over when t is equal to 1, and receive only the vehicle driving environment data and forward the received data to the automatic driving control module 120 when t > is equal to 2. In addition, the cloud wireless transmission module 110 is further configured to forward the remote control status instruction received in real time to the emergency takeover monitoring module 150, so as to monitor the cloud driving process.
And an automatic driving control module 120 which comprises a first model unit 121 and a second model unit 122.
The first model unit 121 is configured to perform model training based on the training samples, obtain new model parameters, and update the automatic driving control model in the second model unit.
A second model unit 122 for performing a calculation of second data based on the automatic driving control model. That is, according to the vehicle driving environment data representing that the current controlled vehicle t is 1, the vehicle driving behavior data, or the vehicle driving environment data representing that t is 2 received by the cloud wireless transmission module 110, as an input of the online driving calculation, the automatic driving control model pre-constructed by the model training unit 121 is invoked and utilized, and the vehicle control data (corresponding to the current video frame data) at the synchronization time is directly calculated.
The video acquiring module 130 is configured to, in a model training phase, acquire video frames (sequences) of historical vehicle driving environment data of the same unmanned vehicle, and perform a video sequence preprocessing operation of time alignment on the video frame (sequence) data from multiple perspectives according to timestamp information. In addition, the video acquiring module 130 is further configured to, in the running phase, perform a video sequence preprocessing operation of time aligning video frame (sequence) data from multiple perspectives according to the timestamp information on the video frame (sequence) of the vehicle driving environment data of the same unmanned vehicle received in real time, so as to use the multi-perspectives video frame data subjected to the video sequence preprocessing operation as the input information of the online driving calculation. And when the vehicle is in the taking-over moment, namely t is 1, combining the synchronous corresponding real-time vehicle driving behavior data, inputting the data into the automatic driving calculation model, and training the automatic driving calculation model.
Further, the first model unit 121 in the automatic driving control module can be used for constructing the automatic driving control model according to historical vehicle driving environment data and synchronous corresponding historical vehicle driving behavior data in a training phase, so that the second model unit 122 automatically calculates vehicle control data of the controlled vehicle in a real-time running process on line through the created end-to-end automatic driving control model. In addition, the first model unit 121 can also be configured to update the automatic driving control model periodically according to the historical vehicle driving environment data and the synchronous corresponding real-time vehicle driving behavior data, so as to automatically complete the upgrading of the cloud automatic driving strategy. It should be noted that the frequency of updating the model is not specifically limited, and the frequency of automatic upgrade of the automatic driving method may be adjusted and set according to actual requirements.
Specifically, when the model is initially created or periodically updated, as shown in fig. 3, the first model unit 121 specifically performs the following steps:
acquiring historical vehicle driving environment data and historical vehicle driving behavior data (training data in a training set) synchronously corresponding to the historical vehicle driving environment data, aligning the two types of data according to timestamps marked in the two types of data by using the video acquisition module 130, and inputting the aligned data serving as model training data into a current automatic driving control model;
extracting image segment characteristics (driving behavior characteristics) of historical vehicle driving environment data and historical vehicle driving behavior data synchronously corresponding by using a convolutional neural network (CNN network) in the current automatic driving control model;
thirdly, calculating control data corresponding to the driving behavior characteristics in each frame of image and aiming at each driving control component by using an LSTM model in the current automatic driving control model according to the driving behavior characteristics in each frame of image and by combining time sequence information;
comparing the control data in each frame of image with historical real-time vehicle driving behavior data (sequence) of corresponding time to obtain errors of the control data of each type of driving behavior (the driving behavior comprises the control of a steering wheel, an accelerator pedal and a brake pedal), and adjusting parameters of each layer (a convolutional neural network and an LSTM network) in the current automatic driving control model by using the errors and a gradient descent algorithm;
and fifthly, training and verifying the automatic driving control model by using the adjusted model parameters to complete the initial construction or updating of the automatic driving control model.
The operational phase of fig. 3 is described in detail in the cloud-based assisted driving control method described below.
It should be noted that the first model unit 121 in the embodiment of the present invention stores the newly constructed or updated automatic driving control model for being called when performing the automatic driving online calculation. The latest current automatic driving control model comprises a convolutional neural network model and an LSTM model, the convolutional neural network model and the LSTM model are sequentially connected and are provided with a plurality of parallel computing channels, so that corresponding parallel data processing tasks can be completed on input information input to different channels, the automatic driving control model can drive real-time vehicle driving environment data input to the model and coming from one or more controlled vehicles and synchronously corresponding real-time vehicle driving behavior data simultaneously, vehicle control data aiming at each controlled vehicle are generated simultaneously, and the automatic driving control model can be forwarded to the corresponding controlled vehicle through the cloud wireless transmission module 110 in time. In the embodiment of the present invention, the number of parallel channels of the convolutional neural network model and the LSTM model in the current autopilot control model (the original convolutional neural network and the LSTM model before the initial construction, or the autopilot calculation model before the update) is equal, and the output end of the convolutional neural network model corresponds to the input end of the LSTM model.
In addition, in the implementation of the present invention, in order to improve the accuracy of the automatic driving control model of the present invention, a plurality of training data sources are adopted to process and store the historical vehicle driving environment data and the synchronous corresponding real-time vehicle driving behavior data so as to obtain training data from a plurality of sources. The data storage module 140 comprises a first data storage module 141, and the first data storage module 141 is used for acquiring first data which is not subjected to auxiliary driving control at the vehicle end and storing the first data as first type training data; the first data comprises vehicle running environment data and synchronous corresponding real-time vehicle driving behavior data.
The data storage module 140 further includes a second data storage module 142, which is connected to a manual remote control terminal (driving simulator 310), and is configured to obtain first data of the unmanned vehicle during operation, which is acquired during remote control driving in a real or simulated environment by using the driving simulator, and store the first data as second type of training data. It should be noted that the driving simulator in the present invention may be physically (in the form of a remote driving remote control device) disposed in a remote control center to perform remote driving control training in a real driving environment, or disposed on a system server side to perform simulated driving in a virtual driving environment constructed on the server side, so that the second data storage module 142 can obtain the second type of training data from the driving simulator under control in the above manner.
The data storage module 140 further includes a third data storage module 143, which is connected to the vehicle end of the data collection vehicle, wherein the data collection vehicle according to the embodiment of the present invention may further include a collection module installed therein to obtain third type training data. Specifically, the third data storage module is further configured to acquire first data acquired by the acquisition vehicle in real time during the operation process, and store the first data as third-class training data.
Therefore, the three ways can be used as training data of the end-to-end automatic driving control model in the embodiment of the invention, continuous feedback optimization is carried out regularly, the performance of the cloud is continuously improved, and automatic upgrade of the end-to-end automatic driving control model of the cloud is realized. The invention utilizes the training data obtained by the three ways as supplement to each other, further enriches the driving scene and behavior data, and improves the robustness and universality of the end-to-end automatic driving calculation model.
In conclusion, due to the easy expandability of the online remote automatic driving task implemented by utilizing the end-to-end deep learning model and the convenience of parallel computation, the technical scheme can be simultaneously used for remote control of multiple vehicles, and the remote auxiliary driving control of the multiple vehicles is realized.
The emergency takeover monitoring module 150 is mainly used for monitoring whether the cloud driving process of the controlled vehicle is successful or not, and starting a manual remote control driving task when the cloud driving process of the controlled vehicle is failed. Specifically, in an embodiment, the emergency takeover monitoring module 150 is further configured to monitor a feedback time of vehicle control data of the controlled vehicle, and when the feedback time reaches a preset takeover success time threshold, that is, a time interval during which the cloud sends the vehicle control data to the vehicle end corresponding to the controlled vehicle is greater than the preset time threshold, if the vehicle control data for the currently controlled vehicle is not successfully calculated, a remote control driving request instruction including a vehicle identification code of the currently controlled vehicle is generated, and the remote control driving request instruction is sent to a driving simulator 310 (a remote control device) connected to the cloud server and disposed in a remote control center, and the driving simulator is used to perform remote manual remote control driving control on the controlled vehicle.
Thus, by the arrangement of the emergency takeover monitoring module 150, in the embodiment of the invention, under the condition of a cloud driving task takeover error based on the end-to-end automatic driving control model, the takeover can still be performed in a mode of manually operating a driving simulator, double takeover guarantee is ensured, and the safety is high.
3. Manual remote control terminal
The manual remote control end, namely the emergency connecting end, mainly utilizes a driving simulator of a remote control center to carry out auxiliary driving control on a controlled vehicle.
The driving simulator 310 is configured to obtain an identification code of a vehicle that needs auxiliary control according to a remote control driving request instruction sent by the emergency takeover monitoring module 150, synchronize vehicle driving environment data and vehicle driving behavior data of a controlled vehicle if the driving simulator is in an idle state, and obtain manual control information as vehicle control data to send to a vehicle end corresponding to the controlled vehicle.
A cloud-based assistant driving control method according to a second embodiment of the present invention is based on the cloud-based assistant driving control system, as shown in fig. 4, and includes the following steps:
step S100, when the vehicle end is switched to a remote control state based on a control state switching instruction, acquiring first data of a corresponding controlled vehicle in real time and sending the first data to the cloud end; the first data comprises vehicle running environment data and vehicle driving behavior data;
step S200, the cloud acquires vehicle control data corresponding to a controlled vehicle as second data through a preset automatic driving control model based on the acquired first data of the controlled vehicle, and sends the second data to a corresponding vehicle end;
and step S300, the vehicle side controls the vehicle based on the acquired second data.
In order to more clearly describe the cloud-based assistant driving control method, the following describes in detail various steps in an embodiment of the method according to the present invention with reference to the drawings.
Step S100, when the vehicle end is switched to a remote control state based on a control state switching instruction, acquiring first data of a corresponding controlled vehicle in real time and sending the first data to the cloud end; the first data comprises vehicle running environment data and vehicle driving behavior data.
In this embodiment, the vehicle control unit additionally detects the abnormal conditions of each sensor or component of the current vehicle, when the abnormal conditions are detected, the vehicle control unit is switched to the remote control state based on the control state switching instruction, and the vehicle end acquires the vehicle running environment data and the vehicle driving behavior data of the current vehicle and sends the data to the cloud end through the wireless communication link.
And S200, the cloud acquires vehicle control data corresponding to the controlled vehicle as second data through a preset automatic driving control model based on the acquired first data of the controlled vehicle, and sends the second data to the corresponding vehicle terminal.
In this embodiment, the cloud acquires the vehicle control data of the controlled vehicle through the trained automatic driving control model based on the received first data of the requested controlled vehicle, and sends the vehicle control data to the corresponding vehicle terminal.
And step S300, the vehicle side controls the vehicle based on the acquired second data.
In this embodiment, based on the received vehicle control data, the vehicle controller module performs analysis to obtain control commands for various driving operation components, and sends the control commands to the controllers of the corresponding operation components to drive the components to act according to the corresponding control commands.
Wherein the training process of the automatic driving control model refers to the updating process of the first model unit 121. Meanwhile, the invention can update the model regularly according to the data acquired by history so as to automatically complete the upgrading of the remote cloud automatic driving strategy.
In addition, in the above steps, when the control state switching instruction is switched to the remote control state, the cloud starts to take over, and starts to time from the moment of taking over, when t is 1, the vehicle driving environment data and the vehicle driving behavior data of the current vehicle are acquired, and after taking over, that is, when t is 2, only the vehicle driving environment data of the current vehicle needs to be acquired, and then the vehicle control data at the current moment is acquired according to the vehicle control data calculated by the last inference and sent to the vehicle side until the taking over is finished.
Those of skill in the art would appreciate that the various illustrative modules, method steps, and modules described in connection with the embodiments disclosed herein may be implemented as electronic hardware, computer software, or combinations of both, and that programs corresponding to the software modules, method steps may be located in Random Access Memory (RAM), memory, Read Only Memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art. To clearly illustrate this interchangeability of electronic hardware and software, various illustrative components and steps have been described above generally in terms of their functionality. Whether such functionality is implemented as electronic hardware or software depends upon the particular application and design constraints imposed on the solution. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.
The terms "first," "second," and the like are used for distinguishing between similar elements and not necessarily for describing or implying a particular order or sequence.
The terms "comprises," "comprising," or any other similar term are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus.
So far, the technical solutions of the present invention have been described in connection with the preferred embodiments shown in the drawings, but it is easily understood by those skilled in the art that the scope of the present invention is obviously not limited to these specific embodiments. Equivalent changes or substitutions of related technical features can be made by those skilled in the art without departing from the principle of the invention, and the technical scheme after the changes or substitutions can fall into the protection scope of the invention.
Claims (10)
1. The auxiliary driving control system based on the cloud end is characterized by comprising the cloud end arranged on a remote server and a vehicle end arranged on a controlled vehicle; the vehicle end is connected with the cloud end through a wireless communication link;
the vehicle end is configured to acquire first data and send the first data to the cloud end when switching to a remote control state based on a control state switching instruction, and acquire second data sent by the cloud end to control a vehicle; the first data comprises vehicle running environment data and vehicle driving behavior data; the second data comprises vehicle control data;
and the cloud end is configured to receive the first data, calculate vehicle control data corresponding to the controlled vehicle as the second data based on a preset automatic driving control model, and send the second data to the corresponding vehicle end.
2. The cloud-based assisted driving control system of claim 1, further comprising an emergency docking end; the cloud end also comprises an emergency takeover monitoring module; the emergency connection pipe end is a manual remote control end;
the emergency take-over monitoring module is configured to establish an information channel between the vehicle end and the emergency take-over end when vehicle control data calculation fails;
the emergency receiving end is configured to receive first data of a controlled vehicle, acquire manual control information as vehicle control data and send the manual control information to a vehicle end corresponding to the controlled vehicle;
and when the vehicle control data is failed to be calculated, the time interval for sending the vehicle control data to the vehicle end of the corresponding controlled vehicle by the automatic driving control model at the cloud end is greater than a preset time threshold value.
3. The cloud-based assisted driving control system of claim 1, wherein the vehicle side comprises a first acquisition module and a second acquisition module;
the first acquisition module is configured to acquire vehicle running environment video data of a vehicle in a running process in real time and mark a timestamp in real time to form vehicle running environment data;
the second acquisition module is configured to acquire control information of vehicle driving in the running process of the vehicle in real time and mark a timestamp in real time to form vehicle driving behavior data.
4. The cloud-based assisted-driving control system of claim 1, wherein the vehicle end further comprises a vehicle controller module;
and the vehicle controller module is configured to acquire and analyze the second data to obtain control instructions for various driving control components and send the control instructions to controllers of corresponding control components so as to drive the components to act according to the corresponding control instructions.
5. The cloud-based aided driving control system of any one of claims 1-4, wherein the automatic driving control model is constructed based on a Convolutional Neural Network (CNN) and a long-short term memory network (LSTM).
6. The cloud-based auxiliary driving control system according to any one of claims 1 to 4, wherein the automatic driving model is provided in an automatic driving control module of the cloud, and the automatic driving module includes a first model unit and a second model unit;
the first model unit is used for carrying out model training based on a training sample, acquiring new model parameters and updating the automatic driving control model in the second model unit;
the second model unit is used for calculating the second data based on an automatic driving control model.
7. The cloud-based assisted-driving control system of any one of claims 1-4, wherein the cloud comprises a data storage module for storing training samples of the automatic driving control model; the data storage module comprises a first data storage module, a second data storage module and a third data storage module;
the first data storage module is configured to acquire first data which are not subjected to auxiliary driving control at the vehicle end and store the first data as a first type of training sample;
the second data storage module is configured to acquire first data acquired by the manual remote control end in a real or simulated environment and store the first data as a second type of training sample;
and the third data storage module is configured to acquire the first data acquired by the acquisition vehicle in the real-time operation process and store the first data as a third type of training sample.
8. The cloud-based assisted-driving control system of claim 1, wherein the wireless communication link is a 5G wireless communication link.
9. A cloud-based assistant driving control method, based on any one of claims 1 to 8, comprising the following steps:
step S100, when the vehicle end is switched to a remote control state based on a control state switching instruction, acquiring first data of a corresponding controlled vehicle in real time and sending the first data to the cloud end; the first data comprises vehicle running environment data and vehicle driving behavior data;
step S200, the cloud acquires vehicle control data corresponding to a controlled vehicle as second data through a preset automatic driving control model based on the acquired first data of the controlled vehicle, and sends the second data to a corresponding vehicle end;
and step S300, the vehicle side controls the vehicle based on the acquired second data.
10. The cloud-based assistant driving control method according to claim 9, wherein if a time interval for the cloud to acquire vehicle control data corresponding to the controlled vehicle is greater than a preset time threshold, an artificial remote control end arranged in the remote control center acquires first data of the controlled vehicle, and acquires artificial control information as vehicle control data to send to a vehicle end corresponding to the controlled vehicle.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911238364.2A CN110850711A (en) | 2019-12-06 | 2019-12-06 | Auxiliary driving control system and method based on cloud |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911238364.2A CN110850711A (en) | 2019-12-06 | 2019-12-06 | Auxiliary driving control system and method based on cloud |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN110850711A true CN110850711A (en) | 2020-02-28 |
Family
ID=69608391
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911238364.2A Pending CN110850711A (en) | 2019-12-06 | 2019-12-06 | Auxiliary driving control system and method based on cloud |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110850711A (en) |
Cited By (32)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111428646A (en) * | 2020-03-25 | 2020-07-17 | 东风汽车集团有限公司 | Driver take-over guarantee method and system based on edge cloud computing |
| CN111477030A (en) * | 2020-04-14 | 2020-07-31 | 北京汽车集团有限公司 | Vehicle collaborative risk avoiding method, vehicle end platform, cloud end platform and storage medium |
| CN111688719A (en) * | 2020-06-28 | 2020-09-22 | 广州小鹏汽车科技有限公司 | Driving method and device |
| CN111856966A (en) * | 2020-07-14 | 2020-10-30 | 东风汽车集团有限公司 | HIL simulation test system and method for 5G parallel driving system |
| CN112000104A (en) * | 2020-08-28 | 2020-11-27 | 清华大学 | Unmanned vehicle system based on cloud service |
| CN112085563A (en) * | 2020-09-04 | 2020-12-15 | 浪潮云信息技术股份公司 | CNN-based cloud hard disk resource management method |
| CN112351102A (en) * | 2020-11-10 | 2021-02-09 | 上海汽车集团股份有限公司 | Remote driving method and system |
| CN112389464A (en) * | 2020-10-19 | 2021-02-23 | 雅析安全系统(上海)有限公司 | Automatic driving remote control method and system meeting safety level requirements |
| CN112650247A (en) * | 2020-12-22 | 2021-04-13 | 北京百度网讯科技有限公司 | Remote control method, cockpit, cloud server and automatic driving vehicle |
| CN112702660A (en) * | 2020-12-18 | 2021-04-23 | 北京百度网讯科技有限公司 | Multimedia data transmission method and device, automatic driving vehicle and cloud server |
| CN112721952A (en) * | 2021-01-29 | 2021-04-30 | 重庆长安汽车股份有限公司 | Complex road section driving control method and device of L3-grade automatic driving automobile, automobile and computer storage medium |
| CN113110415A (en) * | 2021-03-17 | 2021-07-13 | 北京汽车研究总院有限公司 | Unmanned vehicle, control method and system thereof, cloud terminal and storage medium |
| CN113132474A (en) * | 2021-04-14 | 2021-07-16 | 东软睿驰汽车技术（沈阳）有限公司 | Control method, device and equipment for automatically driving vehicle |
| CN113260548A (en) * | 2020-05-20 | 2021-08-13 | 深圳元戎启行科技有限公司 | Method and control device for managing vehicle controller, and method for remotely controlling vehicle |
| CN113285993A (en) * | 2021-05-17 | 2021-08-20 | 科大讯飞股份有限公司 | Remote assistant driving access matching method, device and equipment |
| CN113459971A (en) * | 2021-06-27 | 2021-10-01 | 深圳新联胜光电科技有限公司 | Vehicle driving data acquisition method and system |
| CN113570936A (en) * | 2021-07-22 | 2021-10-29 | 广州小鹏汽车科技有限公司 | Driver simulator interface processing method, remote control platform and system |
| CN113589820A (en) * | 2021-08-12 | 2021-11-02 | 广州小鹏自动驾驶科技有限公司 | Auxiliary processing method, device and system for remote driving |
| CN113821010A (en) * | 2021-08-11 | 2021-12-21 | 安途智行(北京)科技有限公司 | Monitoring method for automatic driving vehicle drive test |
| CN113867315A (en) * | 2021-09-24 | 2021-12-31 | 同济大学 | Virtual-real combined high-fidelity traffic flow intelligent vehicle test platform and test method |
| CN113904959A (en) * | 2021-11-02 | 2022-01-07 | 广州小鹏自动驾驶科技有限公司 | Time delay analysis method and device, vehicle and storage medium |
| CN114115246A (en) * | 2021-11-09 | 2022-03-01 | 上海车右智能科技有限公司 | Remote driving auxiliary system and method thereof |
| CN114194210A (en) * | 2021-11-19 | 2022-03-18 | 岚图汽车科技有限公司 | Expressway navigation system, control method and equipment thereof |
| CN114464006A (en) * | 2022-04-13 | 2022-05-10 | 新石器慧通（北京）科技有限公司 | Method and device for allocating autonomous vehicles |
| CN114500502A (en) * | 2022-01-11 | 2022-05-13 | 武汉格罗夫氢能汽车有限公司 | Remote transmission method of vehicle-mounted T-box data |
| CN114895682A (en) * | 2022-05-19 | 2022-08-12 | 上海伯镭智能科技有限公司 | Unmanned mine car walking parameter correction method and system based on cloud data |
| CN115209164A (en) * | 2022-07-13 | 2022-10-18 | 郑州大学 | Block chain-based multi-view video transmitting method, server and storage medium |
| WO2022257056A1 (en) * | 2021-06-10 | 2022-12-15 | 深圳元戎启行科技有限公司 | Autonomous driving system, method and apparatus for vehicle, and computer device and storage medium |
| CN115497318A (en) * | 2022-09-28 | 2022-12-20 | 东风悦享科技有限公司 | Auxiliary driving platform suitable for public road remote driving |
| WO2023050606A1 (en) * | 2021-09-30 | 2023-04-06 | 中睿智能交通技术有限公司 | Vehicle remote interpolation fitting smooth control method and apparatus |
| CN116302010A (en) * | 2023-05-22 | 2023-06-23 | 安徽中科星驰自动驾驶技术有限公司 | Automatic driving system upgrade package generation method and device, computer equipment and medium |
| WO2024007691A1 (en) * | 2022-07-07 | 2024-01-11 | 腾讯科技（深圳）有限公司 | Remote driving control method and apparatus, computer-readable medium, and electronic device |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CA2984795A1 (en) * | 2015-05-06 | 2016-11-10 | Crown Equipment Corporation | Diagnostic tag for an industrial vehicle tag reader |
| CN106375738A (en) * | 2016-11-28 | 2017-02-01 | 东莞职业技术学院 | Intelligent vehicle video unmanned control device and intelligent vehicle video unmanned control system |
| CN108023937A (en) * | 2017-11-10 | 2018-05-11 | 北京汽车股份有限公司 | For the interchanger of vehicle, vehicle remote control apparatus, method and vehicle |
| CN108428357A (en) * | 2018-03-22 | 2018-08-21 | 青岛慧拓智能机器有限公司 | A kind of parallel remote driving system for intelligent network connection vehicle |
| CN108520238A (en) * | 2018-04-10 | 2018-09-11 | 东华大学 | A kind of scene prediction method of the night vision image based on depth prediction coding network |
| CN108549384A (en) * | 2018-05-21 | 2018-09-18 | 济南浪潮高新科技投资发展有限公司 | A kind of remote control automatic Pilot method under 5G environment |
| CN109446897A (en) * | 2018-09-19 | 2019-03-08 | 清华大学 | Scene recognition method and device based on image context information |
| CN109597412A (en) * | 2018-12-06 | 2019-04-09 | 江苏萝卜交通科技有限公司 | A kind of Unmanned Systems and its control method |
| CN109656134A (en) * | 2018-12-07 | 2019-04-19 | 电子科技大学 | A kind of end-to-end decision-making technique of intelligent vehicle based on space-time joint recurrent neural network |
-
2019
- 2019-12-06 CN CN201911238364.2A patent/CN110850711A/en active Pending
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CA2984795A1 (en) * | 2015-05-06 | 2016-11-10 | Crown Equipment Corporation | Diagnostic tag for an industrial vehicle tag reader |
| CN106375738A (en) * | 2016-11-28 | 2017-02-01 | 东莞职业技术学院 | Intelligent vehicle video unmanned control device and intelligent vehicle video unmanned control system |
| CN108023937A (en) * | 2017-11-10 | 2018-05-11 | 北京汽车股份有限公司 | For the interchanger of vehicle, vehicle remote control apparatus, method and vehicle |
| CN108428357A (en) * | 2018-03-22 | 2018-08-21 | 青岛慧拓智能机器有限公司 | A kind of parallel remote driving system for intelligent network connection vehicle |
| CN108520238A (en) * | 2018-04-10 | 2018-09-11 | 东华大学 | A kind of scene prediction method of the night vision image based on depth prediction coding network |
| CN108549384A (en) * | 2018-05-21 | 2018-09-18 | 济南浪潮高新科技投资发展有限公司 | A kind of remote control automatic Pilot method under 5G environment |
| CN109446897A (en) * | 2018-09-19 | 2019-03-08 | 清华大学 | Scene recognition method and device based on image context information |
| CN109597412A (en) * | 2018-12-06 | 2019-04-09 | 江苏萝卜交通科技有限公司 | A kind of Unmanned Systems and its control method |
| CN109656134A (en) * | 2018-12-07 | 2019-04-19 | 电子科技大学 | A kind of end-to-end decision-making technique of intelligent vehicle based on space-time joint recurrent neural network |
Cited By (43)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111428646A (en) * | 2020-03-25 | 2020-07-17 | 东风汽车集团有限公司 | Driver take-over guarantee method and system based on edge cloud computing |
| CN111428646B (en) * | 2020-03-25 | 2023-01-24 | 东风汽车集团有限公司 | Driver take-over guarantee method and system based on edge cloud computing |
| CN111477030A (en) * | 2020-04-14 | 2020-07-31 | 北京汽车集团有限公司 | Vehicle collaborative risk avoiding method, vehicle end platform, cloud end platform and storage medium |
| CN113260548A (en) * | 2020-05-20 | 2021-08-13 | 深圳元戎启行科技有限公司 | Method and control device for managing vehicle controller, and method for remotely controlling vehicle |
| WO2021232285A1 (en) * | 2020-05-20 | 2021-11-25 | 深圳元戎启行科技有限公司 | Method for managing vehicle controllers, control device and method for remotely controlling vehicle |
| CN113260548B (en) * | 2020-05-20 | 2024-04-19 | 深圳元戎启行科技有限公司 | Method for managing vehicle controller, control device and method for remotely controlling vehicle |
| CN111688719A (en) * | 2020-06-28 | 2020-09-22 | 广州小鹏汽车科技有限公司 | Driving method and device |
| CN111856966A (en) * | 2020-07-14 | 2020-10-30 | 东风汽车集团有限公司 | HIL simulation test system and method for 5G parallel driving system |
| CN112000104A (en) * | 2020-08-28 | 2020-11-27 | 清华大学 | Unmanned vehicle system based on cloud service |
| CN112085563A (en) * | 2020-09-04 | 2020-12-15 | 浪潮云信息技术股份公司 | CNN-based cloud hard disk resource management method |
| CN112389464A (en) * | 2020-10-19 | 2021-02-23 | 雅析安全系统(上海)有限公司 | Automatic driving remote control method and system meeting safety level requirements |
| CN112351102A (en) * | 2020-11-10 | 2021-02-09 | 上海汽车集团股份有限公司 | Remote driving method and system |
| CN112702660A (en) * | 2020-12-18 | 2021-04-23 | 北京百度网讯科技有限公司 | Multimedia data transmission method and device, automatic driving vehicle and cloud server |
| CN112702660B (en) * | 2020-12-18 | 2023-11-10 | 北京百度网讯科技有限公司 | Multimedia data transmission method and device, automatic driving vehicle and cloud server |
| CN112650247A (en) * | 2020-12-22 | 2021-04-13 | 北京百度网讯科技有限公司 | Remote control method, cockpit, cloud server and automatic driving vehicle |
| CN112721952A (en) * | 2021-01-29 | 2021-04-30 | 重庆长安汽车股份有限公司 | Complex road section driving control method and device of L3-grade automatic driving automobile, automobile and computer storage medium |
| CN112721952B (en) * | 2021-01-29 | 2022-07-05 | 重庆长安汽车股份有限公司 | Complex road section driving control method and device for L3-grade automatic driving automobile, automobile and computer storage medium |
| CN113110415A (en) * | 2021-03-17 | 2021-07-13 | 北京汽车研究总院有限公司 | Unmanned vehicle, control method and system thereof, cloud terminal and storage medium |
| CN113132474A (en) * | 2021-04-14 | 2021-07-16 | 东软睿驰汽车技术（沈阳）有限公司 | Control method, device and equipment for automatically driving vehicle |
| CN113285993A (en) * | 2021-05-17 | 2021-08-20 | 科大讯飞股份有限公司 | Remote assistant driving access matching method, device and equipment |
| WO2022257056A1 (en) * | 2021-06-10 | 2022-12-15 | 深圳元戎启行科技有限公司 | Autonomous driving system, method and apparatus for vehicle, and computer device and storage medium |
| CN113459971A (en) * | 2021-06-27 | 2021-10-01 | 深圳新联胜光电科技有限公司 | Vehicle driving data acquisition method and system |
| CN113459971B (en) * | 2021-06-27 | 2023-10-03 | 深圳新联胜光电科技有限公司 | Vehicle driving data acquisition method and system |
| CN113570936A (en) * | 2021-07-22 | 2021-10-29 | 广州小鹏汽车科技有限公司 | Driver simulator interface processing method, remote control platform and system |
| CN113821010A (en) * | 2021-08-11 | 2021-12-21 | 安途智行(北京)科技有限公司 | Monitoring method for automatic driving vehicle drive test |
| CN113589820A (en) * | 2021-08-12 | 2021-11-02 | 广州小鹏自动驾驶科技有限公司 | Auxiliary processing method, device and system for remote driving |
| CN113867315A (en) * | 2021-09-24 | 2021-12-31 | 同济大学 | Virtual-real combined high-fidelity traffic flow intelligent vehicle test platform and test method |
| WO2023050606A1 (en) * | 2021-09-30 | 2023-04-06 | 中睿智能交通技术有限公司 | Vehicle remote interpolation fitting smooth control method and apparatus |
| CN113904959B (en) * | 2021-11-02 | 2023-04-07 | 广州小鹏自动驾驶科技有限公司 | Time delay analysis method and device, vehicle and storage medium |
| CN113904959A (en) * | 2021-11-02 | 2022-01-07 | 广州小鹏自动驾驶科技有限公司 | Time delay analysis method and device, vehicle and storage medium |
| CN114115246B (en) * | 2021-11-09 | 2024-01-23 | 上海车右智能科技有限公司 | Remote driving auxiliary system and method thereof |
| CN114115246A (en) * | 2021-11-09 | 2022-03-01 | 上海车右智能科技有限公司 | Remote driving auxiliary system and method thereof |
| CN114194210A (en) * | 2021-11-19 | 2022-03-18 | 岚图汽车科技有限公司 | Expressway navigation system, control method and equipment thereof |
| CN114500502A (en) * | 2022-01-11 | 2022-05-13 | 武汉格罗夫氢能汽车有限公司 | Remote transmission method of vehicle-mounted T-box data |
| CN114464006B (en) * | 2022-04-13 | 2022-06-28 | 新石器慧通（北京）科技有限公司 | Method and device for allocating automatic driving vehicle |
| CN114464006A (en) * | 2022-04-13 | 2022-05-10 | 新石器慧通（北京）科技有限公司 | Method and device for allocating autonomous vehicles |
| CN114895682B (en) * | 2022-05-19 | 2024-04-30 | 上海伯镭智能科技有限公司 | Unmanned mine car walking parameter correction method and system based on cloud data |
| CN114895682A (en) * | 2022-05-19 | 2022-08-12 | 上海伯镭智能科技有限公司 | Unmanned mine car walking parameter correction method and system based on cloud data |
| WO2024007691A1 (en) * | 2022-07-07 | 2024-01-11 | 腾讯科技（深圳）有限公司 | Remote driving control method and apparatus, computer-readable medium, and electronic device |
| CN115209164B (en) * | 2022-07-13 | 2023-04-07 | 郑州大学 | Block chain-based multi-view video transmitting method, server and storage medium |
| CN115209164A (en) * | 2022-07-13 | 2022-10-18 | 郑州大学 | Block chain-based multi-view video transmitting method, server and storage medium |
| CN115497318A (en) * | 2022-09-28 | 2022-12-20 | 东风悦享科技有限公司 | Auxiliary driving platform suitable for public road remote driving |
| CN116302010A (en) * | 2023-05-22 | 2023-06-23 | 安徽中科星驰自动驾驶技术有限公司 | Automatic driving system upgrade package generation method and device, computer equipment and medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110850711A (en) | Auxiliary driving control system and method based on cloud | |
| US11842638B2 (en) | Parallel remote control driving system for intelligent network vehicle | |
| CN109808705B (en) | System for remotely controlling driving | |
| CN113327442B (en) | Cooperative control system and method based on end cloud fusion | |
| EP3418996A1 (en) | Real-time vehicle state trajectory prediction for vehicle energy management and autonomous drive | |
| CN107571864B (en) | The collecting method and device of automatic driving vehicle | |
| US20170262330A1 (en) | Computer system and method for safety-critical applications | |
| CN112650279B (en) | Cloud flight management system and cloud flight management method for airplane | |
| EP4280573A1 (en) | Method for returning data on vehicle, controller on vehicle, cloud server, and vehicle | |
| CN116931596A (en) | Unmanned aerial vehicle flight system with flight program automatically arranged | |
| US20220250637A1 (en) | Vehicle control and task processing method and apparatus, computing device and system | |
| CN109918390A (en) | The abnormality eliminating method and system of ship autonomous navigation | |
| CN115755923A (en) | Intelligent vehicle map interaction method, device, equipment and storage medium | |
| CN115314850A (en) | Intelligent motion system based on cloud edge cooperative control | |
| KR101190598B1 (en) | Autonomous apparatus and method for transitting to normal mode from anomaly status of geosynchronous satellite | |
| CN113211434A (en) | AR complex system equipment remote maintenance system based on Internet of things | |
| CN113110584A (en) | Multi-rotor aircraft cloud background network system and control method thereof | |
| CN112666980B (en) | Unmanned aerial vehicle cluster cooperation system, cooperation method and unmanned aerial vehicle cluster | |
| CN116698059B (en) | Processing method, storage medium and equipment for high-precision map reference line | |
| CN117539230B (en) | Intelligent driving domain controller pilot auxiliary function test method, equipment and storage medium | |
| CN115577318B (en) | Semi-physical-based data fusion evaluation method, system, equipment and storage medium | |
| CN115107791A (en) | Data acquisition triggering method, data acquisition method and data acquisition device | |
| CN108766003B (en) | Intelligent vehicle and driving control method thereof | |
| CN113037867B (en) | Unified data access and processing system for multiple unmanned aerial vehicles | |
| CN114442651B (en) | Method for judging working mode of airborne flight management system by cloud flight management system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20200228 |