CN113844465B

CN113844465B - Automatic driving method and system

Info

Publication number: CN113844465B
Application number: CN202111456284.1A
Authority: CN
Inventors: 王立
Original assignee: Shanghai Cheyou Intelligent Technology Co ltd
Current assignee: Shanghai Cheyou Intelligent Technology Co ltd
Priority date: 2021-12-02
Filing date: 2021-12-02
Publication date: 2022-07-15
Anticipated expiration: 2041-12-02
Also published as: CN113844465A

Abstract

The embodiment of the specification provides an automatic driving method and system, which are applied to the technical field of automatic driving, wherein the automatic driving method comprises the following steps: the remote take-over server receives vehicle behavior data reported by each automatic driving vehicle through a communication network; and monitoring the automatic driving vehicle according to the vehicle behavior data and the high-precision road data, and transmitting an intervention request to a remote take-over console when the automatic driving vehicle is determined to need remote take-over in the monitoring. By uploading vehicle behavior data occupying small bandwidth by the automatic driving vehicle, the requirement on network bandwidth resources in automatic driving can be reduced, the influence of network delay on automatic driving can be reduced, and automatic driving monitoring on all automatic driving vehicles can be met.

Description

Automatic driving method and system

Technical Field

The invention relates to the technical field of automatic driving, in particular to an automatic driving method and system.

Background

The "hierarchy for automation of automobile driving" (GB/T40429-2021) issued by the day ago, the function of automation of automobile driving has been divided into 6 levels from L0 to L5, where L3 (i.e., three-level driving automation) is conditional automatic driving, L4 (i.e., four-level driving automation) is highly automatic driving, L5 (i.e., five-level driving automation) is fully automatic driving, and three-level, four-level automatic driving is not unmanned.

In the existing automatic driving scenario, remote take-over of automatic driving is realized by utilizing the characteristics of high bandwidth and low time delay of a mobile communication network (such as 5G technology), and the remote take-over becomes a very important application scenario in the field of automatic driving, for example, remote automatic driving is realized based on C-V2X (car networking technology), where C-V2X is a Vehicle wireless communication technology formed by evolution of Cellular communication technologies such as 3G/4G/5G, and "C" is a capital letter of Cellular, and V2X (Vehicle to evolution) indicates that a Vehicle "communicates with other things outside.

However, the existing remote driving scheme is an automatic driving scheme based on 4G/5G mobile communication technology, in which a remote driver directly operates a vehicle, and the working process thereof is briefly described as follows: a high-speed communication network is established between the automobile and a remote driver so as to carry out signal transmission and man-machine interaction of a large amount of data; the local control system of the automobile collects and feeds back the surrounding environment and the self state of the automobile during the operation of the automobile to a remote driver in real time through a local multi-path camera and a vehicle bus; a remote driver issues control commands, operation instructions and the like to the vehicle in the automatic driving process of the vehicle according to the vehicle feedback data; the local control system of the vehicle receives the control command, the operation command and other commands issued by the remote driver and then automatically controls the vehicle.

Therefore, the existing automatic driving scheme requires a remote driver to operate the vehicle, which is far from the requirement of automatic driving (such as three-level, four-level, or even five-level automatic driving), and a new automatic driving scheme is urgently needed.

Disclosure of Invention

In view of this, embodiments of the present specification provide an automatic driving method and system, which can reduce occupation of network bandwidth resources, improve network bandwidth utilization, and satisfy remote take-over of multiple vehicles at the same time.

The embodiment of the specification provides the following technical scheme:

the embodiment of the specification provides an automatic driving method, which is applied to a remote take-over server and comprises the following steps: receiving vehicle behavior data reported by each autonomous driving vehicle through a communication network, wherein the vehicle behavior data are generated by the autonomous driving vehicle in executing a dynamic driving task, and the vehicle behavior data comprise vehicle high-precision positioning data and vehicle actuator data; and monitoring a dynamic driving task of the automatic driving vehicle according to the vehicle behavior data and the high-precision road data, wherein the high-precision road data comprises high-precision map data and road surface data, and transmitting an intervention request to a remote takeover console when the automatic driving vehicle is determined to need remote takeover in the monitoring, so that a user of the remote takeover console makes a secondary decision on the intervention request based on human driving behaviors.

The embodiment of the present specification further provides an automatic driving method applied to an automatic driving vehicle, where the automatic driving vehicle is equipped with a communication unit, and the automatic driving method includes:

in the process of executing a dynamic driving task, transmitting vehicle behavior data corresponding to the dynamic driving task to a communication network through a communication unit according to a preset first sending strategy so as to transmit the vehicle behavior data to a remote takeover server, wherein the communication unit is matched with the communication network, the vehicle behavior data comprises vehicle high-precision positioning data and vehicle actuator data, and the remote takeover server is used for monitoring the dynamic driving task of the automatic driving vehicle according to the high-precision road data and the vehicle behavior data and transmitting an intervention request to a remote takeover control console when the automatic driving vehicle is determined to need remote takeover in the monitoring;

when a remote driving planning instruction is received through the communication unit, the dynamic driving task is continuously executed in combination with the remote driving planning instruction, wherein the remote driving planning instruction is generated by the remote take-over console according to a response of a user of the remote take-over console to the intervention request after the intervention request is received by the remote take-over console.

An embodiment of the present specification further provides an automatic driving method, which is applied to a remote takeover console, where the automatic driving method includes:

receiving an intervention request sent by a remote takeover server, wherein the intervention request is sent when the remote takeover server determines that a target automatic driving vehicle needs remote takeover in monitoring a dynamic driving task of the target automatic driving vehicle according to vehicle behavior data and high-precision road data;

presenting the intervention request to a user of the remote take-over console;

obtaining response data of a user of the remote takeover console to the intervention request;

and when the response data comprise a remote driving planning instruction, issuing the remote driving planning instruction to the target automatic driving vehicle so that the target automatic driving vehicle continues to execute the dynamic driving task after combining the received remote driving planning instruction.

An embodiment of the present specification further provides an automatic driving method, which is applied to a communication network, and the automatic driving method includes:

receiving vehicle behavior data reported by each autonomous driving vehicle, wherein the vehicle behavior data are generated by the autonomous driving vehicle in executing a dynamic driving task, and the vehicle behavior data comprise vehicle high-precision positioning data and vehicle actuator data;

transmitting the vehicle behavior data to a remote take-over server so that the remote take-over server monitors the dynamic driving task of the automatic driving vehicle according to the vehicle behavior data and the high-precision road data after receiving the vehicle behavior data;

receiving a remote driving planning instruction issued by a remote take-over console to the automatic driving vehicle, wherein the remote driving planning instruction is used for a user of the remote take-over console to remotely take over the automatic driving vehicle to perform a dynamic driving task according to an intervention request, and the intervention request is used for confirming the intervention request sent when the automatic driving vehicle needs to be remotely taken over in the process of monitoring the dynamic driving task of the automatic driving vehicle by a remote take-over server.

An embodiment of the present specification further provides an automatic driving system, including: the system comprises a plurality of automatic driving vehicles, a communication network, a remote take-over server and a remote take-over console, wherein the automatic driving vehicles are provided with communication units, are connected to the communication network through the communication units and are in communication connection with the remote take-over server and the remote take-over console respectively;

the automatic driving vehicle is used for transmitting vehicle behavior data corresponding to a dynamic driving task to the remote take-over server according to a preset first sending strategy in the process of executing the dynamic driving task, wherein the vehicle behavior data are generated by the automatic driving vehicle in the process of executing the dynamic driving task, and comprise vehicle high-precision positioning data and vehicle actuator data;

the remote takeover server is used for monitoring a dynamic driving task of the automatic driving vehicle according to high-precision road data and the vehicle behavior data, and transmitting an intervention request to the remote takeover console when the automatic driving vehicle is determined to need remote takeover in the monitoring, wherein the high-precision road data comprises high-precision map data and road surface data;

the remote takeover console is used for displaying the intervention request to a user of the remote takeover console, acquiring response data of the user of the remote takeover console to the intervention request, and issuing a remote driving planning instruction to the automatic driving vehicle corresponding to the intervention request when the response data comprises the remote driving planning instruction;

the autonomous vehicle is further configured to continue to perform the dynamic driving task in conjunction with a remote driving planning instruction when the remote driving planning instruction is received via the communication unit.

Compared with the prior art, the embodiment of the specification adopts at least one technical scheme which can achieve the beneficial effects that at least:

only a communication unit with a communication function is required to be installed on the automatic driving vehicle, a special and expensive remote driving vehicle-mounted unit (such as a multi-channel video plug flow device in the existing automatic driving) is not required to be installed on the automatic driving vehicle, and the vehicle is not required to carry out special processing on the real-time state data of the vehicle, and only the data are directly sent to a remote take-over server by the communication unit through a communication network, so that the networking cost in the automatic driving can be reduced, the network bandwidth utilization rate can be improved, and the low-delay, safe and reliable automatic driving can be realized based on the existing network;

the remote takeover server monitors the running behavior of the vehicle in automatic driving according to the vehicle behavior data reported by the vehicle, the remote takeover server does not need to spend much effort on processing video data, and transmits an intervention request to a console of a remote takeover operator when determining that the vehicle possibly needs remote takeover intervention, and the remote takeover intervention of a human is carried out by the remote takeover operator based on the human driving behavior, so that the efficiency of data transmission, processing and the like can be improved, and the safety and reliability of decision can be improved;

when the operator decides that the intervention is required to be taken over, the console generates a remote taking-over planning instruction and sends the remote taking-over planning instruction to the vehicle, the automatic driving vehicle combines a decision result (namely the remote taking-over planning instruction) made by the remote taking-over operator with the automatic driving to continue a subsequent dynamic driving task, the operator does not need to directly control the vehicle, the remote taking-over driver can carry out remote taking-over intervention on a plurality of vehicles of different vehicle types at the same time, the influence of fatal factors such as network delay and jitter and misoperation of the driver in the vehicle driving can be reduced, and the safety and the reliability of the automatic driving can be improved.

Drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required to be used in the embodiments will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings without creative efforts.

Fig. 1 is a schematic structural diagram of an automatic driving system provided in an embodiment of the present specification;

FIG. 2 is a schematic diagram of an automatic driving system according to an embodiment of the present disclosure;

FIG. 3 is a schematic diagram of an edge computing gateway providing logical semantic data in an autopilot system according to an embodiment of the present disclosure;

FIG. 4 is a flowchart of an autopilot method applied to a remote takeover server according to an embodiment of the present disclosure;

fig. 5 is a schematic diagram illustrating a monitoring decision performed by a remote takeover server in an automatic driving method according to an embodiment of the present disclosure;

fig. 6 is a schematic diagram illustrating trajectory prediction performed by a remote take-over server in an automatic driving method according to an embodiment of the present disclosure;

fig. 7 is a schematic diagram illustrating a remote takeover server performing a monitoring decision based on logical semantic data provided by an edge computing gateway in an automatic driving method according to an embodiment of the present disclosure;

fig. 8 is a schematic diagram illustrating a remote takeover server performing local path planning based on logical semantic data provided by an edge computing gateway in an automatic driving method according to an embodiment of the present disclosure;

fig. 9 is a schematic diagram illustrating that a remote takeover server performs abnormal behavior decision based on logical semantic data provided by an edge computing gateway in an automatic driving method according to an embodiment of the present specification;

fig. 10 is a flowchart of an automatic driving method applied to an automatic driving vehicle according to an embodiment of the present disclosure;

FIG. 11 is a flowchart of an autopilot method applied to a remote takeover console provided in an embodiment of the present disclosure;

fig. 12 is a flowchart of an automatic driving method applied to a communication network according to an embodiment of the present disclosure.

Detailed Description

The embodiments of the present application will be described in detail below with reference to the accompanying drawings.

The following description of the embodiments of the present application is provided by way of specific examples, and other advantages and effects of the present application will be readily apparent to those skilled in the art from the disclosure herein. It is to be understood that the embodiments described are only a few embodiments of the present application and not all embodiments. The present application is capable of other and different embodiments and its several details are capable of modifications and/or changes in various respects, all without departing from the spirit of the present application. It should be noted that the features in the following embodiments and examples may be combined with each other without conflict. All other embodiments obtained by a person of ordinary skill in the art based on the embodiments in the present application without making any creative effort belong to the protection scope of the present application.

It is noted that various aspects of the embodiments are described below within the scope of the appended claims. It should be apparent that the aspects described herein may be embodied in a wide variety of forms and that any specific structure and/or function described herein is merely illustrative. Based on the present application, one skilled in the art should appreciate that one aspect described herein may be implemented independently of any other aspects and that two or more of these aspects may be combined in various ways. For example, an apparatus may be implemented and/or a method practiced using any number and aspects set forth herein. Additionally, such an apparatus may be implemented and/or such a method may be practiced using other structure and/or functionality in addition to one or more of the aspects set forth herein.

It should be noted that the drawings provided in the following embodiments are only for illustrating the basic idea of the present application, and the drawings only show the components related to the present application rather than the number, shape and size of the components in actual implementation, and the type, amount and ratio of the components in actual implementation may be changed arbitrarily, and the layout of the components may be more complicated.

In addition, in the following description, specific details are provided to facilitate a thorough understanding of the examples. However, it will be understood by those skilled in the art that the aspects may be practiced without these specific details. The terms "first", "second", etc. are used for descriptive purposes only and are not to be construed as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, features described as being defined as "first," "second," etc. may explicitly or implicitly include one or more of the features. In the description of the present invention, "a plurality" means two or more unless otherwise specified.

Currently, the automatic driving schemes are remote automatic driving schemes, that is, a remote driver controls an automatically driven vehicle in real time through a high-speed communication network. However, in practical applications, there are several prominent problems:

one is that it relies heavily on the high bandwidth (for video transmission as needed), low latency (to ensure control efficiency) and reliability of the communication network.

Even if the communication network has the characteristics of high bandwidth, low delay and the like, the communication network has certain delay jitter, such as a few milliseconds, tens of milliseconds and the like, so that when a remote driver issues a control command in automatic driving, even if the network has the jitter of only a few milliseconds, unpredictable vehicle control deviation can still be brought to the automatic driving vehicle.

In addition, the communication network with the characteristics of high bandwidth, low delay and the like belongs to precious resources, the construction cost is high, and when the communication network is applied to automatic driving scenes of a large number of vehicles, the construction cost is high, and the resources of the network are easy to catch the minds.

Secondly, the driving ability of the remote driver is greatly depended on, and various vehicle control problems which may occur when the human driver drives locally, such as insufficient response, control errors and the like, cannot be avoided.

Third, when a plurality of vehicles requiring the remote driver to operate in the same period belong to vehicles having different vehicle dynamics characteristics, it is difficult (or substantially impossible) for the remote driver to smoothly and quickly switch from the control of one vehicle to another. Therefore, the remote driver cannot manage a plurality of remote vehicles, and more manpower and material resources are wasted to manage the plurality of vehicles, so that the original purpose of improving economic benefits by adopting 'automatic driving + remote driving' is not achieved.

In view of this, after deep research and improved exploration of the automatic driving scene, a new automatic driving application scheme is proposed: as shown in fig. 1, each autonomous driving vehicle (for example, a remote autonomous driving vehicle 1 to a remote autonomous driving vehicle n in the figure, where n is an integer greater than 1) is installed with a communication unit, and the communication unit is connected to a communication network (for example, a 4G/5G cellular mobile communication network), and during executing a dynamic driving task, each autonomous driving vehicle generates vehicle behavior data (i.e., vehicle real-time status data in the figure) in real time and transmits the vehicle behavior data to a remote take-over server through the communication network according to a preset transmission strategy; the remote takeover server monitors the automatic driving behaviors of all remote automatic driving vehicles in the area by combining high-precision road data (such as centimeter-level precision map data and road surface data in a driving area) with vehicle real-time state data reported by respective automatic driving vehicles, forms a monitoring result corresponding to each automatic driving vehicle in the monitoring, transmits an intervention request (such as a takeover request and a takeover suggestion) corresponding to a vehicle to an operator console (which can be called a remote takeover console) for remote takeover when determining that the vehicle possibly needs to take over remotely according to the monitoring result, so that an operator (namely a remote takeover driver) of the remote takeover can timely know the running condition of the vehicle needing to be taken over through the console and make a response decision, for example, when the vehicle really needs to take over, and issuing the remote taking-over planning instruction to the automatic driving vehicle by the control console, and combining the remote taking-over planning instruction with automatic driving by the automatic driving vehicle to continuously execute the dynamic driving task.

The vehicle behavior data may include vehicle high-precision positioning data and vehicle actuator data, such as vehicle high-precision position positioning data (e.g., positioning data of GPS, beidou, etc.), for example, the actuator data may be an instruction executed by an actuator in vehicle automatic driving, such as a braking instruction, an acceleration instruction, a steering instruction, and corresponding attitude data after execution of various instructions. In practice, the vehicle behavior data may be directly from an automatic driving system of the vehicle, which is not limited herein.

In the scheme, only a communication unit with a communication function is required to be installed on the automatic driving vehicle, wherein the communication unit is a communication unit matched with a communication network, for example, when the communication network is a 4G/5G cellular mobile communication network, the communication unit can be a 4G/5G wireless communication gateway, and a special and expensive remote driving vehicle-mounted unit (such as a multi-channel video plug flow device in the existing scheme) is not required to be installed on the automatic driving vehicle, and the vehicle real-time state data is not required to be specially processed by the vehicle, and only the data is required to be directly sent to a remote take-over server through the communication network by the communication unit.

The automatic driving scheme provided in the embodiment of the present specification is that a vehicle reports vehicle behavior data, a remote takeover server monitors an operation behavior of the vehicle during automatic driving, and when it is determined that the vehicle may need remote takeover intervention, the remote takeover server transmits an intervention request (such as a takeover request, a takeover recommendation, and other data) corresponding to the vehicle to a remote takeover operator console where a remote takeover person is located, and further, a human remote takeover operator performs a human secondary decision on the intervention request corresponding to the vehicle, and when it is determined that takeover is needed, the console generates a remote takeover planning instruction and issues the remote takeover planning instruction to the vehicle, and the automatic driving vehicle combines a decision result (i.e., the remote takeover planning instruction) made by the remote takeover operator to an automatic driving to continue a subsequent dynamic driving task without installing expensive auxiliary equipment (such as a multi-channel video streaming device) on the automatic driving vehicle, the vehicle is not required to send video data occupying extremely large bandwidth, the networking cost can be reduced, the network bandwidth utilization rate can be improved, low-delay, safe and reliable automatic driving can be realized based on the existing network, meanwhile, the remote take-over server is not required to spend the calculation power to process the video data, the calculation power requirement of the remote take-over server can be reduced, can improve the efficiency, reliability and safety of automatic driving decision, and can carry out secondary decision on the corresponding intervention request of the vehicle by human operators, and the decision result is sent to the vehicle, a remote takeover operator (namely a remote takeover driver) is not needed to directly control the vehicle, the remote control system can meet the requirement that a driver remotely takes over a plurality of vehicles at the same time, can reduce the influence of fatal factors such as network delay and jitter, misoperation of the driver and the like in vehicle driving, and can improve the safety and reliability of automatic driving.

It should be noted that, in the automatic driving solution provided in the embodiment of the present disclosure, the high-precision road data may be data in a remote takeover server, and may also be data provided by a high-precision map server, which is not limited herein. And the high-precision map server may remotely take over a server in the server, or may be a server other than the remote take-over server, which is not limited herein.

It should be noted that, in the automatic driving scheme provided in the embodiment of the present specification, the remote takeover server may be a server located on one side of the remote takeover console, and may also be a server located in a cloud, so that a connection relationship between the remote takeover server and the server is not limited here.

And the server may be a single server or a server cluster, and the form of the server does not limit the embodiments of the present specification.

It should be noted that, in the automatic driving scheme in the embodiment of the present specification, the number of remote automatic driving vehicles that are not limited may be simultaneously taken over, so in practical applications, the number of remote automatic driving vehicles that can be taken over may be configured according to conditions such as the performance of the deployed remote take-over server, the network bandwidth of the communication network, and the number of operators equipped for the failure rate of the remote vehicles, where the number of vehicles is not limited.

It should be noted that the remote autonomous vehicles may be spatially distributed at any location, as long as the location is convenient for the communication unit to connect with the communication network, for example, when the communication network is a 4G/5G cellular mobile communication network, the communication network can basically cover each road, and thus the vehicle may be located at any location under the coverage of the communication network.

The technical solutions provided by the embodiments in the present specification are described below with reference to the accompanying drawings.

Embodiments in this specification provide an automatic driving system, which may include: a plurality of automatic driving vehicles, a communication network, a remote take-over server and a remote take-over console.

As shown in fig. 1, in view of the fact that each autonomous vehicle is equipped with a communication unit, here, an autonomous vehicle is taken as an example for illustration, and the autonomous vehicle is equipped with a communication unit, so that an autonomous subsystem for performing dynamic driving tasks in the vehicle can be connected to a communication network through the communication unit, that is, the autonomous vehicle is connected to the communication network through the communication unit, and is further connected to a remote take-over server and a remote take-over console through the communication network.

In implementation, the automatic driving vehicle is configured to transmit vehicle behavior data corresponding to a dynamic driving task to the remote takeover server according to a preset first transmission policy in executing the dynamic driving task. The vehicle behavior data may be vehicle behavior data generated by the autonomous vehicle in executing a dynamic driving task, and the vehicle behavior data includes vehicle high-precision positioning data and vehicle actuator data.

In a specific implementation, the remote autonomous vehicle may be equipped with a 4G or 5G cellular mobile communication gateway as a communication unit, and the vehicle may directly transmit the vehicle behavior data to the server through the communication unit according to a first transmission strategy, for example, the transmission strategy during normal driving of the vehicle may be a timed transmission, for example, the vehicle transmits data to a 4G/5G base station around the vehicle at a certain frequency (e.g., 30 Hz), and for example, when the vehicle has a remote intervention, the transmission strategy may be a real-time transmission of the vehicle behavior data to the 4G/5G base station around the vehicle.

It should be noted that in the dynamic driving task, the autonomous vehicle may obtain the sensing, decision and execution behaviors required by the vehicle driving, such as vehicle lateral motion control, vehicle longitudinal motion control, target and event detection and response, driving decision, vehicle lighting and signal device control, and the like, under the control of its own autonomous driving subsystem, and the strategic functions, such as navigation, route planning, destination and path selection, and the like. Therefore, the dynamic driving task may be a driving task corresponding to any one of the automatic driving levels L3 to L5 in the "automatic classification of automobile driving", and is not limited herein.

In implementation, the remote takeover server may be configured to monitor a dynamic driving task of the autonomous vehicle according to the high-precision road data and the vehicle behavior data, and transmit an intervention request to the remote takeover console when it is determined in the monitoring that the autonomous vehicle needs to take over remotely, where the high-precision road data includes high-precision map data and road surface data.

In implementation, the remote takeover server is used as a core component of the whole automatic driving system and can perform whole-course monitoring decision on the driving behaviors of the automatic driving vehicles for executing dynamic driving tasks, so that the remote takeover server can combine vehicle behavior data (namely dynamic data) reported by the vehicles with high-precision road data (such as high-precision map data, road surface data and other static data), and utilize the existing automatic driving planning and control decision system to monitor and analyze the operation behaviors of all remote automatic driving vehicles in an area in real time, and further form decision results into intervention requests when the vehicles are decided to possibly need remote takeover, and give takeover requests, takeover suggestions and other intervention request related information which is convenient for the human drivers to make decisions to the human drivers.

For example, the takeover recommendation may include at least one of the following: new vehicle planning paths; keeping the original planned route for running; parking for waiting until further instructions are obtained; automatically driving the vehicle to a specified position and adjusting the vehicle to a specified posture; manually identifying the road surface barrier; communicating with a remote outside traffic commander and providing further path planning for the automatic driving vehicle according to the instruction; and (5) emergency parking.

In implementation, the remote takeover console is configured to display the intervention request to a user of the remote takeover console, acquire response data of the user of the remote takeover console to the intervention request, and issue the remote driving planning instruction to the autonomous vehicle corresponding to the intervention request when the response data includes a remote driving planning instruction.

The remote take-over console displays information in the intervention request, such as related planning and control information of the remote take-over server, take-over suggestions and the like, to a human being serving as a remote take-over operator, so that the human being can conveniently make secondary decisions. And then, the remote take-over console acquires response data of a human remote take-over operator for the intervention request after combining the information through human-computer interaction, and when the response data comprises a remote driving planning instruction corresponding to information such as selection and decision of a control method for the remote vehicle, the remote driving planning instruction is used for issuing an automatic driving vehicle corresponding to the intervention request so as to carry out remote take-over control on the remote automatic driving vehicle.

Therefore, the remote take-over operator (i.e. the user of the remote take-over console) does not directly control the autonomous vehicle, but issues the remote driving planning instruction to the vehicle, and the vehicle combines the remote driving planning instruction into the dynamic driving task being executed, i.e. the autonomous vehicle is further configured to continue to execute the dynamic driving task in combination with the remote driving planning instruction when receiving the remote driving planning instruction through the communication unit, so as to realize remote control and enable the vehicle to enter into safe driving in the dynamic driving task.

In an implementation, the remote driving planning instruction may be a planning instruction for guiding the autonomous vehicle to perform a dynamic driving task, such as a control instruction generated accordingly according to the takeover recommendation information mentioned in the intervention request.

It should be noted that the remote takeover console may provide a human-machine interaction interface for the remote takeover operator, and the human-machine interaction interface may be configured to present, to the remote takeover operator, information such as a monitoring analysis result and a takeover suggestion provided by the remote takeover server and related to the intervention request, and may be configured to obtain a secondary decision result, such as a human driving decision instruction, made by the human remote takeover operator, where the human-machine interaction is not limited.

In some embodiments, an edge computing gateway may be deployed in a key road segment (e.g., a key intersection, a road segment with high traffic flow, etc.), and then the edge computing gateway is used to monitor a dynamic traffic scene of the key road segment (e.g., the key intersection), and the edge computing gateway is used to provide more detailed traffic flow information of the key road segment, so that a remote takeover server makes a decision.

In an implementation, the automatic driving system may further include a plurality of edge computing gateways, where the edge computing gateways are deployed in a preset road segment, and are configured to acquire real-time traffic scene data of the road segment, generate logical semantic data according to the real-time traffic scene data, and transmit the logical semantic data to the remote takeover server, where the logical semantic data is logical information used to restore a dynamic traffic scene of the road segment, and the logical information includes the following data for representing each traffic participant in the road segment: type data, high precision position data, and pose data.

In some embodiments, the edge computing gateway may include a number of radar sensors configured to obtain point cloud data corresponding to each traffic participant in a dynamic traffic scene of the road segment, and a first computing subsystem configured to compute and generate the logical semantic data from the point cloud data;

and/or the edge computing gateway comprises a plurality of visual sensors and a second computing subsystem, wherein the visual sensors are used for acquiring video streams corresponding to all traffic participants in the dynamic traffic scene of the road section, and the second computing subsystem is used for generating the logic semantic data according to the video stream computation.

As shown in fig. 3, a plurality of laser radars (e.g. laser radar 1 to laser radar m in the figure) and a plurality of high definition cameras (e.g. high definition camera 1 to high definition camera n in the figure) may be used to obtain a dynamic traffic scene in real time, and real-time dynamic data related to each traffic participant in the dynamic traffic scene is restored to logical semantic data, such as real-time high-precision position data (e.g. position location at the centimeter level) of all traffic participants, attitude data (e.g. attitude less than 0.1 °) and type data (e.g. classification information of pedestrians, automobiles, motorcycles, bicycles, etc.) of the traffic participants in the dynamic traffic scene through a corresponding computing and processing unit, and the data is sent to a remote driving server through a 4G/5G wireless communication base station, so that the remote driving server can use the data, and in combination with other state data, performing driving planning and decision on the remote automatic driving vehicle monitored by the remote automatic driving vehicle at a server side, and performing fusion comparison with vehicle behavior data (such as locally executed instructions) sent by the remote automatic driving vehicle to decide whether the remote automatic driving vehicle monitored by the remote automatic driving vehicle has abnormal behavior and needs to remotely take over the intervention of an operator, and the like.

By deploying the edge computing gateway in the road section, the edge computing gateway extracts the dynamic scene data in the road section into the logic semantic data which can restore the dynamic traffic scene in the remote takeover server, so that the data volume uploaded to the remote takeover server is greatly reduced, and meanwhile, the requirement on the computing capacity of the remote takeover server is also reduced, and thus, the remote automatic driving vehicle can be monitored and taken over in a very large range.

The above description is for each subsystem in the automatic driving system, and the following description is for each embodiment for the automatic driving method that each subsystem needs to execute in the whole automatic driving scheme, and the same contents as those in the foregoing embodiment are briefly and schematically described.

Based on the same inventive concept, the embodiment of the specification further provides an automatic driving method, which can be applied to a remote take-over server, so that the remote take-over server can conveniently monitor and make decisions on driving behaviors of all automatic driving vehicles in executing dynamic driving tasks.

As shown in fig. 4, the automatic driving method may include the steps of:

step S202, vehicle behavior data reported by each automatic driving vehicle are received through a communication network, wherein the vehicle behavior data are generated by the automatic driving vehicles in executing dynamic driving tasks, and the vehicle behavior data comprise vehicle high-precision positioning data and vehicle actuator data.

And S204, monitoring the dynamic driving task of the automatic driving vehicle according to the vehicle behavior data and the high-precision road data.

In an implementation, the high-precision road data may include high-precision map data and road surface data, where the high-precision map data and the road surface data may be traffic road-related data obtained from a high-precision electronic map, and the road surface data may be road surface-related data, such as friction coefficient, undulation, and the like, which is not limited herein.

It should be noted that a high-precision electronic map (also referred to as a high-precision map) is map data for an autonomous vehicle, which is capable of accurately and comprehensively characterizing road features, and has a high absolute position precision (e.g., 1 m) and a higher relative position precision (e.g., on the order of centimeters, such as 10 cm). In addition, the high-precision map can also be a map recorded with more driving behavior details, such as typical driving behaviors, optimal acceleration points and braking points, road condition complexity, road vehicle conditions, signal coverage conditions and the like. The high-precision map can be a static high-precision map, a dynamic high-precision map and the like, wherein the static high-precision map generally comprises a large amount of vector map information such as lane models, road components and road attributes, the dynamic high-precision map is generally established on the static high-precision map, and the dynamic high-precision map further comprises more auxiliary information, such as real-time dynamic information of roads, such as road congestion conditions, construction conditions, traffic accidents, traffic control, weather influences and the like, and such as relatively static information of some roads, such as traffic lights, pedestrian crossings and the like. Therefore, the high-accuracy map data may be map data from a static high-accuracy map, a dynamic high-accuracy map, or the like, and the high-accuracy map data is not limited herein.

Step S206, determining whether the automatic driving vehicle is likely to need remote take-over; if yes, go to step S208; if not, go to step S210.

And step S208, transmitting an intervention request to the remote take-over console.

And S210, continuously monitoring the dynamic driving task of the automatic driving vehicle.

Through the steps S202 to S210, the remote takeover server restores the real-time driving scene of the automatic driving vehicle by using a small amount of data uploaded by the automatic driving vehicle as dynamic data and combining high-precision road data as static data, and timely sends an intervention request to the remote takeover console when the automatic driving vehicle is determined to possibly need remote takeover in monitoring, so that a user of the remote takeover console as a remote takeover operator can make a secondary decision on automatic driving behavior by using human thinking, and the automatic driving vehicle is continuously monitored without remote takeover. Therefore, in view of the fact that the number of transmission in automatic driving is small, the occupied bandwidth is small, time delay is achieved, all automatic driving vehicles can be monitored at the same time, the remote taking-over server primarily makes a decision that remote taking-over is possibly needed, then, human beings are requested to make secondary decisions, and safety and reliability of the automatic driving vehicles in the process of executing dynamic driving tasks can be guaranteed.

In some embodiments, the remote takeover server may employ the functional modules shown in fig. 5 to work together to complete the monitoring decision process in the automatic driving method.

As shown in fig. 5, the vehicle dynamics model provides a vehicle model in the remote take-over server for each autonomous driving vehicle; the track prediction module predicts a running track according to a vehicle model by combining road surface data information and vehicle real-time state data reported by a vehicle; the server-side automatic driving decision planning module can perform decision planning on the predicted driving track according to vehicle real-time state data, road-related high-precision map data information and/or real-time dynamic traffic data (namely logic semantic data representing dynamic traffic data) from the edge computing gateway to perform monitoring decision on the driving behavior of the vehicle, and when the fact that the vehicle possibly needs remote takeover is determined, the server-side automatic driving decision planning module further performs analysis through a vehicle abnormal behavior analysis module to request a remote takeover operator to take over the vehicle if the fact that the vehicle needs remote takeover is determined, and can also perform local path planning on the automatic driving vehicle through a vehicle local planning suggestion module to determine remote takeover control suggestions and the like of the automatic driving vehicle.

It should be noted that, the functional modules in the remote takeover server may be determined according to the needs of the actual application deployment, and are only described as an illustrative example here.

In some embodiments, although the autonomous vehicle does not need to upload a real-time video stream of the autonomous vehicle (for example, a video stream obtained by a camera in real time during autonomous driving), when a remote takeover operator actively requests video data, the remote takeover server may transmit the video stream uploaded by the autonomous vehicle to the remote takeover console, so that the remote takeover operator can view the video stream through the console in real time, and the remote takeover operator can conveniently respond to an intervention request.

In implementation, a remote takeover operator can send a request of video data to the remote takeover server through the console, or the remote takeover server knows that the remote takeover operator needs the video data after receiving the video data, so that the remote takeover server can transmit real-time video data uploaded by a vehicle to the remote takeover console.

Accordingly, the automatic driving method may further include: receiving vehicle real-time video data transmitted by the automatic driving vehicle; and carrying out video fusion on the vehicle real-time video data and then transmitting the video data to the remote takeover console.

It should be noted that the vehicle real-time video data transmitted by the autonomous vehicle may be video stream data sensed for a driving environment in a dynamic driving task performed for the autonomous vehicle, such as a video shot by a camera, video data processed by various sensor sensing data, and the like, and is not limited herein.

Therefore, the automatic driving vehicle uploads the video data only when the remote takeover operator actively requests the video data, so that the network bandwidth can be saved, the bandwidth utilization rate is improved, the calculation performance requirement of the remote takeover server for processing the video data is reduced, the response of the remote takeover operator to the intervention request can be facilitated, and the driving safety and reliability of the automatic driving vehicle are ensured.

In some embodiments, the remote takeover server may restore a vehicle running environment according to vehicle behavior data reported by a vehicle and predict a vehicle running track, so that vehicle behavior data continuously uploaded by the vehicle may be compared according to a prediction result, and a driving behavior of the autonomous vehicle may be reliably monitored and decided.

In practice, the step S204 of monitoring the dynamic driving task of the autonomous vehicle according to the vehicle behavior data and the high-precision road data may include:

predicting the running track of the automatic driving vehicle according to a vehicle dynamic model corresponding to the automatic driving vehicle by combining the vehicle behavior data and the road surface data;

and monitoring the driving track according to the vehicle behavior data and the high-precision map data so as to monitor the dynamic driving task of the automatic driving vehicle.

In some embodiments, the vehicle may be monitored and made decisions by restoring the virtual travel trajectory of the vehicle in the remote takeover server.

In an implementation, predicting the driving trajectory of the autonomous vehicle according to a vehicle dynamics model corresponding to the autonomous vehicle in combination with the vehicle behavior data and the road surface data may include:

inputting vehicle dynamics model parameters, the vehicle actuator data and the road surface data corresponding to the autonomous vehicle into a vehicle dynamics model;

predicting a virtual vehicle trajectory of the autonomous vehicle in the high-precision map data by the vehicle dynamics model;

and correcting the virtual vehicle track according to the vehicle attitude data and the vehicle high-precision positioning data corresponding to the vehicle actuator data to obtain the running track of the automatic driving vehicle.

It should be noted that the vehicle dynamics model may determine a corresponding model according to application requirements, for example, a seven-degree-of-freedom vehicle dynamics model, a fourteen-degree-of-freedom vehicle dynamics model, and the like, which is not limited herein.

In some embodiments, various monitoring results can be generated in the monitoring, so that all vehicles have corresponding monitoring results, and basic data can be provided for subsequent monitoring.

Accordingly, the automatic driving method may further include: and acquiring the real-time track and the acceleration of the automatic driving vehicle according to the running track. The data such as real-time track and/or acceleration of the vehicle can be used for analyzing whether the vehicle has abnormal driving behaviors such as yaw, collision and the like.

In some embodiments, the yaw of the vehicle may be monitored during the monitoring. Therefore, when the driving track is monitored by combining the vehicle behavior data with the high-precision map data, yaw detection can be performed on the driving track according to the vehicle behavior data, and once the situation that the vehicle possibly has yaw is predicted, the yaw can be known in time, so that the yaw can be corrected in a targeted manner subsequently, and the driving safety is improved.

In some embodiments, a functional block diagram of a trajectory prediction module, such as that shown in FIG. 6, may be employed to perform one or more of the correlation steps described above to facilitate real-time, reliable vehicle monitoring decisions.

As shown in fig. 6, the physical vehicle actuator data may be actuator data reported by vehicles, the vehicle dynamics model parameter library may be a parameter library in which dynamics models corresponding to various vehicles are stored, and provides corresponding dynamics model parameters for each vehicle in the current monitoring area, so as to construct a corresponding vehicle dynamics model (i.e., a 14-degree-of-freedom vehicle dynamics model in the drawing), and at this time, the vehicle dynamics model may combine the actuator data with high-precision map data to restore a driving scene of an automatically driven vehicle, so as to obtain a virtual vehicle trajectory. After the virtual track is obtained, the track of the virtual vehicle can be corrected in time by further utilizing vehicle behavior data such as vehicle real-time position and attitude data reported by the vehicle, accurate data such as real-time track, acceleration and the like can be obtained based on the corrected track, and the data can be output as basic data of subsequent monitoring.

In some embodiments, an edge computing gateway may be employed to provide dynamic traffic data for an emphasized road segment. It should be noted that the edge computing gateway only needs to provide logical semantic data for restoring a dynamic traffic scene, which may not only receive network bandwidth, but also reduce the computing processing of the remote takeover server.

In an implementation, the remote takeover server may receive logical semantic data provided by the edge computing gateway, that is, the automatic driving method may further include: receiving logic semantic data sent by each edge computing gateway, wherein the logic semantic data is logic information used for restoring a dynamic traffic scene of a road section where the edge computing gateway is located, and the logic information comprises the following data used for representing each traffic participant in the road section: type data, high precision position data, and pose data.

In some embodiments, the drivable regions between road participants in a road segment may be monitored based on logical semantic data provided by the edge computing gateway.

In real time, the remote take-over server monitors the safety distance of the dynamic driving scene where the automatic driving vehicle is located, that is, the automatic driving method may further include:

according to the logic semantic data, distance detection is carried out on the dynamic target object and distance detection is carried out on the static obstacle, so that a distance detection result corresponding to the automatic driving vehicle is obtained;

and combining the high-precision road data with the distance detection result to determine a travelable area parameter, wherein the travelable area parameter comprises traffic signal sign data and safe distance data, and the safe distance data comprises a safe distance from a dynamic target object and a safe distance from a static obstacle.

It should be noted that the safe distance may be determined and adjusted according to the actual application scenario, for example, the safe distance may consider various factors such as the traffic flow of the road section, the accident situation, and the like, for example, the highway is automatically driven, the safe distance may be set to 100m, and the safe distance may be set to 10m when the vehicle speed is slow on the road at a low speed, and the like, which is not limited herein.

By monitoring the drivable areas of all road participants in real time, unsafe driving behaviors of the automatic driving vehicle in executing a dynamic driving task can be found in time, the monitoring decision accuracy can be improved, and the safe driving of the automatic driving vehicle is ensured.

In some embodiments, the driving behavior of the autonomous vehicle may be further monitored using the travelable region parameters.

In implementation, the remote takeover server may monitor the autonomous vehicle based on the travelable region parameter, that is, the server monitors the dynamic driving task of the autonomous vehicle according to the vehicle behavior data and the high-precision road data, including: detecting the driving speed of the automatic driving vehicle according to the drivable region parameters, the vehicle behavior data and the high-precision road data; and monitoring the dynamic driving task of the automatic driving vehicle according to the driving speed and the safe distance data.

By combining the safe distance data according to the driving speed, whether the automatic driving vehicle and other traffic participants are safe or not can be timely and accurately determined, so that whether the automatic driving vehicle needs to take over remotely or not can be accurately determined.

In some embodiments, the travel speed and safe distance data may be utilized to monitor crash behavior that may exist in an autonomous vehicle in performing a dynamic driving task.

In implementation, the remote takeover server may monitor whether there is a possibility of collision behavior of the autonomous vehicle based on the driving speed and the safe distance data, that is, monitor a dynamic driving task of the autonomous vehicle according to the driving speed and the safe distance data, and includes: and according to the running speed and the safe distance data, performing collision detection on the automatic driving vehicle so as to monitor the dynamic driving task of the automatic driving vehicle.

For example, a monitoring decision is made as to whether there is a possibility of a collision between the autonomous vehicle and a target object, such as a dynamic target, a static target, or the like.

By monitoring the possible collision behaviors, the abnormal condition that whether the vehicle is likely to collide at the next time can be found in time, reliable basic data is provided for intervention of later-period remote takeover, an operator can conveniently take over the data remotely to make secondary decision of human driving behaviors, and the safety and the reliability of the automatic driving vehicle in executing a dynamic driving task are improved.

In some embodiments, one or more of the related steps described above may be performed using a decision-planning functional block diagram as shown in fig. 7, so that the remote takeover server may make monitoring decisions on the autonomous driving behavior of any virtual vehicle in the dynamic traffic scene of the road segment based on the high-precision logical semantic data provided by the edge computing gateway.

As shown in fig. 7, after receiving the high-precision positioning information of other traffic participants transmitted by the edge computing gateway, the remote takeover server may respectively perform safe distance detection on the distance between the autonomous vehicle and static obstacles (such as roadside objects, objects scattered in quantity, etc.) and dynamic objects (such as pedestrians, other vehicles, etc.) in the road section, determine areas where the vehicle can safely travel, such as lane positions, traveling directions, etc., by combining with the high-precision map data, obtain travelable area parameters from the obtained data of the areas where the vehicle can safely travel, determine traffic signal signs based on the travelable area parameters, such as whether the traffic signal signs are green light traffic, red light parking waiting, etc., further perform speed detection on the vehicle based on the vehicle trajectory prediction result, and predict the next-step traveling instruction of the vehicle by speed detection, the corresponding reduction data of the vehicle can be formed, and the vehicle can be monitored and decided based on the reduction data and the data reported by the vehicle.

In some embodiments, monitoring decisions may be made using a video stream perceived by the autonomous vehicle in a dynamic driving task.

In implementation, the remote takeover server may make a monitoring decision based on high-precision positioning information in vehicle behavior data uploaded by the autonomous vehicle, and thus the autonomous driving method may further include: and according to the travelable area parameters and the vehicle high-precision positioning data in the vehicle behavior data, performing local path planning on the dynamic driving task of the automatic driving vehicle.

It should be noted that the local path planning may be to predict a subsequent path of the autonomous vehicle to regenerate a local path in order to combine with the obstacle avoidance requirement, and the specific path planning mode may be set according to an application requirement, which is not limited herein.

By carrying out local path planning on the automatic driving vehicle, basic data of the driving behavior of the automatic driving vehicle in a dynamic driving task can be provided for follow-up monitoring, the basic data can be used as big data to predict the driving behavior of the vehicle subsequently, reference can be provided for obstacle avoidance decisions of the automatic driving vehicle, and the safety of automatic driving is improved.

In some embodiments, one or more of the related steps described above may be performed using a decision planning functional block diagram as shown in fig. 8, so that the remote takeover server may make monitoring decisions on the autonomous driving behavior of any virtual vehicle in the dynamic traffic scenario of the road segment based on high-precision logical semantic data provided by the edge computing gateway.

As shown in fig. 8, according to the high-precision positioning information of the traffic participants provided by the edge computing gateway, after distance detection is performed on a static obstacle and a dynamic target object, a vehicle travelable area parameter can be obtained by combining with high-precision map data, and then local path planning is performed on the vehicle in a local road section, and in the planning, other traffic participant information in the image can be extracted by combining with a remote image (such as an image provided by the edge computing gateway, the vehicle and the like) through an image recognition algorithm, and local path planning can be performed on the travelable area parameter and the other traffic participant information. In addition, the path priority can be judged for the local path planning, and interaction can be conveniently carried out on a user who takes over the console remotely.

In some embodiments, the remote take-over server may provide the local path planning data to a remote take-over console, facilitating the remote take-over operator to use as reference data in responding to the intervention request.

In an implementation, the remote take-over server may carry these local path planning data in case of sending an intervention to the remote take-over console, i.e. the autopilot method may further include: transmitting the local path plan to the remote take-over console.

The local path planning result is provided for the remote take-over console (or a user of the remote take-over console, namely, the user serves as a human remote take-over operator), so that the remote take-over operator can make secondary decisions based on human driving behaviors, and the decision accuracy of the remote take-over operator in responding to intervention situations is improved.

In some embodiments, monitoring decisions may be made based on historical data of the autonomous vehicle in combination with real-time data reported by the vehicle.

In implementation, the remote takeover server may perform a monitoring decision based on historical data and current data, that is, monitor the dynamic driving task of the autonomous vehicle according to the vehicle behavior data and the high-precision road data, including: inputting the vehicle behavior data and historical behavior data of the autonomous vehicle into a preset deep neural network, wherein the deep neural network is used for predicting vehicle actuator data in a dynamic driving task; determining an actuator deviation result according to the vehicle actuator data predicted by the deep neural network and the actuator data reported by the automatic driving vehicle; and monitoring the dynamic driving task of the automatic driving vehicle according to the deviation result of the actuator.

It should be noted that the preset deep neural network may be a neural network used for performing predictive classification on the output data of the vehicle actuator after identifying the target object in automatic driving, and may be set according to actual application requirements, which is not limited herein.

In some embodiments, the remote takeover server monitors the autonomous vehicle for anomalies in real-time in the monitoring decision.

In an implementation, the remote takeover server determining that the autonomous vehicle needs remote takeover in the monitoring may include: determining in the monitoring that at least one of the following abnormal situations exists with the autonomous vehicle: yaw exists in the real-time track; the real-time track exceeds the travelable area; the collision distance is less than a preset distance; the output deviation of the actuator exceeds a preset threshold value.

As shown in fig. 9, vehicle characteristics are identified from vehicle real-time status data (i.e., vehicle behavior data) reported by an autonomous vehicle and vehicle behavior data in a database through a deep neural network to obtain vehicle actuator output deviation data, and whether the output deviation belongs to an abnormal behavior can be determined through behavior comparison; yaw detection is carried out on the vehicle real-time position data and the vehicle track prediction data, so that the yaw condition of the vehicle, whether the vehicle runs in a drivable area and the like can be determined, and whether the current driving behavior belongs to abnormal driving behavior is determined through behavior comparison; the distance between the vehicle and the target object can be judged by utilizing the real-time position data of the vehicle, the high-precision map data and the like, the collision detection can be detected, whether the collision is possible or not can be determined through behavior comparison, and if the collision is possible, the collision is determined as abnormal behavior.

Based on the same inventive concept, the embodiment of the present specification further provides an automatic driving method, which can be applied to a remote automatic driving vehicle, is convenient for the automatic driving vehicle to provide various data to a remote takeover server, and is helpful for the automatic driving vehicle to perform a dynamic driving task by itself, wherein the automatic driving vehicle is provided with a communication unit, and the communication unit is used for being in communication connection with the remote takeover server and a remote takeover console through a communication network.

As shown in fig. 10, the automatic driving method may include the steps of:

step S402, in the process of executing the dynamic driving task, transmitting the vehicle behavior data corresponding to the dynamic driving task to a communication network through the communication unit according to a preset first transmission strategy so as to transmit the vehicle behavior data to a remote take-over server.

In implementation, the communication unit is matched with the communication network, the vehicle behavior data includes vehicle high-precision positioning data and vehicle actuator data, and the remote take-over server is configured to monitor a dynamic driving task of the autonomous vehicle according to the high-precision road data and the vehicle behavior data, and transmit an intervention request to a remote take-over console when it is determined in the monitoring that the autonomous vehicle needs to be remotely taken over.

Step S404, judging whether the intervention of remote takeover is needed; if yes, go to step S406; if not, go to step S408.

In implementation, when the remote taking-over intervention is needed, the remote taking-over console issues a remote driving planning instruction through the communication network, and at the moment, the automatic driving vehicle can timely receive the remote driving planning instruction through the vehicle-mounted communication unit. Therefore, whether a remote driving planning instruction is received or not can be determined by judging the received data of the communication unit, and then a corresponding dynamic driving task is executed according to the received remote driving planning instruction.

Step S406, continuing to execute the dynamic driving task in combination with the remote driving planning instruction, where the remote driving planning instruction is generated by the remote takeover console according to a response of the user of the remote takeover console to the intervention request after receiving the intervention request.

It should be noted that, the continuing of the dynamic driving task in conjunction with the remote driving planning instruction may be the automatic driving subsystem of the automatic driving vehicle integrating the remote driving planning instruction into a specific dynamic driving task, for example, when the remote driving planning instruction is deceleration, integrating the deceleration instruction into driving for deceleration, for example, when the remote driving planning instruction is roadside parking, integrating the parking instruction into driving, parking in a place where parking is possible, and the like.

And step S408, continuing to execute the dynamic driving task.

By receiving a remote driving planning instruction issued by a remote take-over console, the transmission data volume is small, the occupied bandwidth is small, and a remote take-over operator is not needed to directly control the vehicle, so that the influence of network delay on automatic driving is reduced, and the safety and the reliability are improved.

In some embodiments, the autonomous vehicle may provide a video stream of the sense of driving to the operator at the active request of the remote take-over operator.

In an implementation, when a remote take-over operator intervention autopilot task is received, i.e. when the remote driving planning instruction comprises a take-over intervention instruction, the autopilot method further comprises: and transmitting the real-time video data of the vehicle to the remote take-over server through the communication network according to a preset second sending strategy, so that the remote take-over server performs video fusion on the real-time video data of the vehicle and then transmits the video data to the remote take-over console.

The video stream is provided for the remote takeover operator, although a certain network bandwidth is occupied, the remote takeover operator can make secondary decision according to the real-time video data, the safety of automatic driving can be improved, and the video data is only transmitted when the takeover is accessed, so that the utilization rate of the network bandwidth can be improved.

In some embodiments, a 4G/5G communication unit may be installed in an autonomous vehicle, and thus a 4G/5G cellular mobile communication network with wider coverage, higher transmission bandwidth, and lower latency may be reused.

In implementation, the communication network comprises a 4G/5G cellular communication network, the communication unit comprises a 4G/5G cellular mobile communication gateway, and data transmission is realized through 4G/5G communication, so that the transmission reliability is improved, and the safety of automatic driving is improved.

Based on the same inventive concept, the embodiment of the specification further provides an automatic driving method which can be applied to a remote control console, and is convenient for a user of the remote control console to make secondary decisions on human driving behaviors.

As shown in fig. 11, the automatic driving method may include the steps of:

step S602, receiving an intervention request sent by a remote take-over server;

in implementation, the intervention request is an intervention request sent by the remote takeover server when the remote takeover server determines that the target automatic driving vehicle needs remote takeover in monitoring a dynamic driving task of the target automatic driving vehicle according to vehicle behavior data and high-precision road data;

step S604, the intervention request is displayed to a user of the remote takeover console;

step S606, obtaining response data of the user of the remote control console to the intervention request;

step S608, judging whether to take over intervention remotely; if yes, go to step S610; if not, go to step S612.

In implementation, whether the response data includes the remote driving planning instruction or not can be judged, if yes, the operator needs to take over the intervention remotely, and if not, the operator does not take over the intervention remotely temporarily.

And step S610, issuing the remote driving planning instruction to the target automatic driving vehicle so that the target automatic driving vehicle continues to execute the dynamic driving task after combining the received remote driving planning instruction.

And step S612, continuing to wait for the next intervention request.

It should be noted that step S612 is only schematically illustrated, and in practical applications, the step may be set according to actual needs, which is not limited herein.

By issuing a remote driving planning instruction to the automatic driving vehicle instead of directly controlling the vehicle by a remote takeover operator, the automatic driving planning instruction is convenient for the operator to simultaneously carry out remote takeover intervention on different vehicles, a plurality of vehicles and the like, for example, the automatic driving planning instruction smoothly transits from the takeover of one vehicle type to the takeover of another vehicle type, so that the safety of automatic driving can be improved, the adaptability of automatic driving in various scenes is improved, and the application and popularization of automatic driving are facilitated.

In some embodiments, the remote takeover console may provide a human-machine interaction interface to its user (i.e., the remote takeover operator), and may provide human-machine interaction content in a friendly interaction interface for the operator to respond to intervention requests sent by the remote takeover server.

It should be noted that, the content of the human-computer interaction may be set according to the actual application requirement, and is not limited here.

In some embodiments, the remote takeover operator can actively request real-time video data of the vehicle, so as to conveniently grasp corresponding environmental data in the real driving of the vehicle, facilitate secondary decision making, and improve the accuracy of the secondary decision making.

In an implementation, the remote takeover console may obtain the video stream according to the response data, that is, when the response data further includes a takeover intervention instruction, the automatic driving method further includes: and receiving vehicle real-time video data transmitted by the remote takeover server, wherein the vehicle real-time video data is the vehicle real-time video data transmitted by the target automatic driving vehicle to the remote takeover server.

In some embodiments, the remote takeover server may transmit relevant data to the remote takeover console that will assist the operator in making a secondary decision when it is initially determined that the vehicle requires remote takeover intervention.

In an implementation, the remote takeover console further receives related data provided by the remote takeover server, that is, when the response data further includes a takeover intervention instruction, the automatic driving method further includes: receiving monitoring data transmitted by the remote take-over server, wherein the monitoring data comprises at least one of the following data generated in the process that the remote take-over server monitors the dynamic driving task of the target automatic driving vehicle: real-time track, acceleration, yaw detection results, travelable region parameters, collision detection results, local path planning, and actuator deviation results.

After the remote takeover console obtains the data, the data can be displayed to an operator, so that the operator can conveniently make secondary decisions, remotely intervene in driving tasks and the like.

Based on the same inventive concept, the embodiment of the present specification further provides an automatic driving method, which can be applied to a communication network, wherein in automatic driving, the communication network serves as a data transmission bridge, and only a small amount of bandwidth resources are required to be provided for data transmission.

As shown in fig. 12, the automatic driving method may include:

step S802, vehicle behavior data reported by each autonomous driving vehicle is received, wherein the vehicle behavior data are generated by the autonomous driving vehicle in executing a dynamic driving task, and the vehicle behavior data comprise vehicle high-precision positioning data and vehicle actuator data;

step S804, the vehicle behavior data is transmitted to a remote take-over server, so that after the remote take-over server receives the vehicle behavior data, the dynamic driving task of the automatic driving vehicle is monitored according to the vehicle behavior data and the high-precision road data;

step S806, receiving a remote driving planning instruction issued by a remote take-over console to the autonomous vehicle, wherein the remote driving planning instruction is used for a user of the remote take-over console to remotely take over the autonomous vehicle to perform a dynamic driving task according to an intervention request, and the intervention request is sent by the remote take-over server when the remote take-over server determines that the autonomous vehicle needs to be remotely taken over in monitoring the dynamic driving task of the autonomous vehicle.

Only a small amount of data occupying a small amount of bandwidth resources need to be transmitted in the communication network, so that reliable transmission of the small amount of data in automatic driving is realized, the transmission requirement of the communication network is reduced, the adaptability of automatic driving in various scenes is improved, and the application and popularization of automatic driving are facilitated.

In some embodiments, the communication network comprises a 4G/5G cellular communication network. The data in automatic driving is transmitted through the 4G/5G cellular communication network, so that the characteristics of high bandwidth, low delay, wide coverage and the like of the communication network can be fully utilized, and the high safety of the automatic driving vehicle in executing dynamic driving tasks is guaranteed.

The embodiments in the present specification are described in a progressive manner, and the same and similar parts among the embodiments are referred to each other, and each embodiment focuses on differences from other embodiments. In particular, for the embodiment described later, since it is corresponding to the previous embodiment, the description is simple, and the relevant points can be referred to the partial description of the previous embodiment.

The above description is only for the specific embodiments of the present application, but the scope of the present application is not limited thereto, and any changes or substitutions that can be easily conceived by those skilled in the art within the technical scope of the present application should be covered within the scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims

1. An automatic driving method is applied to a remote takeover server, and comprises the following steps:

receiving vehicle behavior data reported by respective autonomous driving vehicles through a communication network, wherein the vehicle behavior data are generated by the autonomous driving vehicles in executing dynamic driving tasks and comprise vehicle high-precision positioning data and vehicle actuator data;

monitoring the dynamic driving tasks of all the automatically driven vehicles in the area by combining the vehicle behavior data reported by the automatically driven vehicles in the area covered by the communication network with high-precision road data, wherein the high-precision road data comprises high-precision map data and road surface data, and transmits an intervention request to a remote takeover console when it is determined in the monitoring that the autonomous vehicle needs remote takeover, so that a user of the remote take-over console makes a secondary decision on the intervention request, so that when the remote take-over console decides that take-over is required, so that the remote take-over console generates a remote take-over planning instruction and issues the automatic driving vehicle needing remote take-over, and causing the autonomous vehicle requiring remote takeover to incorporate the remote takeover planning instruction into autonomous driving to continue with subsequent dynamic driving tasks.

2. The automated driving method according to claim 1, further comprising:

receiving vehicle real-time video data transmitted by the automatic driving vehicle;

and carrying out video fusion on the vehicle real-time video data and then transmitting the video data to the remote take-over console.

3. The autopilot method of claim 1 wherein monitoring dynamic driving tasks of the autonomous vehicle based on the vehicle behavior data and high accuracy roadway data comprises:

predicting the driving track of the automatic driving vehicle according to a vehicle dynamic model corresponding to the automatic driving vehicle by combining the vehicle behavior data and the road surface data;

4. The autonomous driving method of claim 3, wherein predicting the trajectory of the autonomous vehicle based on a vehicle dynamics model corresponding to the autonomous vehicle in combination with the vehicle behavior data and the road surface data comprises:

5. The automated driving method according to claim 4, further comprising: and acquiring the real-time track and acceleration of the automatic driving vehicle according to the running track.

6. The autopilot method of claim 4 wherein monitoring the travel trajectory in accordance with the vehicle behavior data in combination with the high accuracy map data comprises:

and carrying out yaw detection on the driving track according to the vehicle behavior data so as to monitor the driving track.

7. The automated driving method according to claim 1, further comprising:

receiving logic semantic data sent by each edge computing gateway, wherein the logic semantic data is logic information used for restoring a dynamic traffic scene of a road section where the edge computing gateway is located, and the logic information comprises the following data used for representing each traffic participant in the road section: type data, high precision position data, and pose data.

8. The automated driving method according to claim 7, further comprising:

and combining the high-precision road data with the distance detection result to determine travelable area parameters, wherein the travelable area parameters comprise traffic signal sign data and safe distance data, and the safe distance data comprise safe distances from dynamic targets and static obstacles.

9. The autonomous driving method of claim 8, wherein monitoring dynamic driving tasks of the autonomous vehicle based on the vehicle behavior data and high accuracy road data comprises:

detecting the driving speed of the automatic driving vehicle according to the drivable region parameters, the vehicle behavior data and the high-precision road data;

and monitoring the dynamic driving task of the automatic driving vehicle according to the driving speed and the safe distance data.

10. The autonomous driving method of claim 9, wherein monitoring dynamic driving tasks of the autonomous vehicle based on the travel speed and the safe distance data comprises:

and according to the running speed and the safe distance data, performing collision detection on the automatic driving vehicle so as to monitor the dynamic driving task of the automatic driving vehicle.

11. The automated driving method according to claim 8, further comprising:

and according to the travelable region parameters and the vehicle high-precision positioning data in the vehicle behavior data, performing local path planning on the dynamic driving task of the automatic driving vehicle.

12. The automated driving method according to claim 11, further comprising: transmitting the local path plan to the remote take-over console.

13. The autopilot method of claim 1 wherein monitoring dynamic driving tasks of the autonomous vehicle based on the vehicle behavior data and high accuracy roadway data comprises:

inputting the vehicle behavior data and historical behavior data of the autonomous vehicle into a preset deep neural network, wherein the deep neural network is used for predicting vehicle actuator data in a dynamic driving task;

determining an actuator deviation result according to the vehicle actuator data predicted by the deep neural network and the actuator data reported by the automatic driving vehicle;

and monitoring the dynamic driving task of the automatic driving vehicle according to the deviation result of the actuator.

14. The autonomous driving method of any of claims 1-13, wherein determining in the monitoring that the autonomous vehicle requires remote take-over comprises: determining in the monitoring that the autonomous vehicle has at least one of the following anomalies: yaw exists in the real-time track; the real-time track exceeds the travelable area; the collision distance is smaller than a preset distance; the output deviation of the actuator exceeds a preset threshold value.

15. An automated driving method applied to an automated driving vehicle equipped with a communication unit, comprising:

in the process of executing a dynamic driving task, vehicle behavior data corresponding to the dynamic driving task is transmitted to a communication network through the communication unit according to a preset first sending strategy so as to transmit the vehicle behavior data to a remote take-over server, wherein the communication unit is a communication unit matched with the communication network, the vehicle behavior data comprises vehicle high-precision positioning data and vehicle actuator data, the remote take-over server is used for monitoring all dynamic driving tasks of automatic driving vehicles in a region by combining high-precision road data with the vehicle behavior data reported by each automatic driving vehicle in the coverage region of the communication network, and transmits an intervention request to the remote take-over console when the remote take-over console determines that the automatic driving vehicles need to take over remotely, enabling the remote takeover control console to generate a remote takeover planning instruction and send the remote takeover planning instruction to the automatic driving vehicle needing remote takeover;

when a remote driving planning instruction is received through the communication unit, the dynamic driving task is continuously executed in combination with the remote driving planning instruction, wherein the remote driving planning instruction is generated by the remote take-over console according to the response of the user of the remote take-over console to the intervention request after the intervention request is received.

16. The autopilot method of claim 15 wherein when the remote driving planning instruction comprises a take-over intervention instruction, the autopilot method further comprises:

and transmitting the vehicle real-time video data to the remote take-over server through the communication network according to a preset second sending strategy, so that the remote take-over server performs video fusion on the vehicle real-time video data and then transmits the video data to the remote take-over console.

17. The autopilot method of claim 15 wherein the communications network comprises a 4G/5G cellular communications network and the communications unit comprises a 4G/5G cellular mobile communications gateway.

18. An autopilot method for use with a remote take-over console, the autopilot method comprising:

receiving an intervention request sent by a remote takeover server, wherein the intervention request is sent by the remote takeover server when vehicle behavior data reported by respective automatically-driven vehicles in a communication network coverage area are combined with high-precision road data to monitor dynamic driving tasks of all target automatically-driven vehicles in the area, and the intervention request is sent when the target automatically-driven vehicles need to take over remotely, the vehicle behavior data is generated by the automatically-driven vehicles in executing the dynamic driving tasks, and the vehicle behavior data comprises vehicle high-precision positioning data and vehicle actuator data;

presenting the intervention request to a user of the remote take-over console;

19. The autopilot method of claim 18 wherein when the response data further includes an intervention take-over instruction, the autopilot method further comprises:

and receiving vehicle real-time video data transmitted by the remote takeover server, wherein the vehicle real-time video data is the vehicle real-time video data transmitted by the target automatic driving vehicle to the remote takeover server.

20. The automated driving method of claim 18, wherein when the response data further includes a take-over instruction, the automated driving method further comprises:

receiving monitoring data transmitted by the remote takeover server, wherein the monitoring data comprises at least one of the following data generated in the process that the remote takeover server monitors the dynamic driving task of the target autonomous vehicle: real-time track, acceleration, yaw detection results, travelable region parameters, collision detection results, local path planning, and actuator deviation results.

21. An automatic driving method, which is applied to a communication network, is characterized by comprising the following steps:

transmitting the vehicle behavior data to a remote takeover server, so that the remote takeover server monitors dynamic driving tasks of all automatic driving vehicles in the area by combining the vehicle behavior data reported by the automatic driving vehicles in the area with high-precision road data after receiving the vehicle behavior data reported by the automatic driving vehicles in the area covered by a communication network;

receiving a remote driving planning instruction issued by a remote take-over console to the automatic driving vehicle, wherein the remote driving planning instruction is used for a user of the remote take-over console to remotely take over the automatic driving vehicle to carry out a dynamic driving task according to an intervention request, and the intervention request is the intervention request sent by the remote take-over server when the automatic driving vehicle needs to be remotely taken over and is determined in the process of monitoring the dynamic driving task of the automatic driving vehicle.

22. The autopilot method of claim 21 wherein the communication network comprises a 4G/5G cellular communication network.

23. An autopilot system, comprising: the system comprises a plurality of automatic driving vehicles, a communication network, a remote take-over server and a remote take-over console, wherein the automatic driving vehicles are provided with communication units, are connected to the communication network through the communication units and are in communication connection with the remote take-over server and the remote take-over console respectively;

the remote takeover server is used for monitoring dynamic driving tasks of all the automatic driving vehicles in the area by combining high-precision road data with the vehicle behavior data reported by the respective automatic driving vehicles in the communication network coverage area, and transmitting an intervention request to the remote takeover console when the automatic driving vehicles need to take over remotely in the monitoring, wherein the high-precision road data comprises high-precision map data and road surface data;

the remote takeover console is used for displaying the intervention request to a user of the remote takeover console, acquiring response data of the user of the remote takeover console to the intervention request, and issuing a remote driving planning instruction to the automatic driving vehicle corresponding to the intervention request when the response data comprise the remote driving planning instruction;

24. The autopilot system of claim 23 further comprising a plurality of edge computing gateways deployed on a predetermined road segment for obtaining real-time traffic scene data for the road segment, generating logical semantic data from the real-time traffic scene data, and transmitting the logical semantic data to the remote takeover server, wherein the logical semantic data is logical information for restoring a dynamic traffic scene of the road segment, and the logical information includes the following data for characterizing each traffic participant in the road segment: type data, high precision position data, and pose data.

25. The autopilot system of claim 24 wherein the edge computing gateway includes a plurality of radar sensors for obtaining point cloud data corresponding to each traffic participant in a dynamic traffic scene of the road segment and a first computing subsystem for computing the logical semantic data from the point cloud data;

and/or the edge computing gateway comprises a plurality of visual sensors and a second computing subsystem, wherein the visual sensors are used for acquiring video streams corresponding to all traffic participants in the dynamic traffic scene of the road section, and the second computing subsystem is used for generating the logic semantic data according to the video stream computing.