WO2019047596A1

WO2019047596A1 - Method and device for switching driving modes

Info

Publication number: WO2019047596A1
Application number: PCT/CN2018/093357
Authority: WO
Inventors: 姜雨; 郁浩; 闫泳杉; 郑超; 唐坤; 张云飞
Original assignee: 百度在线网络技术（北京）有限公司
Priority date: 2017-09-05
Filing date: 2018-06-28
Publication date: 2019-03-14
Also published as: CN107564363A; CN107564363B

Abstract

The objective of the present invention is to provide a method and device for switching driving modes. Compared to existing technology, the present invention trains a deep learning decision model for switching between an end-to-end driving mode and a tracking driving mode by using safe scenario data and unsafe scenario data acquired by an onboard sensor of an automatic driving vehicle. In practice, the model may sense whether a current actual scenario is safe, make a decision, and output an instruction to switch between the two driving modes. The present invention utilizes the decision ability of deep learning to automatically switch between the tracking driving mode and the end-to-end driving mode, train the model having a reasoning decision-making ability, and naturally fuse the tracking driving mode and the end-to-end automatic driving mode so as to greatly improve the safety of automatic driving.

Description

Method and device for driving mode switching

This patent application claims Chinese patent filed on September 5, 2017, with the application number 201710792452.1, the applicant is Baidu Online Network Technology (Beijing) Co., Ltd., and the invention is entitled "A Method and Device for Driving Mode Switching" Priority of the application, the entire contents of which are incorporated herein by reference.

Technical field

The present invention relates to the field of vehicle automatic driving technology, and in particular, to a technology for driving mode switching.

Background technique

The two main modes of the existing automatic driving are the tracking driving mode and the end-to-end automatic driving mode. The tracking driving mode has high safety, but the trajectory must be preset, and the utility is not strong; the end-to-end automatic driving The mode is flexible and practical, but it is less secure. The existing combination of the two is either a simple combination, does not achieve the complementary effect, or automatically switches based on manual definition rules, the timing of switching is difficult to grasp correctly, and even counterproductive.

Therefore, how to make the autonomous vehicle switch accurately and efficiently in the two driving modes becomes one of the problems to be solved in the field.

Summary of the invention

It is an object of the present invention to provide a method and apparatus for driving mode switching.

According to an aspect of the present invention, a method for driving mode switching is provided, wherein the method comprises:

a acquiring sensor data collected by an onboard sensor of the self-driving vehicle, wherein the sensor data includes safety scene data and unsafe scene data of the self-driving vehicle;

b taking the sensor data as an input, corresponding end-to-end driving mode or tracking driving mode as an output, training mode switching model;

Wherein, the method further comprises:

x acquiring the current actual scene collected by the onboard sensor of the self-driving vehicle in real time;

y switching according to the current actual scenario, based on the mode switching model, in an end-to-end driving mode or a tracking driving mode.

Preferably, the unsafe scene data in the step a is collected by the driver intentionally performing an unsafe behavior.

Preferably, the step b comprises:

A convolutional neural network model is established, and the sensor data is taken as an input, and the corresponding end-to-end driving mode or the tracking driving mode is output as an output, and the mode switching model is trained.

More preferably, a negative feedback layer is added to each of the two convolutional layers of the convolutional neural network model, and a dropout layer is added to each of the three convolutional layers.

Preferably, the acquired sensor data collected by the onboard sensor is divided into a test set and a training set;

Wherein the step b comprises:

The training set is taken as an input, and the corresponding end-to-end driving mode or the tracking driving mode is output as an output, and the mode switching model is trained.

More preferably, the step b comprises:

Taking the training set as an input, and corresponding end-to-end driving mode or tracking driving mode as an output, training obtains a plurality of candidate switching models;

Wherein, the method further comprises:

Determining the mode switching model from the plurality of candidate switching models according to the test set.

Preferably, the method further comprises:

Recording the driver's takeover behavior of the self-driving vehicle during the automatic driving process, and acquiring sensor correction data collected by the onboard sensor at the corresponding time;

The mode switching model is corrected based on the sensor correction data.

According to another aspect of the present invention, there is also provided an apparatus for driving mode switching, wherein the apparatus comprises:

a collecting device, configured to acquire sensor data collected by an onboard sensor of the self-driving vehicle, wherein the sensor data includes safety scene data and unsafe scene data of the self-driving vehicle;

a training device, configured to use the sensor data as an input, and the corresponding end-to-end driving mode or the tracking driving mode as an output, and the training mode switching model;

Wherein, the device further comprises:

Obtaining means for acquiring the current actual scene collected by the onboard sensor of the self-driving vehicle in real time;

And a switching device, configured to switch in an end-to-end driving mode or a tracking driving mode based on the mode switching model according to the current actual scenario.

Preferably, the unsafe scene data is collected by the driver intentionally performing unsafe behavior.

Preferably, the training device is for:

Wherein the training device is used to:

More preferably, the training device is for:

Wherein, the device further comprises:

And a selecting means, configured to determine the mode switching model from the plurality of candidate switching models according to the test set.

Preferably, the apparatus further comprises correction means for:

The mode switching model is corrected based on the sensor correction data.

According to still another aspect of the present invention, there is also provided a computer readable storage medium storing computer code, the method of any of the foregoing being executed when the computer code is executed .

According to still another aspect of the present invention, there is also provided a computer program product, the method of any of the preceding, when the computer program product is executed by a computer device.

According to still another aspect of the present invention, a computer device is provided, the computer device comprising:

One or more processors;

a memory for storing one or more computer programs;

When the one or more computer programs are executed by the one or more processors, the one or more processors are caused to implement the method of any of the preceding.

Compared with the prior art, the present invention acquires sensor data collected by an on-board sensor of an autonomous vehicle, wherein the sensor data includes safety scene data and unsafe scene data of the self-driving vehicle; Input, a corresponding end-to-end driving mode or a tracking driving mode as an output, a training mode switching model; real-time acquisition of a current actual scene collected by the onboard sensor of the self-driving vehicle; and based on the current actual scenario, based on the mode Switching the model and switching in the end-to-end driving mode or the tracking driving mode; the invention uses the safety scene data and the unsafe scene data collected by the sensor to train one for the end-to-end driving mode and the tracking driving mode In the actual application, the model can sense whether the current actual scene is safe, make a decision, and output instructions to switch between the two driving modes. The invention utilizes the decision-making ability of deep learning to automatically switch the tracking driving mode and the end-to-end driving mode, and trains a model with inference decision-making ability, which naturally integrates the tracking driving mode and the end-to-end automatic driving mode, thereby greatly improving the automatic Driving safety.

Further, the present invention utilizes the characteristics of a small closed park area and strong operability, allowing the driver to intentionally simulate a large number of unsafe behaviors to create data, and training a deep learning decision model for end-to-end automatic driving and tracking switching.

Further, after training the model, the output command of the model is received at any time to switch the tracking driving mode and the end-to-end driving mode. In this process, by recording the driver's takeover behavior, indicating the wrong decision of the model, collecting the sensor at the corresponding moment. The data strengthens the training of the model, making the decision-making ability of the model better and better, and further improving the safety of autonomous driving.

DRAWINGS

Other features, objects, and advantages of the present invention will become more apparent from the Detailed Description of Description

1 shows a block diagram of an exemplary computer system/server 12 suitable for implementing embodiments of the present invention;

2 shows a flow diagram of a method for driving mode switching in accordance with an aspect of the present invention;

3 is a block diagram showing the structure of an apparatus for driving mode switching according to another aspect of the present invention.

The same or similar reference numerals in the drawings denote the same or similar components.

Detailed ways

Before discussing the exemplary embodiments in more detail, it should be noted that some exemplary embodiments are described as a process or method depicted as a flowchart. Although the flowcharts describe various operations as a sequential process, many of the operations can be implemented in parallel, concurrently or concurrently. In addition, the order of operations can be rearranged. The process may be terminated when its operation is completed, but may also have additional steps not included in the figures. The processing may correspond to methods, functions, procedures, subroutines, subroutines, and the like.

By "computer device", also referred to as "computer" in the context, is meant an intelligent electronic device that can perform predetermined processing, such as numerical calculations and/or logical calculations, by running a predetermined program or instruction, which can include a processor and The memory is executed by the processor to execute a predetermined process pre-stored in the memory to execute a predetermined process, or is executed by hardware such as an ASIC, an FPGA, a DSP, or the like, or a combination of the two. Computer devices include, but are not limited to, servers, personal computers, notebook computers, tablets, smart phones, and the like.

The computer device includes a user device and a network device. The user equipment includes, but is not limited to, a computer, a smart phone, a PDA, etc.; the network device includes but is not limited to a single network server, a server group composed of multiple network servers, or a cloud computing based computer Or a cloud composed of a network server, wherein cloud computing is a type of distributed computing, a super virtual computer composed of a group of loosely coupled computers. Wherein, the computer device can be operated separately to implement the present invention, and can also access the network and implement the present invention by interacting with other computer devices in the network. The network in which the computer device is located includes, but is not limited to, the Internet, a wide area network, a metropolitan area network, a local area network, a VPN network, and the like.

It should be noted that the user equipment, the network equipment, the network, and the like are merely examples, and other existing or future possible computer equipment or networks, such as those applicable to the present invention, are also included in the scope of the present invention. It is included here by reference.

The methods discussed below, some of which are illustrated by flowcharts, can be implemented in hardware, software, firmware, middleware, microcode, hardware description languages, or any combination thereof. When implemented in software, firmware, middleware or microcode, the program code or code segments to carry out the necessary tasks can be stored in a machine or computer readable medium, such as a storage medium. The processor(s) can perform the necessary tasks.

The specific structural and functional details disclosed are merely representative and are for the purpose of describing exemplary embodiments of the invention. The present invention may, however, be embodied in many alternative forms and should not be construed as being limited only to the embodiments set forth herein.

It should be understood that although the terms "first," "second," etc. may be used herein to describe the various elements, these elements should not be limited by these terms. These terms are used only to distinguish one unit from another. For example, a first unit could be termed a second unit, and similarly a second unit could be termed a first unit, without departing from the scope of the exemplary embodiments. The term "and/or" used herein includes any and all combinations of one or more of the associated listed items.

It will be understood that when a unit is referred to as "connected" or "coupled" to another unit, it can be directly connected or coupled to the other unit, or an intermediate unit can be present. In contrast, when a unit is referred to as being "directly connected" or "directly coupled" to another unit, there is no intermediate unit. Other words used to describe the relationship between the units should be interpreted in a similar manner (eg "between" and "directly between" and "adjacent to" Than "directly adjacent to", etc.).

The terminology used herein is for the purpose of describing the particular embodiments, The singular forms "a", "an", It is also to be understood that the terms "comprising" and """ Other features, integers, steps, operations, units, components, and/or combinations thereof.

It should also be noted that in some alternative implementations, the functions/acts noted may occur in a different order than that illustrated in the drawings. For example, two figures shown in succession may in fact be executed substantially concurrently or sometimes in the reverse order, depending on the function/acts involved.

The invention is further described in detail below with reference to the accompanying drawings.

FIG. 1 illustrates a block diagram of an exemplary computer system/server 12 suitable for use in implementing embodiments of the present invention. The computer system/server 12 shown in FIG. 1 is merely an example and should not impose any limitation on the function and scope of use of the embodiments of the present invention.

As shown in FIG. 1, computer system/server 12 is embodied in the form of a general purpose computing device. The components of computer system/server 12 may include, but are not limited to, one or more processors or processing units 16, system memory 28, and bus 18 that connects different system components, including system memory 28 and processing unit 16.

Bus 18 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, a graphics acceleration port, a processor, or a local bus using any of a variety of bus structures. For example, these architectures include, but are not limited to, an Industry Standard Architecture (ISA) bus, a Micro Channel Architecture (MAC) bus, an Enhanced ISA Bus, a Video Electronics Standards Association (VESA) local bus, and peripheral component interconnects ( PCI) bus.

Computer system/server 12 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by computer system/server 12, including both volatile and non-volatile media, removable and non-removable media.

Memory 28 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 30 and/or cache memory 32. Computer system/server 12 may further include other removable/non-removable, volatile/non-volatile computer system storage media. By way of example only, storage system 34 may be used to read and write non-removable, non-volatile magnetic media (not shown in FIG. 1, commonly referred to as "hard disk drives"). Although not shown in FIG. 1, a disk drive for reading and writing to a removable non-volatile disk (such as a "floppy disk"), and a removable non-volatile disk (such as a CD-ROM, DVD-ROM) may be provided. Or other optical media) read and write optical drive. In these cases, each drive can be coupled to bus 18 via one or more data medium interfaces. Memory 28 can include at least one program product having a set (e.g., at least one) of program modules configured to perform the functions of various embodiments of the present invention.

A program/utility 40 having a set (at least one) of program modules 42 may be stored, for example, in memory 28, such program modules 42 including, but not limited to, an operating system, one or more applications, other programs Modules and program data, each of these examples or some combination may include an implementation of a network environment. Program module 42 typically performs the functions and/or methods of the described embodiments of the present invention.

Computer system/server 12 may also be in communication with one or more external devices 14 (e.g., a keyboard, pointing device, display 24, etc.), and may also be in communication with one or more devices that enable a user to interact with the computer system/server 12. And/or in communication with any device (e.g., network card, modem, etc.) that enables the computer system/server 12 to communicate with one or more other computing devices. This communication can take place via an input/output (I/O) interface 22. Also, computer system/server 12 may also communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and/or a public network, such as the Internet) through network adapter 20. As shown, network adapter 20 communicates with other modules of computer system/server 12 via bus 18. It should be understood that although not shown in FIG. 1, other hardware and/or software modules may be utilized in conjunction with computer system/server 12, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems. , tape drives, and data backup storage systems.

Processing unit 16 executes various functional applications and data processing by running programs stored in memory 28.

For example, the memory 28 stores therein a computer program for performing the functions and processes of the present invention, and when the processing unit 16 executes the corresponding computer program, the identification of the incoming call intention at the network side by the present invention is implemented.

The specific functions/steps of the present invention for driving mode switching will be described in detail below.

2 shows a flow diagram of a method for driving mode switching in accordance with an aspect of the present invention.

In step S201, the device 1 acquires sensor data collected by an on-board sensor of the self-driving vehicle, wherein the sensor data includes safety scene data and unsafe scene data of the self-driving vehicle.

Specifically, the on-board sensor of the self-driving vehicle may collect corresponding different sensor data, such as video data, image data, radar data, and the like, of the self-driving vehicle during driving. Here, the sensor data may be collected by the self-driving vehicle during the automatic driving process, or may be collected by the self-driving vehicle during the driver's assisted driving process. Here, the sensor data may be safety scene data of the self-driving vehicle, that is, video data, image data, radar data, etc. collected by the self-driving vehicle during a normal automatic driving process or a driver assisted driving process. It may also be unsafe scene data of the self-driving vehicle, that is, an unsafe situation encountered by the self-driving vehicle during an automatic driving process or assisted driving by the driver, such as in the event of a collision, abnormal acceleration, abnormality Video data, image data, radar data, etc. collected by the on-board sensor in the case of deceleration, steering wheel killing, etc.

In step S201, the device 1 collects the above sensor data by an in-vehicle sensor that automatically drives the vehicle. Here, the on-board sensors of the self-driving vehicle include, but are not limited to, an in-vehicle camera, a vehicle-mounted radar, and the like. For example, when the self-driving vehicle is driving automatically, the device 1 collects video or image data of various angles of view through the in-vehicle camera of the self-driving vehicle. Here, the in-vehicle camera is used to simulate the driver's line of sight and angle of view, which can be, for example, located in the cab of the self-driving vehicle, the left side, the right side mirror, the center rear view mirror, etc., which can be, for example, binocular Camera. The video or image data collected by the in-vehicle camera can be regarded as, for example, various surrounding scenes that the driver sees when the autonomous vehicle is driven by the driver. Then, if the self-driving vehicle encounters an unsafe scene, such as sudden braking, the device 1 continues to collect video or image data of various viewing angles through the on-board camera of the self-driving vehicle, and saves the data as unsafe scene for subsequent Model training is used.

Here, the method is applicable to, for example, an end-to-end driving mode in automatic driving of a vehicle, and the end-to-end driving mode refers to an autonomous driving vehicle using an in-vehicle sensor, such as an in-vehicle camera, a vehicle-mounted radar, etc., to sense a surrounding scene to determine how to perform automatic driving. If it is judged whether it is stepping on the accelerator or stepping on the brakes, judging how to drive the steering wheel, etc., the degree of freedom of automatic driving of the vehicle is high; the opposite is the tracking driving mode, which is that the self-driving vehicle uses high-precision GPS to know. Its own position, along the preset trajectory for automatic driving, although relatively safe, but the driving trajectory is fixed, not so flexible.

Further, the above-mentioned end-to-end driving mode or the tracking driving mode is performed, for example, in a closed campus. Here, the closed campus refers to a limited scenario with a limited route and a limited physical area. In reality, a port such as a port is more common. , parking lots, fairs, campus interiors, etc. Of course, the closed park can also be customized.

It should be understood by those skilled in the art that the above-mentioned on-board sensors are merely examples, and other on-board sensors that may be present or may be present in the future, as applicable to the present invention, are also included in the scope of the present invention and are cited herein by way of reference. Included here.

Those skilled in the art should also understand that the above manner of collecting sensor data is only an example, and other existing or future possible methods of collecting sensor data, as applicable to the present invention, are also included in the scope of protection of the present invention. It is hereby incorporated by reference.

Specifically, the driver can assist in driving the self-driving vehicle. When the driver uses manual driving, he can deliberately simulate a large number of unsafe behaviors, such as intentional collision, deliberate sudden stepping on the accelerator, sudden sudden braking, steering wheel planning, etc. Therefore, the on-board sensor on the self-driving vehicle can collect corresponding sensor data when facing these unsafe scenes. For example, for a closed park, because it is safer in a closed park, the driver can deliberately simulate a large number of unsafe behaviors when driving the self-driving vehicle in the closed park.

Here, since the trained driver is difficult to generate effective unsafe scene data during the normal driving process, the closed park site can be used with small features and strong operability, so that the driver deliberately makes various types of insecurity. Behavior to collect corresponding sensor data.

For example, if the driver assists in driving the self-driving vehicle in the closed park and deliberately hits the roadside tree during the running of the vehicle, the on-board camera on the self-driving vehicle can collect the scene when the vehicle collides with the tree, such as Shooting a corresponding video in which the roadside tree is approaching and finally hitting the screen; or the onboard radar on the self-driving vehicle can also collect corresponding information, such as the vehicle radar measuring the tree as an obstacle The distance information with the vehicle; thus, in step S201, the device 1 collects these sensor data and can be used in the subsequent training of the model.

For another example, if the driver is rushing to brake when the autonomous vehicle is constantly approaching the tree at the roadside, the onboard camera on the self-driving vehicle can collect a corresponding scene, such as capturing a corresponding video, the video. The tree in the roadside first approaches and then decelerates to approach the last still picture; or the onboard radar on the self-driving vehicle can also collect corresponding information, such as the onboard radar measures the tree as an obstacle and the vehicle The distance information, if the distance is getting shorter, does not change at the end.

It should be understood by those skilled in the art that the above-mentioned intentional unsafe behavior of the driver is merely an example, and other unsafe behaviors that may exist in the future or in the future may be included in the scope of the present invention if applicable to the present invention. And is hereby incorporated by reference.

Here, the device 1 utilizes the characteristics of a small closed park area and high operability, allowing the driver to intentionally simulate a large number of unsafe behaviors to create data, and training a deep learning decision model for end-to-end automatic driving and tracking switching.

In step S202, the device 1 takes the sensor data as an input, and the corresponding end-to-end driving mode or the tracking driving mode as an output, and the training mode switches the model.

Specifically, in step S202, the device 1 takes the sensor data collected in step S201 as an input, and the sensor data has a corresponding driving mode. For example, the security scene data corresponds to the end-to-end driving mode, and the unsafe scene data corresponds to In the tracking driving mode, the end-to-end driving mode or the tracking driving mode is used as an output, and the training mode switches the model. For example, when the training mode switches the model, the input is sensor data, and the output is the driving mode number that the current self-driving vehicle should be activated, 0 is the end-to-end driving mode, and 1 is the tracking driving mode.

Here, the safety scene data is sensor data collected by the self-driving vehicle during normal driving, and since the degree of freedom of adopting the end-to-end driving mode is high, when the autonomous vehicle does not encounter an unsafe scene, The end-to-end driving mode is used for driving, that is, the safety scene data can correspond to the end-to-end driving mode; the unsafe scene data is the sensor data collected when the self-driving vehicle encounters an unsafe scene during driving. Since the tracking driving mode itself has a high degree of safety, the driving mode can be adopted when the vehicle encounters an unsafe scene, that is, the unsafe scene data can correspond to the tracking driving mode.

Here, the mode switching model may be, for example, a simple classification model, which can be trained by the existing training method for the classification model. For example, here, it is known that the self-driving vehicle is collected in a security scene. The sensor data is also the sensor data collected in an insecure scenario, and the sensor data collected in the known security scenario corresponds to the end-to-end driving mode. The sensor data collected in the unsafe scenario corresponds to According to the tracking driving mode, each classification output is obtained through each classification input. For example, here, the input is sensor data, and the output is the driving mode number that the current self-driving vehicle should be activated, and 0 is the end-to-end driving mode, 1 For the tracking driving mode, the mode switching model is trained; after the mode switching model training is completed, different sensor data can be classified, for example, when the device 1 collects video data through the vehicle camera, the scene analysis model is based on The video data is firstly security scene data or unsafe scene data. According to this driving mode is determined corresponding to-end tracking driving mode or driving mode.

Those skilled in the art should understand that the manner of the above training mode switching model is only an example, and other existing or future training mode switching models may be applicable to the present invention and should also be included in the protection scope of the present invention. And is hereby incorporated by reference.

Preferably, in step S202, the device 1 establishes a convolutional neural network model, taking the sensor data as an input, and corresponding end-to-end driving mode or tracking driving mode as an output, training the mode switching model.

Specifically, in step S202, the device 1 establishes a convolutional neural network model, taking the sensor data collected in step S201 as an input, and the corresponding driving mode as an output, for example, security scene data as an input, corresponding end-to-end driving The mode is used as the output, the unsafe scene data is taken as the input, and the corresponding tracking driving mode is used as the output, thereby training the mode switching model. For example, when the training mode switches the model, the output of the mode switching model is determined as the driving mode number that the autonomous driving vehicle should be currently activated, 0 is the end-to-end driving mode, and 1 is the tracking driving mode, and the safety scene data is used as When input, the output is 0, and when the unsafe scene data is used as an input, the output is 1.

More preferably, in step S202, the device 1 adds a negative feedback layer to each of the two convolutional layers of the convolutional neural network model, and a dropout layer is added to each of the three convolutional layers.

Specifically, in step S202, the device 1 establishes a convolutional neural network model when training the mode switching model, and adds a negative feedback layer to each two-layer convolutional layer of the convolutional neural network model, thereby enhancing the volume. The inference ability of the neural network model is added, and the dropout layer is added to each three-layer convolutional layer of the convolutional neural network model to enhance the generalization ability of the convolutional neural network model.

Preferably, the acquired sensor data collected by the onboard sensor is divided into a test set and a training set; wherein, in step S202, the device 1 takes the training set as an input, and the corresponding end-to-end driving mode or the tracking driving mode As an output, the mode switching model is trained.

Specifically, the sensor data collected by the on-board sensor can be divided into a test set and a training set, and the manner of the classification can be performed without any limitation, for example, only in terms of quantity; wherein the training set is used to train the mode switching model, The test set is used to test the trained mode switching model. For example, in step S202, the device 1 takes the training set as an input, and the training set also includes the security scene data and the unsafe scene data, and the device 1 inputs the security scene data and the unsafe scene data as corresponding inputs, and the corresponding end-to-end driving The mode or the tracking driving mode is used as an output to train the mode switching model.

More preferably, in step S202, the device 1 takes the training set as an input, and the corresponding end-to-end driving mode or the tracking driving mode as an output, and the training obtains a plurality of candidate switching models; wherein the method further includes step S205 (not shown), in step S205, the device 1 selects the mode switching model from the plurality of candidate switching models according to the test set.

Specifically, in step S202, the device 1 takes as input the training set of the sensor data collected by the onboard sensor, and the corresponding end-to-end driving mode or the tracking driving mode as an output, and can train to obtain a plurality of candidate switching models, for example, As time progresses, the model is continuously trained, and a plurality of candidate switching models can be obtained during the process; subsequently, in step S205, the device 1 tests the plurality of candidate switching models according to the test set of sensor data collected by the onboard sensors. For example, the test set also includes security scene data and unsafe scene data, and inputs the security scene data in the test set to the candidate handover model, checks whether the output is an end-to-end driving mode, and tests the insecure scene data in the set. The candidate switching model is input to verify whether the output is a tracking driving mode, thereby selecting a final mode switching model from the plurality of candidate switching models.

It should be understood by those skilled in the art that the manner of determining the mode switching model is merely an example, and other existing or future possible methods for determining the mode switching model, as applicable to the present invention, should also be included in the scope of the present invention. And is hereby incorporated by reference.

Wherein, the method further includes step S203 and step S204.

In step S203, the device 1 acquires the current actual scene collected by the onboard sensor of the self-driving vehicle in real time.

Specifically, the foregoing steps S201 and S202 are trainings on the mode switching model, and belong to the preliminary work. After the mode switching model training is completed, the self-driving vehicle can apply the mode switching model in the actual automatic driving process, thereby determining that Autopilot in end-to-end driving mode or autopilot in tracking driving mode.

The self-driving vehicle can collect data in real time during the actual automatic driving process, for example, an on-board camera located in the cab of the self-driving vehicle, the left side, the right side mirror, the center rear view mirror, and the like. In the actual automatic driving process of the self-driving vehicle, the shooting is continuously performed, and the corresponding video or image data is captured and acquired. In step S203, the device 1 acquires real-time data collected by the on-board sensor through interaction with the on-board sensor of the self-driving vehicle during the automatic driving of the vehicle, for example, the current actual situation of the self-driving vehicle. The scene is input to the mode switching model in real time, and the output of the model is switched according to the mode to determine which driving mode should be used for driving.

In step S204, the device 1 switches in the end-to-end driving mode or the tracking driving mode based on the mode switching model according to the current actual scene.

Specifically, in step S204, the device 1 inputs the current actual scene to the mode switching model according to the current actual scene in which the self-driving vehicle is acquired in step S203, and switches the driving output according to the mode switching model. Mode, switch between end-to-end driving mode or tracking driving mode, for example, suppose the output of the mode switching model is the driving mode number that should be enabled for the current driving vehicle, 0 is the end-to-end driving mode, and 1 is the tracking mode. In the driving mode, according to the current actual scene input to the mode switching model, the mode switching model can output a corresponding driving mode number, according to the driving mode number, the device 1 can know which driving mode the autonomous driving vehicle should currently adopt. Perform automatic driving.

For example, the self-driving vehicle is originally performing normal end-to-end automatic driving, and the on-board camera continuously collects video or image data in real time, and in step S203, the device 1 also continuously acquires the real-time acquisition from the on-board camera. Video or image data, and input to the mode switching model in real time, the output of the mode switching model is an end-to-end driving mode, and the device 1 does not need to switch the driving mode of the self-driving vehicle; thereafter, the autonomous vehicle encounters a certain An unsafe scene, for example, the self-driving vehicle is about to hit a tree on the side of the road, and the onboard camera thereon still continuously collects video or image data in real time, and in step S203, the device 1 also continuously acquires the real time from the onboard camera. The captured video or image data, that is, the current actual scene of the self-driving vehicle is acquired in real time, and input to the mode switching model in real time, and at this time, the output of the mode switching model is the tracking driving mode, then in step S204 The device 1 switches the driving mode of the self-driving vehicle to the tracking driving mode.

Here, since the on-board sensor continuously collects real-time data, the device 1 also continuously acquires the real-time data and inputs it to the mode switching model for determination. Therefore, the device 1 can switch the model once the driving of the mode is output. When the mode changes, the driving mode of the self-driving vehicle is switched; and the collected real-time data can also be used for judging, for example, for a certain amount of real-time sensor data, if the driving mode of the mode switching model is changed When the predetermined threshold is exceeded, the driving mode of the self-driving vehicle is switched to avoid erroneous judgment that a small amount of real-time data may occur. Therefore, some real-time data may be taken to make a judgment to increase the accuracy of the judgment.

It should be understood by those skilled in the art that the manner of switching the driving mode of the self-driving vehicle is merely an example, and other existing or future possible modes of switching the driving mode of the self-driving vehicle, as applicable to the present invention, should also include It is within the scope of the invention and is hereby incorporated by reference.

Here, the device 1 acquires sensor data collected by an on-board sensor of the self-driving vehicle, wherein the sensor data includes safety scene data and unsafe scene data of the self-driving vehicle; and the sensor data is input as an input An end-to-end driving mode or a tracking driving mode as an output, a training mode switching model; real-time acquisition of a current actual scene collected by the onboard sensor of the self-driving vehicle; and according to the current actual scenario, based on the mode switching model, Switching between end-to-end driving mode or tracking driving mode; when the self-driving vehicle senses an unsafe scene, it automatically switches to the tracking driving mode, and if it is a safety scene, it automatically switches to the end-to-end driving mode. The device 1 uses the security scene data and the unsafe scene data collected by the sensor to train a deep learning decision model for switching between the end-to-end driving mode and the tracking driving mode. In practical applications, the model can perceive Whether the current actual scene is safe, makes a decision, and the output command switches between the two driving modes. The invention utilizes the decision-making ability of deep learning to automatically switch the tracking driving mode and the end-to-end driving mode, and trains a model with inference decision-making ability, which naturally integrates the tracking driving mode and the end-to-end automatic driving mode, thereby greatly improving the automatic Driving safety.

Preferably, the method further comprises a step S206 (not shown). In step S206, the device 1 records the driver's takeover behavior of the self-driving vehicle during the automatic driving process, acquires sensor correction data collected by the onboard sensor at the corresponding time; and switches the mode according to the sensor correction data. The model is revised.

Specifically, in the automatic driving process of the self-driving vehicle, the driver can also sit in the assisting driving, and when the autonomous driving vehicle has an inappropriate driving behavior, the driver can perform manual intervention to correct it in time. In step S206, the device 1 may record the driver's take-off behavior during the automatic driving of the self-driving vehicle, and since the onboard sensor of the self-driving vehicle continuously collects sensor data, in step S206, the device 1 It is also possible to acquire sensor data collected by the onboard sensor at the moment when the driver takes over the manual driving of the self-driving vehicle. Here, for convenience of description, it is referred to as sensor correction data, and the actual is also automatic driving. Information such as video, image or radar data collected by the vehicle's onboard sensors. Subsequently, the device 1 can correct the mode switching model based on the sensor correction data.

For example, for a certain sensor data, such as video data that is close to a certain subject, when training the mode switching model, it is summarized into a corresponding end-to-end driving mode, and therefore, the actual driving in the autonomous driving vehicle In the process, it is assumed that the on-board sensor of the self-driving vehicle also collects video data of a certain object that is approaching, and in step S203, the device 1 acquires the video data as the current actual scene of the self-driving vehicle; In step S204, the device 1 inputs the current actual scene to the mode switching model, and the obtained driving mode is an end-to-end driving mode. Therefore, the self-driving vehicle performs automatic driving in the end-to-end driving mode; The player finds that the current actual scene is actually an unsafe scene. Therefore, the driver takes over the self-driving vehicle and performs manual driving, such as taking over the steering wheel, brake or shifting handle of the self-driving vehicle, in step S206. The device 1 records the driver's takeover behavior and acquires the transmission collected by the onboard sensor at this time. Correction data, when the vehicle camera captures that the subject is no longer close, but the conversion angle is far away, therefore, the device 1 can determine that the current actual scene is an unsafe scene, and correct the mode switching model to be input. In the current actual scene, the output is the tracking driving mode, and thereafter, if the self-driving vehicle still touches the scene, it can switch to the tracking driving mode for automatic driving.

Here, after training the model, the device 1 receives the output command of the model at any time to switch the tracking driving mode and the end-to-end driving mode. In this process, by recording the driver's takeover behavior, indicating the wrong decision of the model, collecting the corresponding moment. The sensor data strengthens the training of the model, making the decision-making ability of the model better and better, and further improving the safety of autonomous driving.

The device 1 includes an acquisition device 301, a training device 302, an acquisition device 303, and a switching device 304. The device 1 is, for example, located in a computer device, for example in an autonomous vehicle, or a network device connected to the self-driving vehicle via a network. Further, the device 1 can be partially located in the network device, part of the device The device is located in an autonomous vehicle, for example, the aforementioned acquisition device 301 and training device 302 are located in a network device, and the aforementioned acquisition device 303 and switching device 304 are located in an autonomous vehicle. It should be understood by those skilled in the art that the location of the above device is merely an example, and other existing or future possible devices may be included in the scope of the present invention, and may be included in the scope of the present invention. It is included here by reference.

The collection device 301 acquires sensor data collected by an on-board sensor of the self-driving vehicle, wherein the sensor data includes safety scene data and unsafe scene data of the self-driving vehicle.

The acquisition device 301 collects the above sensor data by an onboard sensor of the self-driving vehicle. Here, the on-board sensors of the self-driving vehicle include, but are not limited to, an in-vehicle camera, a vehicle-mounted radar, and the like. For example, when the self-driving vehicle is driving automatically, the collecting device 301 collects video or image data of respective viewing angles through the in-vehicle camera of the self-driving vehicle. Here, the in-vehicle camera is used to simulate the driver's line of sight and angle of view, which can be, for example, located in the cab of the self-driving vehicle, the left side, the right side mirror, the center rear view mirror, etc., which can be, for example, binocular Camera. The video or image data collected by the in-vehicle camera can be regarded as, for example, various surrounding scenes that the driver sees when the autonomous vehicle is driven by the driver. Then, if the self-driving vehicle encounters an unsafe scene, such as sudden braking, the collecting device 301 continues to collect video or image data of various viewing angles through the in-vehicle camera of the self-driving vehicle, and saves the data as unsafe scenes for Subsequent model training is used.

Here, the method is applicable to, for example, an end-to-end driving mode in automatic driving of a vehicle, and the end-to-end driving mode refers to an autonomous driving vehicle using an in-vehicle sensor, such as an in-vehicle camera, a vehicle-mounted radar, etc., to sense a surrounding scene to determine how to perform automatic driving. If it is judged whether it is stepping on the accelerator or stepping on the brakes, judging how to drive the steering wheel, etc., the degree of freedom of automatic driving of the vehicle is high; the opposite is the tracking driving mode, which is that the self-driving vehicle uses high-precision GPS to know. Its own position, along the preset trajectory for automatic driving, although relatively safe, but the trajectory is fixed, not so flexible.

For example, if the driver assists in driving the self-driving vehicle in the closed park and deliberately hits the roadside tree during the running of the vehicle, the on-board camera on the self-driving vehicle can collect the scene when the vehicle collides with the tree, such as Shooting a corresponding video in which the roadside tree is approaching and finally hitting the screen; or the onboard radar on the self-driving vehicle can also collect corresponding information, such as the vehicle radar measuring the tree as an obstacle Distance information with the vehicle; thus, the acquisition device 301 collects these sensor data and can be used during subsequent training of the model.

The training device 302 takes the sensor data as an input, a corresponding end-to-end driving mode or a tracking driving mode as an output, and a training mode switching model.

Specifically, the training device 302 takes the sensor data collected by the collection device 301 as an input, and the sensor data has a corresponding driving mode. For example, the security scene data corresponds to the end-to-end driving mode, and the unsafe scene data corresponds to the tracking driving mode. The end-to-end driving mode or the tracking driving mode is used as an output, and the training mode switches the model. For example, when the training mode switches the model, the input is sensor data, and the output is the driving mode number that the current self-driving vehicle should be activated, 0 is the end-to-end driving mode, and 1 is the tracking driving mode.

Here, the mode switching model may be, for example, a simple classification model, which can be trained by the existing training method for the classification model. For example, here, it is known that the self-driving vehicle is collected in a security scene. The sensor data is also the sensor data collected in an insecure scenario, and the sensor data collected in the known security scenario corresponds to the end-to-end driving mode. The sensor data collected in the unsafe scenario corresponds to According to the tracking driving mode, each classification output is obtained through each classification input. For example, here, the input is sensor data, and the output is the driving mode number that the current self-driving vehicle should be activated, and 0 is the end-to-end driving mode, 1 The tracking mode is trained to train the mode switching model; after the mode switching model training is completed, different sensor data can be classified, such as when the collecting device 301 collects video data through the vehicle camera, the scene analysis model According to the video data, whether it is security scene data or unsafe scene first It is, and accordingly determine the corresponding driving mode is the driving mode or end to end tracking driving mode.

Preferably, the training device 302 establishes a convolutional neural network model with the sensor data as an input, the corresponding end-to-end driving mode or the tracking driving mode as an output, training the mode switching model.

Specifically, the training device 302 establishes a convolutional neural network model, and takes the sensor data collected by the acquisition device 301 as an input, and the corresponding driving mode as an output, for example, the security scene data as an input, and the corresponding end-to-end driving mode as an output. The unsafe scene data is taken as an input, and the corresponding tracking driving mode is used as an output, thereby training the mode switching model. For example, when the training mode switches the model, the output of the mode switching model is determined as the driving mode number that the autonomous driving vehicle should be currently activated, 0 is the end-to-end driving mode, and 1 is the tracking driving mode, and the safety scene data is used as When input, the output is 0, and when the unsafe scene data is used as an input, the output is 1.

More preferably, the training device 302 adds a negative feedback layer to each of the two convolutional layers of the convolutional neural network model, and a dropout layer is added to each of the three convolutional layers.

Specifically, the training device 302 establishes a convolutional neural network model when training the mode switching model, and adds a negative feedback layer to each two-layer convolutional layer of the convolutional neural network model, thereby enhancing the convolutional neural network model. The reasoning ability, and the dropout layer is added to each three-layer convolutional layer of the convolutional neural network model, thereby enhancing the generalization ability of the convolutional neural network model.

Preferably, the acquired sensor data collected by the on-vehicle sensor is divided into a test set and a training set; wherein the training device 302 takes the training set as an input, and the corresponding end-to-end driving mode or the tracking driving mode as an output, training The mode switches the model.

Specifically, the sensor data collected by the on-board sensor can be divided into a test set and a training set, and the manner of the classification can be performed without any limitation, for example, only in terms of quantity; wherein the training set is used to train the mode switching model, The test set is used to test the trained mode switching model. For example, the training device 302 takes the training set as an input, and the training set also includes the security scene data and the unsafe scene data. The training device 302 takes the security scene data and the unsafe scene data as inputs, and the corresponding end-to-end driving mode or The track driving mode is used as an output to train the mode switching model.

More preferably, the training device 302 takes the training set as an input, and the corresponding end-to-end driving mode or the tracking driving mode as an output, and the training obtains a plurality of candidate switching models; wherein the device 1 further includes a selecting device (not shown) And the selecting means selects and determines the mode switching model from the plurality of candidate switching models according to the test set.

Specifically, the training device 302 takes as input the training set of the sensor data collected by the onboard sensor, and the corresponding end-to-end driving mode or the tracking driving mode as an output, and can train to obtain a plurality of candidate switching models, for example, over time. Advancing, the model is continuously trained, and multiple candidate switching models can be obtained during the process; then, the selecting device tests the plurality of candidate switching models according to the test set of the sensor data collected by the onboard sensors, for example, the test set also includes Security scene data and unsafe scene data, input the security scene data in the test set to the candidate switching model, verify whether the output is an end-to-end driving mode, input the unsafe scene data in the test set to the candidate switching model, and verify the output. Whether it is a tracking driving mode, thereby selecting and determining a final mode switching model from the plurality of candidate switching models.

The device 1 further includes an obtaining device 303 and a switching device 304.

The obtaining device 303 acquires the current actual scene collected by the onboard sensor of the self-driving vehicle in real time.

Specifically, the foregoing collecting device 301 and the training device 302 are trainings for the mode switching model, and belong to the preliminary work, and after the mode switching model training is completed, the automatic driving vehicle can apply the mode switching model during the actual automatic driving process. It is thus judged whether the automatic driving is performed in the end-to-end driving mode or the automatic driving in the tracking driving mode.

The self-driving vehicle can collect data in real time during the actual automatic driving process, for example, an on-board camera located in the cab of the self-driving vehicle, the left side, the right side mirror, the center rear view mirror, and the like. In the actual automatic driving process of the self-driving vehicle, the shooting is continuously performed, and the corresponding video or image data is captured and acquired. The obtaining device 303 acquires real-time data collected by the on-board sensor through interaction with the on-vehicle sensor of the self-driving vehicle during the automatic driving of the vehicle, for example, the current actual scene in which the self-driving vehicle is located, and The real-time data is input to the mode switching model in real time, and the output of the model is switched according to the mode to determine which driving mode should be driven.

The switching device 304 switches between the end-to-end driving mode or the tracking driving mode based on the current mode, based on the mode switching model.

Specifically, the switching device 304 inputs the current actual scene to the mode switching model according to the current actual scene where the automatically driving vehicle is acquired by the acquiring device 303, and switches the driving mode output by the model according to the mode. Switching between the end driving mode or the tracking driving mode, for example, assuming that the output of the mode switching model is the driving mode number that the current self-driving vehicle should be activated, 0 is the end-to-end driving mode, and 1 is the tracking driving mode, according to Inputting to the current actual scene of the mode switching model, the mode switching model may output a corresponding driving mode number, and according to the driving mode number, the switching device 304 may know which driving mode the autonomous driving vehicle should currently use for automatic driving.

For example, the self-driving vehicle is originally undergoing normal end-to-end automatic driving, the onboard camera thereon continuously collects video or image data in real time, and the acquisition device 303 continuously acquires the real-time captured video or image from the onboard camera. Data, and input to the mode switching model in real time, the output of the mode switching model is an end-to-end driving mode, and the switching device 304 does not need to switch the driving mode of the self-driving vehicle; thereafter, the autonomous driving vehicle encounters an unsafe scene For example, the self-driving vehicle is about to hit a tree on the side of the road, and the on-board camera thereon still continuously collects video or image data in real time, and the acquisition device 303 also continuously acquires the real-time collected video or image data from the on-board camera. That is, the current actual scene of the self-driving vehicle is acquired in real time and input to the mode switching model in real time, and at this time, the output of the mode switching model is the tracking driving mode, and the switching device 304 drives the self-driving vehicle. The mode is switched to the tracking driving mode.

Here, since the on-board sensor continuously collects real-time data, the acquisition device 303 also continuously acquires the real-time data and inputs it to the mode switching model for determination. Therefore, the switching device 304 can switch the model in the mode once output. When the driving mode changes, the driving mode of the self-driving vehicle is switched; and the collected real-time data can also be used for judging, for example, for a certain amount of real-time sensor data, if the driving mode of the mode switching model is changed If the number of times exceeds a predetermined threshold, the driving mode of the self-driving vehicle is switched to avoid a erroneous judgment that a small amount of real-time data may occur. Therefore, some real-time data may be taken to make a judgment to increase the accuracy of the judgment.

Preferably, the device 1 further comprises a correction device (not shown). The correction device records the driver's takeover behavior of the self-driving vehicle during the automatic driving process, acquires sensor correction data collected by the onboard sensor at the corresponding time; and corrects the mode switching model according to the sensor correction data.

Specifically, in the automatic driving process of the self-driving vehicle, the driver can also sit in the assisting driving, and when the autonomous driving vehicle has an inappropriate driving behavior, the driver can perform manual intervention to correct it and correct the correction. The device may record the driver's takeover behavior during the automatic driving of the self-driving vehicle, and since the onboard sensor of the self-driving vehicle continuously collects sensor data, the correction device may also acquire the manual driving when the driver takes over In the case of the self-driving vehicle, the sensor data collected by the on-vehicle sensor at this time is referred to herein as sensor correction data for convenience of description, and the actual is also the video collected by the on-board sensor of the self-driving vehicle. , image or radar data, etc. Subsequently, the correction device can correct the mode switching model based on the sensor correction data.

For example, for a certain sensor data, such as video data that is close to a certain subject, when training the mode switching model, it is summarized into a corresponding end-to-end driving mode, and therefore, the actual driving in the autonomous driving vehicle In the process, it is assumed that the on-board sensor of the self-driving vehicle also collects video data of a certain object that is approaching, and the acquisition device 303 acquires the video data as the current actual scene of the self-driving vehicle; subsequently, the switching device 304 will The current actual scene is input to the mode switching model, and the obtained driving mode is an end-to-end driving mode. Therefore, the self-driving vehicle performs automatic driving in the end-to-end driving mode; at this time, the driver finds that the current actual scene actually Is an unsafe scene, therefore, the driver takes over the self-driving vehicle and performs manual driving. If the driver takes over the steering wheel, brake or shifting handle of the self-driving vehicle, the correcting device records the driver's take-over behavior and obtains At this time, the sensor correction data collected by the on-board sensor, so the vehicle The image capturing head is no longer close to the subject but the conversion angle is far away. Therefore, the correcting device can determine that the current actual scene is an unsafe scene, and correct the mode switching model to be when the input is the current actual scene. The output is in the tracking driving mode. After that, if the self-driving vehicle still touches the scene, it can switch to the tracking driving mode for automatic driving.

The present invention also provides a computer readable storage medium storing computer code, the method of any of which is performed when the computer code is executed.

The invention also provides a computer program product, the method of any of the preceding one being performed when the computer program product is executed by a computer device.

The invention also provides a computer device, the computer device comprising:

One or more processors;

a memory for storing one or more computer programs;

It should be noted that the present invention can be implemented in software and/or a combination of software and hardware. For example, the various devices of the present invention can be implemented using an application specific integrated circuit (ASIC) or any other similar hardware device. In one embodiment, the software program of the present invention may be executed by a processor to implement the steps or functions described above. Likewise, the software program (including related data structures) of the present invention can be stored in a computer readable recording medium such as a RAM memory, a magnetic or optical drive or a floppy disk and the like. Additionally, some of the steps or functions of the present invention may be implemented in hardware, for example, as a circuit that cooperates with a processor to perform various steps or functions.

It is apparent to those skilled in the art that the present invention is not limited to the details of the above-described exemplary embodiments, and the present invention can be embodied in other specific forms without departing from the spirit or essential characteristics of the invention. Therefore, the present embodiments are to be considered as illustrative and not restrictive, and the scope of the invention is defined by the appended claims instead All changes in the meaning and scope of equivalent elements are included in the present invention. Any reference signs in the claims should not be construed as limiting the claim. In addition, it is to be understood that the word "comprising" does not exclude other elements or steps. A plurality of units or devices recited in the system claims can also be implemented by a unit or device by software or hardware. The first, second, etc. words are used to denote names and do not denote any particular order.

Claims

A method for driving mode switching, wherein the method comprises:

a acquiring sensor data collected by an onboard sensor of the self-driving vehicle, wherein the sensor data includes safety scene data and unsafe scene data of the self-driving vehicle;

b taking the sensor data as an input, corresponding end-to-end driving mode or tracking driving mode as an output, training mode switching model;

Wherein, the method further comprises:

x acquiring the current actual scene collected by the onboard sensor of the self-driving vehicle in real time;

y switching according to the current actual scenario, based on the mode switching model, in an end-to-end driving mode or a tracking driving mode.
The method of claim 1 wherein said step b comprises:

A convolutional neural network model is established, and the sensor data is taken as an input, and the corresponding end-to-end driving mode or the tracking driving mode is output as an output, and the mode switching model is trained.
The method of claim 2, wherein a negative feedback layer is added to each of the two convolutional layers of the convolutional neural network model, and a dropout layer is added to each of the three convolutional layers.
The method according to any one of claims 1 to 3, wherein the acquired sensor data collected by the on-vehicle sensor is divided into a test set and a training set;

Wherein the step b comprises:

The training set is taken as an input, and the corresponding end-to-end driving mode or the tracking driving mode is output as an output, and the mode switching model is trained.
The method of claim 4 wherein said step b comprises:

Taking the training set as an input, and corresponding end-to-end driving mode or tracking driving mode as an output, training obtains a plurality of candidate switching models;

Wherein, the method further comprises:

Determining the mode switching model from the plurality of candidate switching models based on the test set.
The method according to any one of claims 1 to 5, wherein the method further comprises:

Recording the driver's takeover behavior of the self-driving vehicle during the automatic driving process, and acquiring sensor correction data collected by the onboard sensor at the corresponding time;

The mode switching model is corrected based on the sensor correction data.
A device for driving mode switching, wherein the device comprises:

a collecting device, configured to acquire sensor data collected by an onboard sensor of the self-driving vehicle, wherein the sensor data includes safety scene data and unsafe scene data of the self-driving vehicle;

a training device, configured to use the sensor data as an input, and the corresponding end-to-end driving mode or the tracking driving mode as an output, and the training mode switching model;

Wherein, the device further comprises:

Obtaining means for acquiring the current actual scene collected by the onboard sensor of the self-driving vehicle in real time;

And a switching device, configured to switch in an end-to-end driving mode or a tracking driving mode based on the mode switching model according to the current actual scenario.
The device of claim 7 wherein said training device is for:

A convolutional neural network model is established, and the sensor data is taken as an input, and the corresponding end-to-end driving mode or the tracking driving mode is output as an output, and the mode switching model is trained.
The apparatus according to claim 8, wherein a negative feedback layer is added to each of the two convolutional layers of the convolutional neural network model, and a dropout layer is added to each of the three convolutional layers.
The apparatus according to any one of claims 7 to 9, wherein the acquired sensor data collected by the on-vehicle sensor is divided into a test set and a training set;

Wherein the training device is used to:

The training set is taken as an input, and the corresponding end-to-end driving mode or the tracking driving mode is output as an output, and the mode switching model is trained.
The device of claim 10 wherein said training device is for:

Taking the training set as an input, and corresponding end-to-end driving mode or tracking driving mode as an output, training obtains a plurality of candidate switching models;

Wherein, the device further comprises:

And a selecting means, configured to determine the mode switching model from the plurality of candidate switching models according to the test set.
A device according to any one of claims 7 to 11, wherein the device further comprises correction means for:

Recording the driver's takeover behavior of the self-driving vehicle during the automatic driving process, and acquiring sensor correction data collected by the onboard sensor at the corresponding time;

The mode switching model is corrected based on the sensor correction data.
A computer readable storage medium storing computer code, the method of any one of claims 1 to 6 being executed when the computer code is executed.
A computer program product, when the computer program product is executed by a computer device, the method of any one of claims 1 to 6 being performed.
A computer device, the computer device comprising:

One or more processors;

a memory for storing one or more computer programs;

When the one or more computer programs are executed by the one or more processors, the one or more processors are caused to implement the method of any one of claims 1 to 6.