WO2023045936A1 - 一种模型自动化迭代方法、设备及存储介质 - Google Patents

一种模型自动化迭代方法、设备及存储介质 Download PDF

Info

Publication number
WO2023045936A1
WO2023045936A1 PCT/CN2022/120034 CN2022120034W WO2023045936A1 WO 2023045936 A1 WO2023045936 A1 WO 2023045936A1 CN 2022120034 W CN2022120034 W CN 2022120034W WO 2023045936 A1 WO2023045936 A1 WO 2023045936A1
Authority
WO
WIPO (PCT)
Prior art keywords
vehicle
model
car
data
side model
Prior art date
Application number
PCT/CN2022/120034
Other languages
English (en)
French (fr)
Inventor
张放
徐成
赵勍
刘涛
夏洋
李晓飞
王肖
张德兆
霍舒豪
Original Assignee
北京智行者科技股份有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 北京智行者科技股份有限公司 filed Critical 北京智行者科技股份有限公司
Publication of WO2023045936A1 publication Critical patent/WO2023045936A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F30/00—Computer-aided design [CAD]
    • G06F30/20—Design optimisation, verification or simulation
    • G06F30/27—Design optimisation, verification or simulation using machine learning, e.g. artificial intelligence, neural networks, support vector machines [SVM] or training a model
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F30/00—Computer-aided design [CAD]
    • G06F30/10—Geometric CAD
    • G06F30/15—Vehicle, aircraft or watercraft design
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N5/00—Computing arrangements using knowledge-based models
    • G06N5/04—Inference or reasoning models
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2111/00—Details relating to CAD techniques
    • G06F2111/02—CAD in a network environment, e.g. collaborative CAD or distributed simulation
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
    • Y02T10/00—Road transport of goods or passengers
    • Y02T10/10—Internal combustion engine [ICE] based vehicles
    • Y02T10/40—Engine management systems

Definitions

  • the present application relates to the technical field of automatic driving, and in particular to a model automation iteration method, device and storage medium.
  • Autonomous driving long-tail scenarios refer to sudden, low-probability, and unpredictable scenarios, such as intersections with traffic signal failures, drunk-driving vehicles, and balloons in the middle of the road. How to deal with long-tail scenarios has always been a difficult problem in the industry, and has gradually become the key to restricting the development of autonomous driving. To solve these problems, the autonomous driving system needs to accumulate a large amount of data and continuously optimize the model.
  • the traditional model iterative verification method adopts the method of model iteration driven by functional testing.
  • data collection is driven by requirements and problems, and then the data is manually analyzed and marked to design an optimization plan.
  • Testing finally forms a serial iterative process of labeling, development, and testing.
  • This method is effective for software function development. Limited manpower can solve limited problems and realize functions within a specific range.
  • the traditional iterative verification method of the model is difficult to make autonomous driving truly land, so that the entire industry can achieve safe operation at all times and under all working conditions.
  • the traditional problem-driven approach relies on a serial development model to optimize the model. The development and testing cycle is long and cannot be carried out in parallel.
  • the method of manually labeling data takes a long time and the labeling efficiency is low; thirdly, most of the tests verify the model through manual construction of typical scenarios or random tests, and the coverage of actual running scenarios is low.
  • the above aspects show that the problem-driven approach can no longer meet the needs of solving a large number of problems in real scenarios, and cannot automatically solve most of the problems, and cannot efficiently achieve the goal of autonomous driving.
  • the embodiments of the present application aim to solve at least one of the above technical problems.
  • the embodiment of the present application provides a model automation iterative method, including:
  • the embodiment of the present application provides a vehicle-end model automation iteration method, including:
  • the in-use car-side model In cooperation with cloud computing resources, iterate the in-use car-side model into a trained car-side model; wherein, the trained car-side model is valuable to the performance improvement of the car-side model by cloud computing resources
  • the model obtained by optimizing the data and using the optimization results to train the car-end model.
  • the embodiment of the present application provides a cloud model automation iteration method, including:
  • the embodiment of the present application provides a vehicle-end execution device, including:
  • the vehicle-side calculation module is equipped with a vehicle-side model, and the result is obtained through vehicle-side model reasoning;
  • the car-end acquisition module is used to cooperate with the cloud execution device, and collect valuable data for improving the performance of the car-end model according to the result of the car-end model reasoning; wherein, the cloud execution device improves the performance of the car-end model Optimize with valuable data, and use the optimization results to train the car model;
  • the vehicle-side calculation module is also used to cooperate with the cloud execution device to iterate the configured vehicle-side model into a trained vehicle-side model.
  • the embodiment of the present application provides a cloud execution device, including:
  • the cloud acquisition module is used to cooperate with the vehicle-side execution equipment, and collect valuable data for improving the performance of the vehicle-side model according to the results of vehicle-side execution equipment reasoning through the vehicle-side model;
  • the data optimization module is used to optimize the data valuable to the performance improvement of the car end model
  • the training module is used to train the vehicle end model by using the optimization result
  • the iteration module is used to cooperate with the vehicle-end execution equipment to iterate the vehicle-end model being used by the vehicle-end execution equipment into a trained vehicle-end model.
  • the embodiment of the present application provides an electronic device, which includes: at least one processor, and a memory connected in communication with the at least one processor, wherein the memory stores information that can be used by the at least one processor Executable instructions, the instructions are executed by the at least one processor, so that the at least one processor can execute the steps of the aforementioned iterative method for vehicle-end model automation.
  • the embodiment of the present application provides an automatic driving vehicle, including the aforementioned electronic device.
  • the embodiment of the present application provides a storage medium, on which a computer program is stored, and when the program is executed by a processor, the steps of the foregoing automatic iteration method for the vehicle end model are implemented.
  • the embodiment of the present application provides an electronic device, including: at least one processor, and a memory communicatively connected to the at least one processor, wherein the memory stores information executable by the at least one processor. instructions, the instructions are executed by the at least one processor, so that the at least one processor can execute the steps of the aforementioned cloud model automation iterative method.
  • the embodiment of the present application provides a storage medium, on which a computer program is stored, and when the program is executed by a processor, the steps of the aforementioned cloud model automation iteration method are implemented.
  • the model automation iteration method provided by this application adopts the vehicle-side reasoning-cloud training mode, that is, deploying a multi-task, lightweight vehicle-side model on the vehicle side, and automatically and targetedly collect the results based on the vehicle-side model reasoning Use valuable data to improve the performance of the car-side model, and then use the powerful computing power and data storage capacity of the cloud to automatically complete a series of operations such as training data set generation, model training, and model iteration in real time; this car-side reasoning --The cloud training model takes full advantage of the resource advantages of the cloud and improves the efficiency of model iteration for autonomous driving vehicles.
  • the method provided by this application automatically collects valuable data for improving the performance of the vehicle-side model in an environment with limited vehicle-to-cloud communication resources.
  • the automatic collection process is not only efficient, but also covers rare, abnormal, and sudden long-term In the end scene, duplicate data and junk data are shielded, the validity, diversity and integrity of the collected data are guaranteed, and a sufficient, high-quality, diverse, effective and reliable data foundation is provided for the automatic completion of model training and model iteration on the cloud .
  • the method provided by this application automatically generates a training data set based on data valuable to the performance improvement of the car-side model.
  • the data optimization module can use a single-task, deep-level data-driven model to automatically complete data annotation to obtain a training data set, or use The mathematical programming model quickly and effectively obtains the training data set that meets the requirements.
  • This method of automatically generating the training data set greatly reduces the manual labeling work, which obviously helps to solve the problem of long time-consuming and slow model iteration due to low labeling efficiency. question.
  • FIG. 1 is a system architecture diagram of the model automation iteration system provided by the embodiment of the present application.
  • Fig. 2 is a schematic flow chart of the model automation iterative method provided by the embodiment of the present application
  • Fig. 3 is an example of data valuable to the performance improvement of the vehicle-end model collected by the vehicle-end acquisition module
  • Figure 4 is an example of the type of elements contained in the scene
  • Fig. 5 is an example of data valuable to the performance improvement of the vehicle-end model collected by the vehicle-end acquisition module
  • Fig. 6 is a kind of schematic diagram of the reasoning result of behavior prediction model and its expected value
  • FIG. 7 is a schematic structural diagram of an autonomous vehicle
  • FIG. 8 is a schematic structural diagram of a vehicle computing system
  • Fig. 9 is a possible example of the self-driving vehicle and the vehicle-mounted execution device
  • FIG. 10 is a schematic structural diagram of a cloud execution device
  • Fig. 11 is another schematic flowchart of the model automation iterative method provided by the embodiment of the present application.
  • Fig. 12 is another schematic flowchart of the model automation iteration method provided by the embodiment of the present application.
  • Fig. 13 is another schematic flowchart of the model automation iterative method provided by the embodiment of the present application.
  • FIG. 14 is a schematic flow diagram of a possible example of iterating the vehicle-side model being used by the vehicle-side computing resources into a trained vehicle-side model;
  • FIG. 15 is a schematic flowchart of a possible example of iterating the data-driven model being used by cloud computing resources into a trained data-driven model.
  • FIG. 1 is a system architecture diagram of the model automation iteration system provided by the embodiment of the present application.
  • cloud execution device 10 and cloud database 40 are cloud execution devices 10 and cloud database 40.
  • the vehicle-side execution device 20 includes: a vehicle-side collection module 20A, and a vehicle-side computing module 20B.
  • the vehicle-side computing module is configured with a vehicle-side model 20C.
  • the vehicle-end execution equipment can be applied to the self-driving vehicle, wherein the self-driving vehicle is provided with at least one sensor, such as vehicle-mounted radar (such as millimeter-wave radar, infrared radar, laser radar, Doppler radar, etc.), light sensor, rainfall sensor, etc. Sensors, visual sensors (such as cameras, driving recorders), vehicle attitude sensors (such as gyroscopes), speed sensors (such as Doppler radar), inertial measurement units (IMUs), etc.
  • vehicle-mounted radar such as millimeter-wave radar, infrared radar, laser radar, Doppler radar, etc.
  • light sensor such as rain sensor, etc.
  • Sensors such as cameras, driving recorders
  • vehicle attitude sensors such as gyroscopes
  • speed sensors such as Doppler radar
  • IMUs inertial measurement units
  • the car-end acquisition module has the function of data acquisition, and sends the collected data to the host computer for analysis and processing. It can be used to collect analog or digital signals collected by various sensors installed on the self-driving vehicle, and can also be used to collect the car-end computing module. The result of inference through the vehicle-side model can also be used to collect vehicle status data, map data, driver operation data, etc.
  • the vehicle end acquisition module has a built-in data acquisition card (that is, a computer expansion card that realizes data acquisition function), which can be connected through USB, PXI, PCI, PCI Express, FireWire (1394), PCMCIA, ISA, Compact Flash, 485, 232, Ethernet, Various wireless networks and other buses collect and send data.
  • the car-end acquisition module also has data processing functions, specifically to cooperate with the cloud-end acquisition module to extract valuable data for improving the performance of the car-end model from the collected data.
  • the vehicle data storage unit has a data storage function, which can be used to store the signals collected by the aforementioned various sensors, the results of vehicle model reasoning, vehicle status data, map data, driver operation data, and can also be used to store operating systems, applications, etc. .
  • the vehicle-end data storage unit can be implemented by embedded multimedia card (eMMC), single-level cell flash memory (SLC NAND), universal flash memory (UFS), solid-state drive (SSD), etc.
  • the vehicle-end data storage unit may be set in the vehicle-end execution device, or may be an external device other than the vehicle-end execution device.
  • the car-side model has reasoning functions and can be used to implement functions such as target detection, behavior prediction, and decision-making planning for autonomous vehicles.
  • the vehicle end model may be a model of a neural network type or a non-neural network type model. In the embodiment of the present application, only the vehicle end model is a model of a neural network type as an example.
  • the vehicle-side computing module obtains sensor data, vehicle status data, map data, driver operation data, etc., and then uses these data as input data of the vehicle-side model, and uses the vehicle-side model to perform inference to realize target detection and behavior prediction of autonomous vehicles , decision planning and other functions.
  • the cloud execution device 10 includes: a cloud collection module 10A, a data optimization module 10B, a training module 10C, and an iteration module 10D.
  • the cloud execution device can be implemented by a cloud server.
  • the data transmission between the vehicle-side execution device and the cloud execution device is realized through a communication interface, which can use vehicle wireless communication technology V2X, vehicle Ethernet, 3G/4G/5G mobile communication technology, etc. for communication.
  • the cloud acquisition module has the function of data acquisition, and sends the collected data to the host computer for analysis and processing. There is a data transmission relationship between the cloud acquisition module and the vehicle-end acquisition module, and the cloud-end acquisition module obtains data from the vehicle-end acquisition module according to requirements.
  • the cloud acquisition module has a built-in data acquisition card, which can collect and send data through buses such as USB, PXI, PCI, PCI Express, FireWire (1394), PCMCIA, ISA, Compact Flash, 485, 232, Ethernet, and various wireless networks.
  • the cloud acquisition module also has a data processing function, specifically to cooperate with the vehicle-end acquisition module to collect valuable data for improving the performance of the vehicle-end model.
  • the cloud database has a data storage function, which can be realized by using cloud storage technology, cloud database technology, etc.
  • the data optimization module has data processing functions, which can realize data processing functions such as data labeling, data fitting, and data noise reduction.
  • the training module uses the optimization results obtained by the data optimization module to train the vehicle-end model.
  • the iterative module uses the vehicle-side model trained by the training module to iteratively update the vehicle-side model being used by the vehicle-side execution device.
  • vehicle-side computing resources include but are not limited to vehicle-side execution devices, vehicle-side data storage units, and may also include other computing resources set up on self-driving vehicles.
  • cloud computing resources include but are not limited to cloud execution devices and cloud databases, and may also include other resources based on cloud computing technologies.
  • FIG. 2 is a schematic flow chart of the model automation iteration method provided by the embodiment of the present application.
  • the model automation iteration method provided by the embodiment of the present application may include:
  • the vehicle-end calculation module uses sensor data, vehicle state data, map data, driver operation data, etc. as the input data of the vehicle-end model, uses the vehicle-end model to perform inference, and realizes target detection, behavior prediction, and decision-making of autonomous vehicles. Planning and other functions to obtain target detection results, behavior prediction results, decision-making planning results, etc.
  • the result of vehicle-end model reasoning is stored in the vehicle-end data storage unit and acquired by the vehicle-end acquisition module.
  • the vehicle-side acquisition module can directly obtain the vehicle-side model reasoning result from the vehicle-side computing module, or can obtain the vehicle-side model reasoning result from the vehicle-side data storage unit.
  • the result of car-side model reasoning can have the following form:
  • Target detection results which may include: target detection frames based on images or laser point clouds, target types, confidence levels, etc.
  • Behavior prediction results which may include: target behavior, target intention, motion trajectory, etc.; among them, target behavior can be lane change, overtaking, acceleration and deceleration, left turn, right turn, U-turn, parking, etc.; target intention can be Changing lanes, going straight, overtaking, crossing the road, parking, etc.; the trajectory can be the position of the target at multiple time points in the future.
  • (C) Decision planning results which can include: self-vehicle behavior decision-making, self-vehicle trajectory planning, etc.; vehicles, etc.; ego vehicle trajectory planning can be path planning, speed planning, vertical and horizontal coupling planning, trajectory tracking, following vehicle planning, lane changing trajectory planning, U-turn trajectory planning, in-situ steering trajectory planning, escape trajectory planning, etc.
  • the vehicle-side model can adopt a multi-task and lightweight network structure.
  • multitasking means that the network structure has the characteristics of sharing parameters and tasks
  • lightweight means that the network structure has the characteristics of satisfying computing efficiency and capability under the condition of limited storage space and power consumption constraints.
  • multi-task means that the feature information of the image can be reused, and the results required for multiple tasks can be obtained through one model reasoning, such as simultaneous detection of pedestrians, vehicles and signal lights, etc.; lightweight It can adapt to the limited computing power of the car and meet the reasoning efficiency of the car;
  • multi-task means that the feature information of point cloud can be reused, and the results required for multiple tasks can be obtained through one model reasoning, such as simultaneous detection of pedestrians, vehicle types and obstacles Dynamic and static properties, etc.; light weight can adapt to the limited computing power of the car end, and meet the reasoning efficiency of the car end;
  • multi-task means that the sensory fusion results and map feature information can be reused, and the results required for multiple tasks can be obtained through one model reasoning, such as simultaneously predicting obstacle behavior categories, future path shapes and Future speed change trends, etc.; lightweight can adapt to the limited computing power of the car end, and meet the reasoning efficiency of the car end;
  • multi-task means that the feature information of perception fusion results, maps and behavior predictions can be reused, and the results required for multiple tasks can be obtained through one model reasoning, such as giving the expected behavior category and expected behavior at the same time.
  • Path shape and expected speed configuration, etc.; light weight can adapt to the limited computing power of the car end and meet the reasoning efficiency of the car end.
  • the car-side model can also adopt a network structure with multi-dimensional characteristics.
  • the car-side model is a vision-based target detection model
  • the multi-dimensional network structure can realize the purpose of mining the internal relationship between multiple targets.
  • the car-end acquisition module cooperates with the cloud-end acquisition module to collect valuable data for improving the performance of the car-end model according to the results of the inference of the car-end model.
  • the iterative method of model training commonly used in the field of autonomous driving is to use all the results of model reasoning in subsequent model training. Whether the reasoning effect of the scene is good enough or not good enough, the reasoning results are uniformly used for model training. This kind of training without distinguishing key points cannot achieve the training purpose quickly and in a targeted manner. It can be adapted to common typical scenarios, but for rare, sudden, and abnormal long-tail scenarios, this model iteration method will be difficult to adapt.
  • this application collects data that is valuable for improving the performance of the car-side model, and then uses these data that are valuable for improving the performance of the car-side model to train and iterate the car-side model. In this way, valuable data can be extracted in a targeted manner according to the training purpose, so as to achieve the training goal quickly and effectively.
  • valuable data for improving the performance of the car-side model includes not only the inference results of the car-side model itself, but also environmental data, map data, and vehicle state data that are synchronized with the inference results of the car-side model in time and space.
  • driver’s operation data and other time-space synchronization information are combined with the inference results of the car-side model, which can fully reflect the scene where the autonomous vehicle is in, and it is more meaningful for the training model.
  • environmental data may include: static environment (fixed obstacles, building facilities, traffic facilities, roads), dynamic environment (dynamic traffic lights, traffic police), communication environment (signal strength, signal delay time, electromagnetic interference strength), traffic Participants (pedestrians, motor vehicles, non-motor vehicles, animals), meteorological environment (temperature, humidity, light conditions, weather conditions), etc.;
  • Map data can include: high-precision maps, traffic control information, navigation information, etc.;
  • Vehicle state data can include: basic attributes of the vehicle (such as body weight, geometric dimensions, basic performance), vehicle position (coordinates, lane position), motion state (lateral motion state, longitudinal motion state), human-computer interaction (entertainment, driving task )wait;
  • basic attributes of the vehicle such as body weight, geometric dimensions, basic performance
  • vehicle position coordinates, lane position
  • motion state lateral motion state, longitudinal motion state
  • human-computer interaction entity, driving task
  • the driver's operation data may include: whether to take over the vehicle, the driver's specific actions, etc.
  • collecting data that is valuable for improving the performance of the car-end model can include the following situations:
  • the car-side acquisition module constructs a scene using the result of car-side model reasoning and its space-time synchronization information, and uploads it to the cloud.
  • the result of vehicle-side model reasoning and its space-time synchronization information are collected as valuable data for improving the performance of the vehicle-side model.
  • the scene refers to the overall dynamic description of the comprehensive interaction process between the autonomous driving vehicle and other vehicles, roads, traffic facilities, weather conditions and other elements in the driving environment within a certain time and space range.
  • the organic combination of the driving scene and the driving environment includes not only various entity elements, but also the actions performed by entities and the connection relationship between entities.
  • Fig. 4 shows an embodiment of element types included in a scene.
  • the cloud acquisition module compares the scene uploaded by the car-end acquisition module If it is found that the scene does not exist in the scene library, it means that the car-end model cannot cover (adapt) to this scene, and this scene needs to be added to the scene library. After receiving the command, the inference result of the car-side model corresponding to this scene and its space-time synchronization information are collected as valuable data for improving the performance of the car-side model.
  • the cloud acquisition module compares the scene uploaded by the vehicle end acquisition module with the scene library, if the following two situations occur, it can be counted as the scene missing in the scene library:
  • the scene library does not cover the category corresponding to the scene.
  • the road type in the scene library covers three categories: urban roads, expressways, and park roads, while the scene category uploaded by the vehicle-side collection module is rural roads. At this point, it can be determined that the scene is missing in the scene library;
  • the vehicle-end acquisition module can encode the scene and upload the scene code to the cloud.
  • the vehicle-end acquisition module can also Store the code library corresponding to the scene library (which contains the scene code corresponding to each scene in the scene library), and the cloud acquisition module compares the scene code uploaded by the car terminal acquisition module with the code library.
  • the cloud acquisition module sends a command, and the car-end acquisition module will immediately
  • the corresponding inference results of the vehicle-side model and its space-time synchronization information are collected as valuable data for improving the performance of the vehicle-side model.
  • the vehicle-end acquisition module should encode the scene according to a predetermined coding rule.
  • the predetermined coding rule can be coded according to scene elements.
  • the scene elements are encoded according to the order of the parent node elements.
  • the number after # indicates the order of the current element in its parent node elements:
  • the scene contains the state of lateral motion, then from left to right, the code corresponding to the vehicle itself is 1, the code corresponding to the motion state is 3, the code corresponding to the lateral motion state is 1, and the scene code correspondingly contains the number 131;
  • the scene code contains corresponding data groups (232, 131).
  • the purpose of the training is to make the car-end model cover (adapt to) long-tail scenarios such as rare, sudden, and abnormal
  • the vehicle-end acquisition module monitors that the vehicle-side model reasoning results and/or space-time synchronization information do not belong to conventional scenarios
  • the vehicle-side model reasoning results and their space-time synchronization information are used as the result of vehicle-side model reasoning to improve the performance of the vehicle-side model. valuable data is collected.
  • the conventional scene here refers to the ubiquitous and common traffic scene in the physical world, such as vehicles running normally on the road, traffic lights, traffic signs, lane lines, road shoulders and other conventional traffic facilities appearing on the road.
  • the opposite is the long-tail scene, that is, rare, sudden, abnormal traffic scenes that rarely or almost impossible to appear in the physical world, for example, vehicles driving on the sky ⁇ flower beds ⁇ buildings, suddenly appearing in the road Paints ⁇ Buildings ⁇ Large floating objects (such as balloons), etc.
  • long-tail scenarios often mean a high risk factor and complex operation and processing.
  • long-tail scenarios The corresponding information is valuable data for improving the performance of the car model.
  • the self-driving vehicle is in a rare, sudden, or abnormal long-tail scene. Its space-time synchronization information is collected as valuable data for improving the performance of the car-side model.
  • the inference result of the target detection model based on the laser point cloud is that the target vehicle starts to drive on the building on the side of the road from a certain frame, and this situation lasts for multiple frames, and before this frame
  • the monitoring result has always been that the target vehicle is driving on the road.
  • This reasoning result (the vehicle is driving on the building) does not belong to the conventional scene (the vehicle is driving on the road).
  • This situation may be based on the target of the laser point cloud.
  • There is an error in the reasoning of the detection model it may also be a lidar failure, or even the target vehicle is actually driving on a flower bed, etc.
  • These abnormal or rare scenarios are all long-tail scenarios, which need to be covered by the vehicle-side model.
  • the inference results and their space-time synchronization information at this time are collected as valuable data for improving the performance of the car-end model, and are used for subsequent model training.
  • a specific example is that the behavior prediction model predicts the trajectory of the target in a certain time period in the future as a dotted line in the figure, and when the time period passes, based on the multi-sensor (visual sensor , lidar, millimeter-wave radar, etc.) shows that the real trajectory of the target is the solid line in the figure, and the solid line in the figure is used as the expected value.
  • the reasoning result of the behavior prediction model does not match the expected value (the solid line and The overlapping degree of the dotted line is low), which indicates that the inference result of the behavior prediction model is abnormal.
  • the learning effect (reasoning effect) of the behavior prediction model on the target behavior in the past period of time is not good enough, and the car-side model needs to be trained To adapt to these target behaviors, it is necessary to collect the inference results and their space-time synchronization information at this time as valuable data for improving the performance of the car-side model for subsequent model training.
  • the reasoning result of the decision planning model is to control the vehicle to continue going straight, but the driver suddenly takes over the vehicle and controls the vehicle to change lanes. This situation is likely to be abnormal in the reasoning result of the decision planning model. It is in line with the common operations of human drivers in the same scene, indicating that the learning effect of the decision-making planning model is not good enough for this scene. Therefore, it is necessary to use the reasoning results and their space-time synchronization information at this time as a valuable tool for improving the performance of the car-side model. The data is collected for subsequent model training.
  • the reasoning results of the decision planning model are evaluated, and the evaluation results show that the comfort of the driver and passengers is lower than the predetermined comfort threshold (expected value), or the safety risk index is higher than the predetermined risk threshold ( expected value), this situation indicates that the learning effect of the decision-making planning model on the current scene is not good enough, so it is necessary to collect the inference results and their space-time synchronization information at this time as valuable data for improving the performance of the car-end model for subsequent use. model training.
  • the object detection results based on images show that obstacles are dynamic obstacles, while the target detection results based on laser point cloud and millimeter wave point cloud both show that obstacles are static obstacles.
  • the results obtained by the logic of this algorithm are checked for consistency.
  • the result of the test is that the consistency of the three is not good (the one based on the image is a dynamic obstacle, and the one based on the laser point cloud and millimeter-wave radar is a static obstacle), and it has not reached the predetermined Consistency lower limit (for example, the three must be completely consistent).
  • the inference result of the image-based target detection model may not be accurate enough, or the target detection model based on laser point cloud and/or millimeter wave point cloud.
  • the inference result is not accurate enough, which means that among the target detection models of these three types of algorithmic logic, at least one target detection model is not good enough for the reasoning effect of the current scene. It is necessary to improve the reasoning ability of the target detection model in the current scene, so it is necessary to use
  • the inference results and their space-time synchronization information at this time are collected as valuable data for improving the performance of the car-end model, and are used for subsequent model training.
  • the result given by the decision planning based on deep learning is to control the vehicle to change lanes to the left, but the result given by the decision planning based on rules is to control the vehicle to change lanes to the right.
  • the result obtained by the logic is checked for consistency.
  • the result of the test is that the consistency between the two is not good, and the lower limit of consistency has not been reached (for example, the two must be completely consistent).
  • This situation may be based on deep learning decision planning
  • the inference result is not accurate enough, or the result of the rule-based decision planning is not accurate enough. This shows that among the decision planning models of these two types of algorithmic logic, at least one of the decision planning models is not good enough for the reasoning effect of the current scene, and the decision planning needs to be improved.
  • the reasoning ability of the model in the current scene, so the reasoning results and their space-time synchronization information at this time need to be collected as valuable data for improving the performance of the car-side model for subsequent model training.
  • the scene library it is necessary to make the scene library cover more scenes with very good reasoning effect of the car-side model.
  • the behavior prediction model if the predicted target trajectory obtained by reasoning overlaps well with the real target trajectory displayed by the multi-sensor fusion results.
  • the real trajectory of the target based on multi-sensor fusion is the expected value, that is, the matching degree between the inference result of the behavior prediction model and the expected value of the target behavior has reached a good level (for example, the predetermined matching threshold has been reached), which shows that the prediction model has a good effect on The inference effect (learning effect) of the target behavior is very good, and the behavior prediction model needs to continue to maintain this good reasoning ability. Therefore, it is also necessary to use the reasoning results and their space-time synchronization information at this time as valuable data for improving the performance of the vehicle-side model. Collect data for subsequent model training.
  • the target detection results based on images, the target detection results based on laser point cloud, and the target detection results based on millimeter wave point cloud all show that obstacles are static obstacles.
  • the results obtained by the logic are checked for consistency.
  • the test results show that the three are completely consistent (both are static obstacles), reaching the predetermined upper limit of consistency.
  • This situation shows the inference effect of the target detection model based on these three types of algorithm logic. Very good, the target detection model needs to continue to maintain this good reasoning ability, so it is also necessary to collect the reasoning results and their space-time synchronization information at this time as valuable data for improving the performance of the car-side model for subsequent model training .
  • the result given by the decision-making plan based on deep learning is to control the vehicle to change lanes to the left, and the result given by the rule-based decision-making plan is also to control the vehicle to change lanes to the left.
  • the results obtained by the two algorithm logics are checked for consistency. The result is that the two are completely consistent, reaching the predetermined upper limit of consistency. This situation shows that the reasoning effect of decision planning based on deep learning and decision planning based on rules is very good, and the decision planning model needs to continue to maintain this good reasoning ability. , so it is also necessary to collect the inference results and their space-time synchronization information at this time as valuable data for improving the performance of the car-side model for subsequent model training.
  • the embodiment provided by this application realizes the automatic collection of data that is beneficial to the performance improvement of the vehicle-side model through the mutual cooperation of the vehicle-side computing resources and the cloud computing resources.
  • This data collection method is not only fast, but also targeted, and can In the case of limited communication resources with the cloud, useful data can be collected more efficiently to provide an effective and reliable data basis for subsequent training of the vehicle model.
  • the valuable data is optimized by the data optimization module to obtain an effective and reliable training data set (optimization result) to prepare for the next step of training the vehicle-side model.
  • the data optimization module can use different algorithm processing methods to optimize valuable data:
  • the data optimization module uses the data-driven model to label the valuable data, and then uses the labeled data set to train the car-end model to improve its reasoning ability .
  • the "data-driven model” referred to in the embodiment of the present application refers to a data-driven model, such as a deep learning model, a traditional machine learning model, and the like.
  • the data-driven model is a traditional machine learning model, which can use support vector machine algorithm (SVM), Adaboost algorithm, logistic regression algorithm, hidden Markov algorithm, K nearest neighbor algorithm (KNN), three-layer artificial Any traditional machine learning algorithm such as neural network algorithm, Bayesian algorithm, decision tree algorithm, etc.
  • SVM support vector machine algorithm
  • Adaboost algorithm logistic regression algorithm
  • hidden Markov algorithm hidden Markov algorithm
  • KNN K nearest neighbor algorithm
  • three-layer artificial Any traditional machine learning algorithm such as neural network algorithm, Bayesian algorithm, decision tree algorithm, etc.
  • the above-mentioned traditional machine learning model (such as SVM or Adaboost) is calculated based on the artificially defined Histogram of Oriented Gradients (HOG) feature to achieve the purpose of labeling valuable data.
  • HOG Histogram of Oriented Gradients
  • the data optimization module uses a data-driven model to automatically label valuable data to obtain an effective and reliable labeled data set, and then uses the labeled data set to train the vehicle-side model.
  • Tags have different definitions for car models with different functions.
  • the label includes information such as the detection frame, category, and confidence level of a specific object.
  • the car-side model as the model for target detection as an example, using the powerful computing power and data storage capacity of cloud computing resources, set a data-driven model in the cloud whose target detection capability is much higher than that of the car-side model, and then use the data-driven model to identify Specific targets in valuable data (such as: pedestrians, motor vehicles, non-motor vehicles, traffic signs, traffic lights, lane lines, etc.), because the recognition ability of the data-driven model is much higher than that of the vehicle-side model, the recognition results are relatively
  • the reasoning ability of the car-side model can be used as a label to train the car-side model. This automatic labeling method can save a lot of manual labeling workload, significantly improve the efficiency of data labeling, and speed up model iteration.
  • the data-driven model can be set to multiple deep learning models with single-task and deep-level feature network structures.
  • the single-task feature means that a single model is only used to perform a single task, and the models are independent of each other and do not share parameters.
  • Using the single-task feature of the model can maximize the recall and recognition accuracy of individual targets; deep-level features It means that the model has multiple hidden layers, which can abstract the input features at multiple levels, better divide different types of data, and use the deep features of the model to improve the target recall and recognition accuracy of individual targets under complex road conditions.
  • multiple single-task, deep-level data-driven models can be set according to specific targets that need to be perceived in specific scenarios.
  • multiple data-driven models are set, and each data-driven model is used separately. It is used to detect pedestrians, motor vehicles, non-motor vehicles, traffic signs, traffic lights, lane lines, etc.
  • the data optimization module uses the mathematical programming model to optimize the valuable data to obtain the expected value, and then uses the expected value to train the car-end model to improve its reasoning ability.
  • the optimization method adopted by the mathematical programming model may be linear programming, integer linear programming, nonlinear programming, multi-objective programming, dynamic programming, and the like.
  • the prediction trajectory of the behavior prediction model for the target in a certain period of time in the future is a dotted line in the figure.
  • the result of perception fusion is the solid line in the figure.
  • the result of multi-sensor perception fusion can be used as the expected value to train the vehicle.
  • the result based on multi-sensor perception fusion is also detected, which may be different from the performance of the target in the real world.
  • the solid line shown in Figure 6 is composed of multiple discrete trajectory points, where M
  • M The distance between the point and the trajectory line formed by other trajectory points is far away, which belongs to the jump point.
  • the trajectory of the target in the real world is generally smooth, and there will be no jump point.
  • the M point needs to be eliminated.
  • the trajectory composed of the remaining trajectory points is the expected value to train the vehicle-side model.
  • the process of eliminating M points is the process of optimizing valuable data.
  • Linear programming and nonlinear programming can be used to process discrete trajectory points. Specifically, least squares and simulated annealing algorithms can be used to achieve data Algorithms for fitting, data noise reduction purposes.
  • the training module trains the car-side model using the labeled data set obtained by the data-driven model.
  • the training module uses the mathematical programming model to optimize the valuable data to train the car-side model.
  • the iteration module determines the model parameters corresponding to the car-side model trained by the training module, and sends the model parameters to the car-side computing module, and the car-side computing module uses the issued model parameters to iterate the car-side model being used as The trained car model.
  • the iteration module will test the trained vehicle-side model, and when the test result meets the iteration requirement (indicating that the reasoning ability of the trained vehicle-side model It is obviously better than the in-vehicle model in use. You can judge whether the test results meet the iteration requirements by setting test indicators for the trained in-vehicle model and the in-use in-vehicle model to represent their respective reasoning capabilities, for example
  • the test index can be set according to the reasoning ability performance of the trained car-end model and the in-use car-end model in the test.
  • the test index can be set to be positively correlated with the reasoning ability of the model, that is, the better the test index, the stronger the reasoning ability , when the test index of the trained car-end model is better than the test index of the car-end model being used, it indicates that the test result meets the iteration requirements), the model parameters will be sent to the car-end computing module to complete the car-end Model iteration operations.
  • the training module can also use the optimization results to train the data-driven model to improve the reasoning ability of the data-driven model in such aspects that are inferior to the vehicle-end target detection model.
  • the iteration module sends the model parameters corresponding to the trained data-driven model to the data optimization module, and iterates the currently-used data-driven model into the trained data-driven model.
  • the iteration module can also test the trained data-driven model, and when the test result meets the iteration requirement (indicating that the reasoning of the trained data-driven model The ability is significantly better than the data-driven model being used. You can judge whether the test results meet the iteration requirements by setting test indicators for the trained data-driven model and the data-driven model in use to represent their respective reasoning capabilities.
  • the test index can be set according to the reasoning performance of the trained data-driven model and the in-use data-driven model in the test. The test index can be set to be positively correlated with the reasoning ability of the model, that is, the better the test index, the better the reasoning ability.
  • the model parameters will be sent to the data optimization module to complete the data-driven Model iteration operations.
  • the model automation iteration method provided by the embodiment of the present application first automatically collects valuable data for improving the performance of the car-side model, then automatically optimizes the valuable data, and finally uses the optimization results to train the car-side model and complete the iteration:
  • This method adopts the car-side reasoning-cloud training mode, that is, deploys a multi-task, lightweight car-side model on the car-side, and automatically and targetedly collects the results of car-side model inference based on the car-side
  • the performance of the model is improved with valuable data, and then using the powerful computing power and data storage capacity of the cloud to automatically complete a series of operations such as training data set generation, model training, and model iteration in real time;
  • the training mode takes full advantage of the resource advantages of the cloud and improves the efficiency of the iteration of the autonomous vehicle model;
  • This method automatically collects valuable data for improving the performance of the vehicle-side model in an environment with limited vehicle-to-cloud communication resources.
  • the automatic collection process is not only efficient, but also covers rare, abnormal, and sudden long-term In the end scene, duplicate data and junk data are shielded, the validity, diversity and integrity of the collected data are guaranteed, and a sufficient, high-quality, diverse, effective and reliable data foundation is provided for the automatic completion of model training and model iteration on the cloud ;
  • This method automatically generates a training data set based on data valuable to the performance improvement of the vehicle-side model.
  • the data optimization module can use the single-task, deep-level data-driven model to automatically complete the data annotation to obtain the training data set, or use The mathematical programming model quickly and effectively obtains the training data set that meets the requirements.
  • This method of automatically generating the training data set greatly reduces the manual labeling work, which obviously helps to solve the problem of long time-consuming and slow model iteration due to low labeling efficiency. question.
  • an embodiment of the present application provides an automatic driving vehicle.
  • the self-driving vehicle ADV can be realized by using the self-driving technology and has the function of carrying people (such as family cars, buses, etc.), and carrying functions (such as ordinary trucks, vans, trailers, and closed trucks). , tank trucks, flatbed trucks, container trucks, dump trucks, trucks with special structures, etc.), tool functions (such as logistics distribution vehicles, patrol vehicles, sanitation vehicles, vacuum vehicles, floor washing vehicles, sweeping robots, cutting lawn machines, golf carts), entertainment functions (such as recreational vehicles, playground automatic driving devices) or vehicles with special rescue functions (such as fire trucks, ambulances, etc.).
  • carrying people such as family cars, buses, etc.
  • carrying functions such as ordinary trucks, vans, trailers, and closed trucks.
  • tool functions such as logistics distribution vehicles, patrol vehicles, sanitation vehicles, vacuum vehicles, floor washing vehicles, sweeping robots, cutting lawn machines, golf carts
  • entertainment functions such
  • Fig. 7 shows the structure of an automatic driving vehicle ADV according to an embodiment of the present application.
  • the autonomous vehicle ADV includes a power system V-110, a sensor system V-120, an actuation system V-130, a peripheral system V-140, and a vehicle computing system V-150.
  • an autonomous vehicle ADV vehicle may include more, fewer, or different units, and each unit may include more, fewer, or different components.
  • the units and components shown in FIG. 7 can also be combined or divided in any number.
  • the powertrain V-110 can be configured to provide sporty power to the vehicle.
  • Powertrain V-110 includes one or more of engine V-111, energy source V112, transmission V113, and wheels V114.
  • Engine V-111 can be any combination of internal combustion, electric motor, steam and Stirling engines, as well as other motors and engines.
  • powertrain V-110 may include multiple types of engines and/or motors.
  • a gas-electric hybrid may include a gasoline engine and an electric motor.
  • Energy source V112 may be an energy source V112 that powers engine V-111 in whole or in part.
  • Engine V-111 may be configured to convert energy source V112 into mechanical energy.
  • Energy sources V112 may include gasoline, diesel, propane, other compressed gas based fuels, ethanol, solar panels, batteries, and other sources of electrical power.
  • Energy source V112 may additionally or alternatively include any combination of fuel tanks, batteries, capacitors, and/or flywheels. In some possible designs, the energy source V112 can also provide energy for other units of the vehicle.
  • the transmission V113 may be configured to send mechanical power from the engine V-111 to the wheels V114.
  • the transmission V113 may include gearboxes, clutches, differentials, drive shafts, and/or other components.
  • the transmission V113 includes a drive shaft
  • the drive shaft may include one or more axles configured to couple to the wheels V114.
  • the wheels V114 can be configured in any form, including single-wheel, double-wheel, three-wheel, four-wheel, six-wheel and other forms. Other wheel V 114 configurations are also possible, such as configurations comprising eight or more wheels. In any event, the wheels V114 may be configured to rotate differentially relative to the other wheels V114. In some possible designs, the wheels V114 may comprise at least one wheel fixedly attached to the transmission V113, and at least one tire coupled to a rim of the vehicle which may come into contact with the road surface. Wheel V114 may comprise any combination of metal and rubber, or other combinations of materials.
  • Powertrain V-110 may additionally or alternatively include other components in addition to the aforementioned components.
  • Sensor system V-120 may include external sensor V-121 and internal sensor V-122.
  • Exterior sensors V- 121 may include a plurality of sensors configured to sense information about the environment in which the vehicle is located, and one or more actuators V 1216 configured to modify the position and/or orientation of the sensors.
  • the external sensors V-121 may include one or more of a position sensor V1217, an inertial sensor V1211, an object sensor V1212, an image sensor V1213.
  • the position sensor V1217 can be any sensor that estimates the geographical location of the vehicle, for example, the global positioning system GPS positioning device, the carrier phase difference RTK positioning device, the Beidou satellite positioning system positioning device, the GLONASS positioning system positioning device, the Galileo positioning system positioning device, the global Navigation satellite system GNSS positioning equipment.
  • the position sensor V1 217 may include a transceiver that estimates the position of the vehicle relative to the earth.
  • the inertial sensor V1211 may be any combination of sensors configured to sense a change in position and orientation of the vehicle from inertial acceleration, such as an inertial measurement unit IMU.
  • Inertial sensors V1211 may include accelerometers and gyroscopes in some possible designs.
  • the object sensor V1212 can be any sensor that uses radio signals or laser signals to sense objects in the vehicle's environment, such as radar, laser range finder, lidar. In some possible designs, in addition to sensing objects, radar and lidar may additionally sense the speed and/or direction of travel of the object. In some possible designs, object sensor V1212 may include an emitter that emits a radio or laser signal and a detector that detects the radio or laser signal.
  • the image sensor V1213 may include any camera (such as a still camera, a video camera, etc.) for capturing images of the environment in which the vehicle is located.
  • any camera such as a still camera, a video camera, etc.
  • the external sensor V-121 may also include other sensors, such as any sensor for detecting the distance of an object, such as a sonar V1214, an ultrasonic sensor V-1216, and the like.
  • the interior sensor V- 122 may include a plurality of sensors configured to detect information corresponding to a driving state of the vehicle.
  • interior sensors V-122 may include one or more of vehicle speed sensor V-1221, acceleration sensor V-1222, and yaw rate sensor V-1223.
  • the vehicle speed sensor V-1221 may be any sensor that detects the speed of the vehicle.
  • the acceleration sensor V-1222 may be any sensor that detects the acceleration of the vehicle.
  • the yaw rate sensor V-1223 may be any sensor that detects the yaw rate (rotational angular velocity) of the vehicle around the vertical axis of the center of gravity, for example, a gyro sensor.
  • the internal sensor V-122 may also include one or more of an accelerator pedal sensor V-1224, a brake pedal sensor V-1225, and a steering wheel sensor V-1226.
  • the accelerator pedal sensor V-1224 may be any sensor that detects the amount of depression of the accelerator pedal, and the accelerator pedal sensor V-1224 is provided, for example, on the shaft portion of the accelerator pedal of the vehicle.
  • the brake pedal sensor V-1225 may be any sensor that detects the depression amount of the brake pedal, and the brake pedal sensor V-1225 is provided, for example, on the shaft portion of the brake pedal.
  • the brake pedal sensor V-1225 can also detect the operating force of the brake pedal (depressing force on the brake pedal, pressure of the master cylinder, etc.).
  • the steering wheel sensor V-1226 can be any sensor that detects the rotation state of the steering wheel.
  • the detection value of the rotation state is, for example, steering torque or rudder angle.
  • the steering wheel sensor V-1226 is, for example, installed on the steering shaft of the vehicle.
  • interior sensors V-122 may also include other sensors, such as sensors that monitor various components inside the vehicle (eg, oxygen monitor, fuel gauge, engine oil temperature gauge, etc.).
  • sensor system V-120 may be implemented as a plurality of sensor assemblies, each sensor assembly configured to be mounted at a corresponding location on the vehicle (e.g., top, bottom, front, rear, left, right wait).
  • Actuation system V- 130 may be configured to control the driving behavior of the vehicle.
  • the actuation system V- 130 may include one or more of a steering module V- 131 , a throttle module V- 132 , and a braking module V- 133 .
  • the steering module V-131 may be any combination of devices that controls the steering torque (or steering torque) of the vehicle.
  • the throttle module V-132 can be any combination of devices that can control the operating speed of the engine V-111 and control the speed of the vehicle by adjusting the air supply to the engine (throttle opening).
  • the braking module V-133 may be any combination of devices that slows the vehicle, for example, the braking module V-133 may use friction to slow the wheels V114.
  • Peripherals system V- 140 may be configured to enable the vehicle to interact with external sensors V- 121 , other vehicles, external computing devices, and/or a user.
  • peripheral system V-140 may include one or more of a wireless communication device V-141, a wired communication interface V-142, a touch screen display V-143, a microphone V-144, and a speaker V-145.
  • Wireless communication device V-141 may be configured to connect directly or wirelessly to powertrain system V-110, sensor system V-120, actuation system V-130, peripheral system V-140, and vehicle computing system V-150 including One or more devices, and directly or wirelessly connect to one or more of other vehicles, central control systems, and entities in the hub service area.
  • the wireless communication device V-141 may include an antenna and a chipset based on wireless communication technology communication, wherein the wireless communication technology may include Global System for Mobile Communications (Global System for Mobile Communications, GSM), General Packet Radio Service (General Packet Radio Service, GPRS), Code Division Multiple Access (CDMA), Wideband Code Division Multiple Access (WCDMA), Time-Division Code Division Multiple Access (TD-SCDMA ), Long Term Evolution (LTE), Bluetooth (Blue Tooth, BT), Global Navigation Satellite System (Global Navigation Satellite System, GNSS), Frequency Modulation (FM), Near Field Communication , NFC), infrared technology (Infrared, IR).
  • GSM Global System for Mobile Communications
  • GSM Global System for Mobile Communications
  • General Packet Radio Service General Packet Radio Service
  • CDMA Code Division Multiple Access
  • WCDMA Wideband Code Division Multiple Access
  • TD-SCDMA Time-Division Code Division Multiple Access
  • LTE Long Term Evolution
  • Bluetooth Bluetooth
  • Bluetooth Bluetooth
  • Global Navigation Satellite System Global
  • GNSS can include Global Positioning System (Global Positioning System, GPS), Global Navigation Satellite System (Global Navigation Satellite System, GLONASS), Beidou Navigation Satellite System (Beidou Navigation Satellite System, BDS), Quasi-zenith Satellite System (Quasi-zenith) Satellite System, QZSS) and/or Satellite Based Augmentation Systems (Satellite Based Augmentation Systems, SBAS).
  • Global Positioning System Global Positioning System, GPS
  • Global Navigation Satellite System Global Navigation Satellite System
  • GLONASS Global Navigation Satellite System
  • Beidou Navigation Satellite System Beidou Navigation Satellite System
  • BDS Beidou Navigation Satellite System
  • Quasi-zenith Satellite System Quasi-zenith Satellite System
  • QZSS Satellite Based Augmentation Systems
  • SBAS Satellite Based Augmentation Systems
  • Wired communication interface V-142 may be configured to directly connect one or more equipment, and directly connect to one or more of other vehicles, central control systems, and entities in the hub service area.
  • the wired communication interface V-142 can include integrated circuit (Inter-Integrated Circuit, I2C) interface, integrated circuit built-in audio (Inter-Integrated Circuit Sound, I2S) interface, pulse code modulation (Pulse Code Modulation, PCM) interface, general asynchronous transceiver Transmitter (Universal Asynchronous Receiver/Transmitter, UART) interface, mobile industry processor interface (Mobile Industry Processor Interface, MIPI), general-purpose input and output (General-Purpose Input/Output, GPIO) interface, subscriber identity module (Subscriber Identity Module, SIM) interface, and/or Universal Serial Bus (Universal Serial Bus, USB) interface, etc.
  • I2C Inter-Integrated Circuit
  • I2S integrated circuit built-in audio
  • PCM pulse code modulation
  • PCM Pulse Code Modulation
  • the touchscreen display V-143 can be used by the user to enter commands into the vehicle.
  • the touchscreen display V- 143 may be configured to sense the position and/or movement of a user's finger through capacitive sensing, resistive sensing, or surface acoustic wave processing.
  • the touchscreen display V-143 is capable of sensing finger movement in a direction parallel or coplanar to the touchscreen surface, perpendicular to the touchscreen surface, or both, and is also capable of sensing the level of pressure applied to the touchscreen surface.
  • Touchscreen display V-143 may be formed from one or more translucent or transparent insulating layers and one or more translucent or transparent conductive layers.
  • the touchscreen display V-143 can also be configured in other forms.
  • Microphone V- 144 may be configured to receive sound signals (eg, voice commands or other audio input) and convert the sound signals to electrical signals.
  • sound signals eg, voice commands or other audio input
  • Speaker V-145 can be configured to output audio.
  • Peripherals system V- 140 may further or alternatively include other components.
  • Vehicle computing system V-150 may include a processor V-151 and a data storage device V-152.
  • Processor V-151 may be configured to execute instructions stored in data storage device V-152 to perform various functions, including but not limited to positioning fusion module V-1501, perception module V-1501 as described below 1502. Functions corresponding to the driving state determination module V-1503, the navigation module V-1504, the decision-making module V-1505, the driving control module V-1506, and the task receiving module V-1507.
  • the processor V-151 can include a general-purpose processor (such as CPU, GPU), a special-purpose processor (such as an application-specific integrated circuit (Application-specific integrated circuit, ASIC)), a field-programmable gate array (FPGA), a digital signal processor ( A combination of one or more of DSP), integrated circuits, microcontrollers, etc. In the case where the processor V-151 includes a plurality of processors V-151, these processors V-151 can work individually or in combination.
  • Data storage V- 152 may include one or more volatile computer-readable storage media and/or one or more non-volatile computer-readable storage media, such as optical, magnetic, and/or organic storage media.
  • the data storage device V-152 may include read only memory (ROM), random access memory (RAM), flash memory, electrically programmable memory (EPROM), electrically programmable and erasable memory (EEPROM), embedded multimedia card (eMMC), hard drive, or any combination of volatile or non-volatile media, etc.
  • the data storage V-152 may be wholly or partially integrated with the processor V-151.
  • the data storage device V-152 may be configured to store instructions executable by the processor V-151 to perform various functions, wherein these functions include but are not limited to the positioning fusion module V-1501, the perception module V-1502 as described below . Functions corresponding to the driving state determination module V-1503, the navigation module V-1504, the decision-making module V-1505, the driving control module V-1506, and the task receiving module V-1507.
  • the positioning fusion module V-1501 can be configured to receive environmental data, location data or other types of data sensed by the sensor system V-120, and obtain the fused data by performing time stamp alignment, fusion calculation and other processing on these data Environmental data and vehicle location data.
  • the positioning fusion module V-1501 may include, for example, Kalman filter, Bayesian network, and algorithms for realizing other functions.
  • the perception module V-1502 may be configured to receive the fused environment data calculated by the location fusion module V-1501, and perform computer vision processing on it to identify objects and/or features in the environment where the vehicle is located, the objects and/or Features include, for example, lane markings, pedestrians, other vehicles, traffic signals, infrastructure, etc.
  • the perception module V-1502 can use object recognition algorithm, structure from motion (SFM) algorithm, video tracking or other computer vision technology. In some possible designs, the perception module V-1502 may be further configured to map the environment, track objects, estimate the speed of objects, and the like.
  • the driving state determination module V-1503 identifies the driving state of the vehicle based on the data obtained by the internal sensor V-122 in the sensor system V-120, including vehicle speed, acceleration or yaw rate, for example.
  • the task receiving module V-1507 can be configured to receive the task, analyze the loading and unloading address, cargo category, loading and unloading time and other information contained in the task, and send this information to the navigation module V-1504.
  • the navigation module V-1504 can be configured as any unit that determines the driving route of the vehicle.
  • the navigation module V-1504 can be further configured to dynamically update the driving route during the operation of the vehicle.
  • the navigation module V-1504 can be configured to be based on the processing results from the positioning fusion module V-1501, the positioning sensor, the object sensor V1212, the task receiving module V-1507 and one or more pre-stored high-precision Map data to determine the driving route for the vehicle.
  • the decision-making module V-1505 can be configured to be based on the driving route calculated by the navigation module V-1504, the vehicle position data calculated by the positioning fusion module V-1501, and the objects in the environment where the vehicle is identified by the perception module V-1502 and/or features, generating waypoint information of the vehicle, where the waypoint in the waypoint information is the track point where the vehicle is moving forward in the driving path.
  • the travel control module V-1506 can be configured to receive the waypoint information generated by the decision module V-1505, and control the actuation system V-130 according to the waypoint information, so that the vehicle travels according to the waypoint information.
  • Data storage device V-152 may also be configured to store other instructions, including sending data to power system V-110, sensor system V-120, actuation system V-130, and/or peripherals system V-140. One or more instructions from which to receive data, interact with it, and/or control it. Data storage device V- 152 may also be configured to store other instructions. For example, the data storage device V-152 may store instructions for controlling the operation of the transmission V113 to improve fuel efficiency, may store instructions for controlling the image sensor V1213 to capture an image of the environment, and may store instructions for controlling the data sensed by the object sensor V1212. Instructions for generating a three-dimensional image of the environment in which the vehicle is located, and instructions for recognizing electrical signals converted by the microphone V- 144 into voice commands may be stored.
  • Data storage device V- 152 may also be configured to store other instructions. In addition to storing instructions, the data storage device V-152 can also be configured to store various information, such as image processing parameters, training data, high-definition maps, route information, and the like. During the operation of the vehicle in automatic mode, semi-automatic mode, manual mode, this information can be used by powertrain system V-110, sensor system V-120, actuation system V-130 and peripheral equipment system V-140, vehicle computing system V-140 One or more of 150 are used.
  • Vehicle computing system V-150 may be communicatively coupled to one or more of powertrain system V-110, sensor system V-120, actuation system V-130, and peripheral system V-140 via a system bus, network, and/or other connection mechanism Multiple.
  • the vehicle computing system V-150 can be directly connected to the wireless communication device V-141 in the peripheral equipment system V-140 through the data line or wirelessly through the wireless communication technology, and then wirelessly connected to the hub service area and the /or central control system.
  • Vehicle computing system V- 150 may also be a plurality of computing devices that distribute control of individual components or individual systems of the vehicle.
  • Vehicle computing system V- 150 may additionally or alternatively include other components.
  • FIG. 7 presents a functional block diagram of the self-driving vehicle 100, and the vehicle computing system V-150 in the self-driving vehicle 100 is introduced below.
  • FIG. 8 is a schematic structural diagram of a vehicle computing system V-150 provided by an embodiment of the present application.
  • the vehicle computing system V-150 includes a processor E-100 coupled to a system bus E-000.
  • Processor E-100 may be any conventional processor, including a Reduced Instruction Set Computing (RISC) processor, a Complex Instruction Set Computing (CISC) processor, or a combination thereof.
  • processor E-100 may be a dedicated device such as an Application Specific Integrated Circuit (ASIC).
  • the processor E-100 may be one or more processors, wherein each processor may include one or more processor cores.
  • the system memory E-900 is coupled with the system bus E-000.
  • the data running in the system memory E-900 may include the operating system E-901 and application programs E-904 of the vehicle computing system V-150.
  • the operating system E-901 includes a shell (Shell) E-902 and a kernel (kernel) E-903.
  • the shell E-902 is an interface between the user and the kernel E-903 of the operating system, and is the outermost layer of the operating system.
  • Shell E-902 manages the interaction between the user and the operating system, waits for user input, interprets user input to the operating system, and processes various operating system output results.
  • the kernel E-903 consists of those parts of the operating system E-901 for managing memory, files, peripherals and system resources. Directly interacting with hardware, the operating system kernel usually runs processes and provides inter-process communication, providing CPU time slice management, interrupts, memory management, I/O management, and so on.
  • the application program E-904 includes an automatic driving related program E-905, such as a program for managing the interaction between the automatic driving vehicle 100 and obstacles on the road, a program for controlling the driving route or speed of the automatic driving device, and controlling the automatic driving vehicle 100 and other automatic driving vehicles on the road Programs for device interaction.
  • Application E-904 also exists on the system of the software deployment server. When the application E-904 needs to be executed, the vehicle computing system V-150 can download the application E-904 from the software deployment server.
  • System bus E-000 is coupled via bus bridge E-200 and I/O bus E-300.
  • the I/O bus E-300 is coupled with the I/O interface E-400.
  • the I/O interface E-400 is connected with the USB interface E-500 to communicate with various I/O devices, such as input devices, media disks, transceivers, cameras, sensors, etc.
  • the input device is such as keyboard, mouse, touch screen, etc.
  • the media disk is such as CD-ROM, multimedia interface, etc.
  • the transceiver is used to send and/or receive radio communication signals
  • the camera is used to capture scenery and dynamic digital video images; It can be various sensors included in the sensing system in FIG. 7 , used to detect the environment around the vehicle computing system V-150 and provide the sensed information to the vehicle computing system V-150.
  • the hard disk drive E-800 is coupled to the system bus E-000 via a hard disk drive interface.
  • the display adapter E-700 is coupled with the system bus E-000 to drive the display.
  • the vehicle computing system V-150 can communicate with the software deployment server through the network interface E-600.
  • the network interface E-600 is a hardware network interface, such as a network card.
  • the network may be an external network such as the Internet, an internal network such as Ethernet or a virtual private network (VPN), or a wireless network such as a WiFi network or a cellular network.
  • VPN virtual private network
  • Vehicle computing system V-150 may include an on-board execution device that may include one or more first processors, one or more first memories, and The computer instructions to run.
  • the first processor executes the functions corresponding to the on-vehicle execution device in various embodiments provided by the present application.
  • the first processor can be configured as one or more general-purpose processors (such as CPU, GPU) in the processor V-151, one or more special-purpose processors (such as ASIC), one or more field programmable A gate array (FPGA), one or more digital signal processors (DSP), one or more integrated circuits, and/or, one or more microcontrollers, etc.
  • the first memory may be configured as one or more read-only memories (ROMs), one or more random-access memories (RAMs), one or more flash memories, one or more An electrically programmable memory (EPROM), one or more electrically programmable and erasable memories (EEPROM), one or more embedded multimedia cards (eMMC), and/or, one or more hard drives, etc.
  • ROMs read-only memories
  • RAMs random-access memories
  • EPROM electrically programmable memory
  • EEPROM electrically programmable and erasable memories
  • eMMC embedded multimedia cards
  • hard drives etc.
  • the functions corresponding to the vehicle-mounted execution device can be implemented as a computer program product, and when the computer program product is run on a computer, the functions corresponding to the vehicle-mounted execution device are realized.
  • the computer program product for realizing the corresponding function may be stored in the first memory.
  • FIG. 9 shows a possible example of an automatic driving vehicle and a vehicle-mounted execution device.
  • the self-driving vehicle 100 is equipped with a vehicle-mounted execution device 50, and the vehicle-mounted execution device 50 includes a first processor 50A, A first memory 50B, and computer instructions stored on the first memory and executable on the first processor.
  • the first processor When the first processor is running the computer instructions in the first memory, execute the method corresponding to the following steps: S91, obtain the reasoning result through the vehicle-side model; S92, collect the performance improvement of the vehicle-side model according to the reasoning result of the vehicle-side model Valuable data; S93, iterating the in-use car-end model into a trained car-end model; wherein, the trained car-end model is valuable to the performance improvement of the car-end model by the cloud execution device
  • the model obtained by optimizing the data and using the optimization results to train the car-side model.
  • the specific implementation form of the on-vehicle execution device 50 may also be other electronic devices with similar memory and processor architecture, such as an automatic driving domain controller.
  • the embodiment of the present application also provides a cloud execution device.
  • the cloud execution device 60 may include one or more second processors 60A, one or more second memories 60B, and Computer instructions stored on the second memory and executable on the second processor.
  • the second processor When the second processor is running the computer instructions in the second memory, it executes the functions corresponding to the cloud execution device in various embodiments provided by the present application.
  • the second processor can be configured as one or more general-purpose processors (such as CPU, GPU), one or more special-purpose processors (such as ASIC), one or more field programmable gate arrays (FPGA), one or multiple digital signal processors (DSPs), one or more integrated circuits, and/or, one or more microcontrollers, and the like.
  • the second memory may be configured as one or more read only memories (ROM), one or more random access memories (RAM), one or more flash memories, one or more electrically programmable memories (EPROM), One or more electrically programmable and erasable memories (EEPROM), one or more embedded multimedia cards (eMMC), and/or, one or more hard drives, etc.
  • the functions corresponding to the cloud execution device can be realized as a computer program product, and when the computer program product is run on a computer, the functions corresponding to the cloud execution device are realized.
  • the computer program product for realizing the corresponding function may be stored in the second memory.
  • FIG. 10 shows a possible example of a cloud execution device 60, including a second processor 60A, a second memory 60B, and computer instructions stored in the second memory and executable on the second processor.
  • the second processor is running the computer instructions in the second memory, execute the method corresponding to the following steps: S101, collect data valuable to the performance improvement of the vehicle end model; S102, valuable to the performance improvement of the vehicle end model optimize the data, and use the optimization results to train the vehicle-side model; S103, iterate the vehicle-side model being used by the vehicle-mounted execution device into a trained vehicle-side model.
  • the specific implementation form of the cloud execution device 60 may also be other electronic devices with similar memory and processor architectures.
  • the disclosed devices and methods may be implemented in other ways.
  • the device embodiments described above are only illustrative.
  • the division of the modules or units is only a logical function division. In actual implementation, there may be other division methods.
  • multiple units or components can be Incorporation or may be integrated into another device, or some features may be omitted, or not implemented.
  • the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces, and the indirect coupling or communication connection of devices or units may be in electrical, mechanical or other forms.
  • the unit described as a separate component may or may not be physically separated, and the component displayed as a unit may be one physical unit or multiple physical units, that is, it may be located in one place, or may be distributed to multiple different places . Part or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.
  • each functional unit in each embodiment of the present application may be integrated into one processing unit, each unit may exist separately physically, or two or more units may be integrated into one unit.
  • the above-mentioned integrated units can be implemented in the form of hardware or in the form of software functional units.
  • the integrated unit is realized in the form of a software function unit and sold or used as an independent product, it can be stored in a readable storage medium.
  • the technical solution of the embodiment of the present application is essentially or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the software product is stored in a storage medium Among them, several instructions are included to make a device (which may be a single-chip microcomputer, a chip, etc.) or a processor (processor) execute all or part of the steps of the methods described in the various embodiments of the present application.
  • the aforementioned storage media include: U disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), magnetic disk or optical disc and other media that can store program codes. .

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Geometry (AREA)
  • General Physics & Mathematics (AREA)
  • Evolutionary Computation (AREA)
  • General Engineering & Computer Science (AREA)
  • Computer Hardware Design (AREA)
  • Software Systems (AREA)
  • Artificial Intelligence (AREA)
  • Computational Mathematics (AREA)
  • Pure & Applied Mathematics (AREA)
  • Mathematical Optimization (AREA)
  • Mathematical Analysis (AREA)
  • Aviation & Aerospace Engineering (AREA)
  • Computational Linguistics (AREA)
  • Data Mining & Analysis (AREA)
  • Computing Systems (AREA)
  • Mathematical Physics (AREA)
  • Automation & Control Theory (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Medical Informatics (AREA)
  • Traffic Control Systems (AREA)

Abstract

一种模型自动化迭代方法、设备及存储介质，所述方法包括：使用车端计算资源通过车端模型获得推理结果；根据车端模型推理的结果，采集对车端模型性能提升有价值的数据；使用云端计算资源，对所述对车端模型性能提升有价值的数据进行优化，并利用优化结果训练车端模型；将车端计算资源正在使用的车端模型迭代为训练后的车端模型。本申请采用车端推理--云端训练的模式，有针对性地自动采集有价值数据，并自动化完成数据优化，充分发挥了云端的资源优势，提高了自动驾驶车端模型迭代的效率。

Description

一种模型自动化迭代方法、设备及存储介质 技术领域
本申请涉及自动驾驶技术领域，尤其涉及一种模型自动化迭代方法、设备及存储介质。
背景技术
自动驾驶长尾场景是指突发的、低概率的、不可预知的场景，比如交通信号灯故障的路口、醉驾的车辆、路中央的气球等。怎样应对长尾场景一直是行业难题，已逐渐成为制约自动驾驶发展的关键，要解决这些问题，自动驾驶系统需要积累大量的数据，持续优化模型。
据统计，全球汽车保有量超过10亿辆，但平均每30s发生一起事故，说明交通事故属于低频出现的事件。要实现自动驾驶落地，就要解决这些低频发生的问题，至少要达到现有人类驾驶员的安全驾驶水平，甚至全面超越人类驾驶员。
根据以上数据，为了充分测试自动驾驶系统的安全性，至少需要进行上亿公里的道路测试，这意味着数万乃至几十万辆车24小时不间断运行几百天，同时，测试过程中产生有效问题数据的效率很低，这些原因导致模型的迭代和验证的成本越来越高。
发明内容
传统的模型迭代验证方式采用以功能测试驱动模型迭代的方式，在开发端以需求和问题驱动数据采集，然后人工分析和标注数据并设计优化方案，在测试端人工搭建场景进行测试或实车随机测试，最终形成标注、开发、测试的串行迭代流程，这种方法对于软件功能开发是有效的，有限的人力解决有限的问题，实现特定范围内的功能。
但传统的模型迭代验证方式难以让自动驾驶真正落地，使整个行业做到全时段全工况的安全运行。一方面，传统问题驱动的方式是依靠串行的开发模式对模型进行优化，开发和测试的周期长，无法并行开展。另一方面，人工标注数据的方式耗时长，标注效率低；第三方面，测试大多通过人工搭建典型场景或随机测试对模型进行验证，对于实际运行场景的覆盖率较低。以上几个方面说明问题驱动的方式已经无法满足解决真实场景中海量问题的需求，不能自动化的解决绝大部分的问题，无法高效的实现自动驾驶落地目标。
在上述背景下，开发一种快速模型优化和验证的方法以有效解决模型迭代周期长、验证效率低等现实问题，成为本领域亟待解决的技术问题。
本申请实施例旨在至少解决上述技术问题之一。
第一方面，本申请实施例提供一种模型自动化迭代方法，包括：
使用车端计算资源通过车端模型获得推理结果；
根据车端模型推理的结果，采集对车端模型性能提升有价值的数据；
使用云端计算资源，对所述对车端模型性能提升有价值的数据进行优化，并利用优化结果训练车端模型；
将车端计算资源正在使用的车端模型迭代为训练后的车端模型。
第二方面，本申请实施例提供一种车端模型自动化迭代方法，包括：
通过车端模型获得推理结果；
与云端计算资源相配合，根据车端模型推理的结果，采集对车端模型性能提升有价值的数据；
与云端计算资源相配合，将正在使用的车端模型迭代为训练后的车端模型；其中，所述训练后的车端模型是由云端计算资源通过对所述对车端模型性能提升有价值的数据进行优化、并利用优化结果对车端模型进行训练得到的模型。
第三方面，本申请实施例提供一种云端模型自动化迭代方法，包括：
与车端计算资源相配合，根据车端计算资源通过车端模型推理的结果，采集对车端模型性能提升有价值的数据；
对所述对车端模型性能提升有价值的数据进行优化，并利用优化结果训练车端模型；
与车端计算资源相配合，将车端计算资源正在使用的车端模型迭代为训练后的车端模型。
第四方面，本申请实施例提供一种车端执行设备，包括：
车端计算模块，配置有车端模型，通过车端模型推理得到结果；
车端采集模块，用于与云端执行设备相配合，根据车端模型推理的结果，采集对车端模型性能提升 有价值的数据；其中，所述云端执行设备对所述对车端模型性能提升有价值的数据进行优化，并利用优化结果训练车端模型；
所述车端计算模块还用于与云端执行设备相配合，将配置的车端模型迭代为训练后的车端模型。
第五方面，本申请实施例提供一种云端执行设备，包括：
云端采集模块，用于与车端执行设备相配合，根据车端执行设备通过车端模型推理的结果，采集对车端模型性能提升有价值的数据；
数据优化模块，用于对所述对车端模型性能提升有价值的数据进行优化；
训练模块，用于利用优化结果训练车端模型；
迭代模块，用于与车端执行设备相配合，将车端执行设备正在使用的车端模型迭代为训练后的车端模型。
第六方面，本申请实施例提供一种电子设备，其包括：至少一个处理器，以及与所述至少一个处理器通信连接的存储器，其中，所述存储器存储有可被所述至少一个处理器执行的指令，所述指令被所述至少一个处理器执行，以使所述至少一个处理器能够执行前述车端模型自动化迭代方法的步骤。
第七方面，本申请实施例提供一种自动驾驶车辆，包括前述电子设备。
第八方面，本申请实施例提供一种存储介质，其上存储有计算机程序，该程序被处理器执行时实现前述车端模型自动化迭代方法的步骤。
第九方面，本申请实施例提供一种电子设备，包括：至少一个处理器，以及与所述至少一个处理器通信连接的存储器，其中，所述存储器存储有可被所述至少一个处理器执行的指令，所述指令被所述至少一个处理器执行，以使所述至少一个处理器能够执行前述云端模型自动化迭代方法的步骤。
第十方面，本申请实施例提供一种存储介质，其上存储有计算机程序，该程序被处理器执行时实现前述云端模型自动化迭代方法的步骤。
本申请提供的模型自动化迭代方法采用车端推理--云端训练的模式，即在车端部署多任务、轻量级的车端模型，基于车端模型推理的结果自动地、有针对性地采集对车端模型的性能提升有价值的数据，再利用云端强大的计算能力和数据存储能力，实时地、自动地完成训练数据集生成、模型训练、模型迭代等一系列操作；这种车端推理--云端训练的模式充分发挥了云端的资源优势，提高了自动驾驶车辆模型迭代的效率。
本申请提供的方法在车端至云端通信资源有限的环境下，自动采集对车端模型的性能提升有价值的数据，该自动采集过程不仅效率高，而且涵盖了罕见、异常、突发的长尾场景，屏蔽了重复数据和垃圾数据，保证了所采集数据的有效性、多样性和完整性，为云端自动化完成模型训练和模型迭代提供了充足、高质、多样、有效、可靠的数据基础。
本申请提供的方法自动化地基于对车端模型的性能提升有价值的数据生成训练数据集，数据优化模块可利用单任务、深层次的数据驱动模型自动化完成数据标注获得训练数据集，也可利用数学规划模型快速、有效地得到满足要求的训练数据集，这种自动生成训练数据集的方式大大减少了人工标注工作，明显有助于解决因为标注效率低导致的模型迭代耗时长、速度慢的问题。
附图说明
为了更清楚地说明本申请实施例的技术方案，下面将对实施例描述中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图是本申请的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他的附图。
图1为本申请实施例提供的模型自动化迭代系统的一种系统架构图；
图2为本申请实施例提供的模型自动化迭代方法的一种流程示意图；
图3为车端采集模块采集对车端模型性能提升有价值的数据的一种示例；
图4为场景所包含要素类型的一种示例；
图5为车端采集模块采集对车端模型性能提升有价值的数据的一种示例；
图6为行为预测模型的推理结果及其期望值的一种示意图；
图7为自动驾驶车辆的一种结构示意图；
图8为车辆计算系统的一种结构示意图；
图9为自动驾驶车辆和车载执行设备的一种可能的示例；
图10为云端执行设备的一种结构示意图；
图11为本申请实施例提供的模型自动化迭代方法的另一种流程示意图；
图12为本申请实施例提供的模型自动化迭代方法的又一种流程示意图；
图13为本申请实施例提供的模型自动化迭代方法的再一种流程示意图；
图14为将车端计算资源正在使用的车端模型迭代为训练后的车端模型的一种可能的示例的流程示意图；
图15为将云端计算资源正在使用的数据驱动模型迭代为训练后的数据驱动模型的一种可能的示例的流程示意图。
具体实施方式
本申请的说明书和权利要求书及上述附图中的术语“第一”、“第二”等是用于区别类似的对象，而不必用于描述特定的顺序或先后次序。应该理解这样使用的术语在适当情况下可以互换，这仅仅是描述本申请的实施例中对相同属性的对象在描述时所采用的区分方式。此外，术语“包括”和“具有”以及他们的任何变形，意图在于覆盖不排他的包含，以便包含一系列单元的过程、方法、系统、产品或设备不必限于那些单元，而是可包括没有清楚地列出的或对于这些过程、方法、产品或设备固有的其它单元。
下面结合附图，对本申请的实施例进行描述。本领域普通技术人员可知，随着技术的发展和新场景的出现，本申请实施例提供的技术方案对于类似的技术问题，同样适用。
在对本申请实施例提供的模型自动化迭代方法进行详细介绍之前，先结合图1对本申请实施例提供的模型自动化迭代系统进行介绍。请先参阅图1，图1为本申请实施例提供的模型自动化迭代系统的一种系统架构图，在图1中，模型自动化迭代系统300包括：车端执行设备20、车端数据存储单元30、云端执行设备10和云端数据库40。
车端执行设备20包括：车端采集模块20A、车端计算模块20B。车端计算模块中配置有车端模型20C。
车端执行设备可应用于自动驾驶车辆中，其中，自动驾驶车辆设置有至少一种传感器，如车载雷达(如毫米波雷达、红外雷达、激光雷达、多普勒雷达等)、光量传感器、雨量传感器、视觉传感器(如摄像头、行车记录仪)、车姿传感器(如陀螺仪)、速度传感器(如多普勒雷达)、惯性测量单元IMU等。
车端采集模块具有数据采集功能，并将采集的数据发送给上位机进行分析、处理，可用于采集自动驾驶车辆上设置的各类传感器采集的模拟或数字信号，也可用于采集车端计算模块通过车端模型推理的结果，也可以用于采集车辆状态数据、地图数据、驾驶员操作数据等。车端采集模块内置数据采集卡(即实现数据采集功能的计算机扩展卡)，可以通过USB、PXI、PCI、PCI Express、火线(1394)、PCMCIA、ISA、Compact Flash、485、232、以太网、各种无线网络等总线采集和发送数据。
车端采集模块还具有数据处理功能，具体为与云端采集模块相配合，从采集的数据中提取对车端模型性能提升有价值的数据。
车端数据存储单元具有数据存储功能，可用于存储前述各类传感器采集的信号、车端模型推理的结果、车辆状态数据、地图数据、驾驶员操作数据，还可用于存储操作系统、应用程序等。在一种可能的设计中，车端数据存储单元可采用嵌入式多媒体卡(eMMC)、单层单元闪存(SLC NAND)、通用闪存(UFS)、固态硬盘(SSD)等实现。在一种可能的设计中，车端数据存储单元可以设置于车端执行设备中，也可以是车端执行设备以外的外部设备。
车端模型具有推理功能，可用于为自动驾驶车辆实现目标检测、行为预测、决策规划等功能。在一种可能的设计中，车端模型可以是神经网络类型的模型或非神经网络类型的模型，本申请实施例中仅以车端模型为神经网络类型的模型为例。
车端计算模块获取传感器数据、车辆状态数据、地图数据、驾驶员操作数据等，然后将这些数据作为车端模型的输入数据，利用车端模型进行推理，实现自动驾驶车辆的目标检测、行为预测、决策规划等功能。
云端执行设备10包括：云端采集模块10A、数据优化模块10B、训练模块10C、迭代模块10D。
云端执行设备可由云服务器实现。车端执行设备与云端执行设备之间通过通信接口实现数据传输，该通信接口可以采用车用无线通信技术V2X、车载以太网、3G/4G/5G移动通信技术等进行通信。
云端采集模块具有数据采集功能，并将采集的数据发送给上位机进行分析、处理。云端采集模块与车端采集模块之间具有数据传输关系，云端采集模块根据需求从车端采集模块获取数据。云端采集模块内置数据采集卡，可以通过USB、PXI、PCI、PCI Express、火线(1394)、PCMCIA、ISA、Compact Flash、485、232、以太网、各种无线网络等总线采集和发送数据。
云端采集模块还具有数据处理功能，具体为与车端采集模块相配合，采集对车端模型性能提升有价 值的数据。
云端数据库具有数据存储功能，可采用云存储技术、云数据库技术等实现。
数据优化模块具有数据处理功能，可实现数据标注、数据拟合、数据降噪等数据处理功能。
训练模块利用数据优化模块处理得到的优化结果训练车端模型。
迭代模块利用训练模块训练后的车端模型，对车端执行设备正在使用的车端模型进行迭代更新。
本申请实施例所称的“车端计算资源”包括但不限于车端执行设备、车端数据存储单元，还可以包括其他设置于自动驾驶车辆上的计算资源。本申请实施例所称的“云端计算资源”包括但不限于云端执行设备和云端数据库，还可以包括其他基于云计算技术的资源。
结合上述描述，下面开始对本申请实施例提供的模型自动化迭代方法的具体实现流程进行描述。
具体的，请参阅图2，图2为本申请实施例提供的模型自动化迭代方法的一种流程示意图，本申请实施例提供的模型自动化迭代方法可以包括：
201，使用车端计算资源通过车端模型获得推理结果。
具体的，车端计算模块将传感器数据、车辆状态数据、地图数据、驾驶员操作数据等作为车端模型的输入数据，利用车端模型进行推理，实现自动驾驶车辆的目标检测、行为预测、决策规划等功能，得到目标检测结果、行为预测结果、决策规划结果等。
此后，车端模型推理的结果被存储至车端数据存储单元中，并被车端采集模块获取。在一种可能的设计中，车端采集模块可以直接从车端计算模块获得车端模型推理的结果，也可以从车端数据存储单元中获得车端模型推理的结果。
根据车端模型对应于不同的功能，车端模型推理的结果可以具有如下形式：
(A)目标检测结果，可以包括：基于图像或激光点云的目标检测框、目标类型、置信度等。
(B)行为预测结果，可以包括：目标行为、目标意图、运动轨迹等；其中，目标行为可以是变道、超车、加减速、左转、右转、掉头、泊车等；目标意图可以是换道、直行、超车、横穿马路、泊车等；运动轨迹可以是目标在未来多个时间点的位置。
(C)决策规划结果，可以包括：自车行为决策、自车轨迹规划等；自车行为决策可以是变道、超车、加减速、循迹、左转、右转、掉头、模式切换、泊车等；自车轨迹规划可以是路径规划、速度规划、纵横向耦合规划、轨迹跟踪、跟车规划、换道轨迹规划、掉头轨迹规划、原地转向轨迹规划、脱困轨迹规划等。
考虑到车端计算资源具有成本高、算力有限、推理速度快等特点，车端模型可采用具有多任务、轻量级等特征的网络结构。其中，多任务是指网络结构具有共享参数、共享任务的特点，轻量级是指网络结构具有在有限存储空间和功耗限制的情况下满足计算效率和能力的特点。
例如：
当车端模型是基于视觉的目标检测模型，多任务是指可以复用图像的特征信息，通过一次模型推理得到多种任务所需的结果，例如同时检测行人、车辆和信号灯等；轻量级可以适配车端有限的算力，并满足车端推理效率；
当车端模型是基于激光点云的目标检测模型，多任务是指可以复用点云的特征信息，通过一次模型推理得到多种任务所需的结果，例如同时检测行人、车辆类别和障碍物动静态属性等；轻量级可以适配车端有限的算力，并满足车端推理效率；
当车端模型是行为预测模型，多任务是指可以复用感知融合结果和地图的特征信息，通过一次模型推理得到多种任务所需的结果，例如同时预测障碍物行为类别、未来路径形状和未来速度变化趋势等；轻量级可以适配车端有限的算力，并满足车端推理效率；
当车端模型是决策规划模型，多任务是指可以复用感知融合结果、地图和行为预测的特征信息，通过一次模型推理得到多种任务所需的结果，例如同时给出期望行为类别、期望路径形状和期望速度配置等；轻量级可以适配车端有限的算力，并满足车端推理效率。
此外，车端模型还可以采用具有多维度特点的网络结构，例如当车端模型是基于视觉的目标检测模型，多维度的网络结构可实现挖掘多目标之间内在联系的目的。
202，根据车端模型推理的结果，采集对车端模型性能提升有价值的数据。
具体的，车端采集模块与云端采集模块相配合，根据车端模型推理的结果，采集对车端模型性能提升有价值的数据。
目前自动驾驶领域常用的模型训练迭代方式是将模型推理的结果全部用于后续的模型训练中去，这种方式并不分辨模型对于场景的推理效果(学习效果)，也就是不论模型对某一场景的推理效果是否足够 好或者还不够好，都统一将推理结果继续用于模型训练，这种不区分重点的训练，不能快速、有针对性地实现训练目的。对于常见的典型场景还能适应，但对于罕见、突发、异常的长尾场景，这种模型迭代方式就会很难适应。
为了克服常规模型训练迭代方式的缺陷，本申请有针对性地采集对车端模型性能提升有价值的数据，再利用这些对车端模型性能提升有价值的数据对车端模型进行训练和迭代，这种方式可根据训练目的有针对性地提取有价值数据，从而快速、有效地达到训练目标。
具体的，对车端模型性能提升有价值的数据，不仅包括车端模型推理结果本身，还应包括与车端模型推理结果在时间及空间上存在同步关系的环境数据、地图数据、车身状态数据、驾驶员操作数据等时空同步信息，这些信息与车端模型推理的结果结合在一起，能够全面地反映自动驾驶车辆所处的场景，对于训练模型也更有意义。
其中，环境数据可以包括：静态环境(固定障碍物、建筑设施、交通设施、道路)、动态环境(动态交通信号灯、交通警察)、通信环境(信号强度、信号延迟时间、电磁干扰强度)、交通参与者(行人、机动车辆、非机动车辆、动物)、气象环境(温度、湿度、光照条件、天气情况)等；
地图数据可以包括：高精地图、交通管制信息、导航信息等；
车辆状态数据可以包括：车辆基础属性(如车身重量、几何尺寸、基本性能)、车辆位置(坐标、车道位置)、运动状态(横向运动状态、纵向运动状态)、人机交互(娱乐、驾驶任务)等；
驾驶员操作数据可以包括：是否接管车辆、驾驶员具体动作等。
根据不同的训练目的，采集对车端模型性能提升有价值的数据，可以包括如下几种情况：
(1)训练目的为令车端模型涵盖(适应)尽可能多的场景
如图3所示，车端采集模块利用车端模型推理的结果及其时空同步信息构建场景，并上传给云端，云端采集模块确定已有场景库中缺少车端采集模块上传的场景时，将车端模型推理的结果及其时空同步信息作为对车端模型性能提升有价值的数据进行采集。
在自动驾驶测试领域，场景是指在一定的时间和空间范围内，自动驾驶车辆与行驶环境中的其它车辆、道路、交通设施、气象条件等元素综合交互过程的总体动态描述，是自动驾驶车辆的驾驶情景与行驶环境的有机组合，既包括各类实体元素，也涵盖了实体执行的动作及实体之间的连接关系。图4所示为一种场景所包含要素类型的实施例。
具体的，云端数据库中存储有场景库，其中包含有车端模型涵盖(适应)的各种场景，若云端采集模块将车端采集模块上传的场景与场景库中已有的场景进行比对后发现场景库中没有该场景，则说明车端模型还不能涵盖(适应)这一场景，需要将这一场景加入到场景库中去，此时，云端采集模块下发命令，车端采集模块收到命令后，即将这一场景对应的车端模型推理结果及其时空同步信息作为对车端模型性能提升有价值的数据进行采集。
在一种可能的设计中，云端采集模块将车端采集模块上传的场景与场景库进行比对时，如果有如下两种情况出现，均可以算作场景库中缺少该场景：
(1.1)已有场景库中缺少该场景对应的类别
这种情况直接表明场景库还没涵盖该场景对应的类别，例如，场景库中道路类型下涵盖城市道路、高速公路、园区道路三个类别，而车端采集模块上传的场景类别为农村道路，此时，就可以确定场景库中缺少该场景；
(1.2)已有场景库中存在该场景对应的类别，但已有场景库中该类别下的数据量未达到预定数量
这种情况表明虽然场景库中已经涵盖了该场景，但是该场景对应的数据量还较少，而模型训练需要足够多的数据量，这时仍然认为场景库中缺少该场景，需要将场景对应的车端模型推理结果及其时空同步信息作为对车端模型性能提升有价值的数据上传给云端采集模块。例如，场景库中道路类型下的农村道路这一类别仅有10条数据，还远达不到有效训练模型的目标，因此，如果再有农村道路场景上传时，还需要继续记录到场景库中。
考虑到场景包含的信息量较大，完整上传场景不仅浪费通信资源，还会影响采集效率，然而实际上并非所有场景都是对车端模型性能提升有价值的(场景库中可能已经包含有该场景)，这种情况下，为了节省通信资源、加快数据采集效率，可以由车端采集模块对场景进行编码，并上传场景编码给云端，而云端数据库中除了存储有场景库之外，还可以存储场景库对应的编码库(其中包含有场景库中每个场景对应的场景编码)，云端采集模块将车端采集模块上传的场景编码与编码库进行比对，当确定编码库中没有该场景编码时，就可以确定车端模型还不能涵盖(适应)这一场景，需要将这一场景加入到场景库中去，此时，云端采集模块下发命令，车端采集模块接收命令后，即将对应的车端模型推理结果及其时空 同步信息作为对车端模型性能提升有价值的数据进行采集。
具体的，车端采集模块应按照预定的编码规则对场景进行编码，在一种可能的设计中，预定的编码规则可以是按照场景要素编码。
例如，针对图4所示的场景，对场景要素按照在父节点要素中的排序进行编码，对于每个具体的要素，#后面的数字表示当前要素在其父节点要素中的排序：
若场景包含行人，则按照从左往右，外部环境要素对应的编码为2，交通参与者对应的编码为3，行人对应的编码为2，则场景编码相应的包含数字232；
若场景包含横向运动状态，则按照从左往右，车辆自身要素对应的编码为1，运动状态对应的编码为3，横向运动状态对应的编码为1，则场景编码相应的包含数字131；
若场景同时包含行人、横向运动状态，则场景编码相应的包含数据组(232、131)。
(2)训练目的为令车端模型涵盖(适应)罕见、突发、异常等长尾场景
如图5所示，车端采集模块监测到车端模型推理的结果和/或时空同步信息不属于常规场景时，将车端模型推理的结果及其时空同步信息作为对车端模型性能提升有价值的数据进行采集。
这里的常规场景是指物理世界中普遍存在的、常见的交通场景，例如车辆在道路上正常行驶，道路中出现交通信号灯、交通指示牌、车道线、路肩等常规交通设施。与之相反的是长尾场景，即鲜少或几乎不可能在物理世界中出现的、罕见、突发、异常的交通场景，例如，车辆在天空\花坛\建筑物上行驶，道路中突然出现野兽\建筑物\大型漂浮物(例如气球)等。长尾场景对于自动驾驶车辆来说，常常意味着危险系数高、操作处理复杂，要想应对长尾场景，就需要提升车端模型面对长尾场景时的推理能力，相应的，长尾场景对应的各种信息就属于对车端模型性能提升有价值的数据。
当监测到车端模型推理的结果和/或时空同步信息不属于常规场景时，说明自动驾驶车辆处于罕见的、或者突发的、或者异常的长尾场景中，需要将此时的推理结果及其时空同步信息作为对车端模型性能提升有价值的数据进行采集。
在一种可能的设计中，基于激光点云的目标检测模型推理的结果为目标车辆从某一帧开始在道路一侧的建筑物上行驶，并且这种情况持续多帧，而在这帧之前的监测结果一直是目标车辆在道路上行驶的，这一推理结果(车辆在建筑物上行驶)不属于常规场景(车辆在道路上行驶)，这种情况的出现可能是基于激光点云的目标检测模型的推理出现了错误，也有可能是激光雷达故障，甚至也可能是目标车辆真的在花坛上行驶等，这些异常或罕见场景都属于长尾场景，需要车端模型去涵盖，因此需要将此时的推理结果及其时空同步信息作为对车端模型性能提升有价值的数据进行采集，用于后续的模型训练。
在另一种可能的设计中，车辆行驶过程中，突然出现一群大象要横穿道路，或者道路上飘来一个气球，或者道路中央有一个房子(例如钉子户)，这些车辆行驶中遇到的突发情况作为时空同步信息(环境信息)被记录下来，并被判断为不属于常规场景，而是罕见的、异常的场景(长尾场景)，需要车端模型去涵盖，因此需要将此时的推理结果及其时空同步信息作为对车端模型性能提升有价值的数据进行采集，用于后续的模型训练。
(3)训练目的为令车端模型在自身推理效果不够好的场景中具备更好的推理能力
具体的，如图5所示，如下两种情况均表明车端模型的推理效果不够好，需要提升车端模型面对相应场景时的推理能力：
(3.1)车端模型推理的结果与期望值不匹配
如图6所示，在一种可能的设计中，具体示例为行为预测模型对目标在未来某一时间段的预测轨迹为图中虚线，而当该时间段经过后，基于多传感器(视觉传感器、激光雷达、毫米波雷达等)的融合结果显示目标的真实轨迹是图中实线，将图中实线作为期望值，根据图6可知，行为预测模型的推理结果与期望值不够匹配(实线与虚线的重叠度较低)，这种情况表明行为预测模型的推理结果出现异常，有可能是行为预测模型对过去一段时间内目标行为的学习效果(推理效果)不够好，还需要训练车端模型对这些目标行为的适应能力，因此需要将此时的推理结果及其时空同步信息作为对车端模型性能提升有价值的数据进行采集，用于后续的模型训练。
在另一种可能的设计中，决策规划模型的推理结果为控制车辆继续直行，然而驾驶员却突然接管车辆并控制车辆换道，这种情况很可能是决策规划模型推理的结果出现异常，不符合人类驾驶员在相同场景下的常见操作，说明决策规划模型对这种场景的学习效果还不够好，因此需要将此时的推理结果及其时空同步信息作为对车端模型性能提升有价值的数据进行采集，用于后续的模型训练。
在另一种可能的设计中，对决策规划模型的推理结果进行评价，其评价结果显示驾乘人员舒适度低于预定的舒适度阈值(期望值)，或者安全风险指数高于预定的风险阈值(期望值)，这种情况表明决策 规划模型对当前场景的学习效果还不够好，因此需要将此时的推理结果及其时空同步信息作为对车端模型性能提升有价值的数据进行采集，用于后续的模型训练。
(3.2)对基于不同算法逻辑得到的车端模型推理的结果进行一致性检验，检验结果未达到预定的一致性下限
在一种可能的设计中，基于图像进行的目标检测结果显示障碍物为动态障碍物，而基于激光点云和毫米波点云进行的目标检测结果均显示障碍物为静态障碍物，对这三种算法逻辑得到的结果进行一致性检验，检验结果为三者的一致性并不好(基于图像的是动态障碍物，基于激光点云和毫米波雷达的是静态障碍物)，没有达到预定的一致性下限(例如要求三者必须完全一致)，这种情况有可能是基于图像的目标检测模型的推理结果不够准确，也可能是基于激光点云和/或毫米波点云的目标检测模型的推理结果不够准确，这说明这三类算法逻辑的目标检测模型中，至少有一种目标检测模型对当前场景的推理效果不够好，需要提升该目标检测模型在当前场景中的推理能力，因此需要将此时的推理结果及其时空同步信息作为对车端模型性能提升有价值的数据进行采集，用于后续的模型训练。
在另一种可能的设计中，基于深度学习的决策规划给出的结果是控制车辆向左换道，但是基于规则的决策规划给出的结果是控制车辆向右换道，对这两种算法逻辑得到的结果进行一致性检验，检验结果为两者的一致性并不好，没有达到预定的一致性下限(例如要求二者必须完全一致)，这种情况有可能是基于深度学习的决策规划推理结果不够准确，也可能是基于规则的决策规划结果不够准确，这说明这两类算法逻辑的决策规划模型中，至少有一种决策规划模型对当前场景的推理效果不够好，需要提升该决策规划模型在当前场景中的推理能力，因此需要将此时的推理结果及其时空同步信息作为对车端模型性能提升有价值的数据进行采集，用于后续的模型训练。
(4)训练目的为令车端模型在自身推理效果非常好的场景中持续优良表现
具体的，如下两种情况下，均表明车端模型的推理效果非常好，需要继续保持这种推理能力：
(4.1)车端模型推理的结果与期望值相匹配，且匹配度达到预定的匹配阈值
这种情况需要令场景库中涵盖更多车端模型推理效果非常好的场景，例如，对于行为预测模型，若推理得出的目标预测轨迹与多传感器融合结果显示的目标真实轨迹重叠度相当好，其中，基于多传感器融合的目标真实轨迹为期望值，即行为预测模型对目标行为的推理结果与期望值的匹配度达到了较好的水平(例如达到了预定的匹配阈值)，这说明预测模型对目标行为的推理效果(学习效果)非常好，需要行为预测模型继续保持这种良好的推理能力，因此也需要将此时的推理结果及其时空同步信息作为对车端模型性能提升有价值的数据进行采集，用于后续的模型训练。
(4.2)对基于不同算法逻辑得到的车端模型推理的结果进行一致性检验，检验结果达到预定的一致性上限
在一种可能的设计中，基于图像进行的目标检测结果、基于激光点云进行的目标检测结果、基于毫米波点云进行的目标检测结果均显示障碍物为静态障碍物，对这三种算法逻辑得到的结果进行一致性检验，检验结果显示三者完全一致(均为静态障碍物)，达到了预定的一致性上限，这种情况就表明基于这三类算法逻辑的目标检测模型的推理效果很好，需要目标检测模型继续保持这种良好的推理能力，因此也需要将此时的推理结果及其时空同步信息作为对车端模型性能提升有价值的数据进行采集，用于后续的模型训练。
基于深度学习的决策规划给出的结果是控制车辆向左换道，基于规则的决策规划给出的结果也是控制车辆向左换道，对这两种算法逻辑得到的结果进行一致性检验，检验结果为两者完全一致，达到了预定的一致性上限，这种情况就表明基于深度学习的决策规划和基于规则的决策规划的推理效果很好，需要决策规划模型继续保持这种良好的推理能力，因此也需要将此时的推理结果及其时空同步信息作为对车端模型性能提升有价值的数据进行采集，用于后续的模型训练。
本申请提供的实施例，通过车端计算资源与云端计算资源的相互配合，实现自动化采集有利于车端模型性能提升的数据，该数据采集方式不仅速度快，而且有针对性，能够在车端与云端通信资源有限的情况下，更高效地采集有用数据，为后续训练车端模型提供有效、可靠的数据基础。
203，使用云端计算资源，对有价值数据(对车端模型性能提升有价值的数据)进行优化，并利用优化结果训练车端模型。
具体的，该步骤是由数据优化模块对有价值数据进行优化，获得有效、可靠的训练数据集(优化结果)，为下一步训练车端模型做准备。
在不同的设计中，数据优化模块可采用不同的算法处理方式优化有价值数据：
(1)在如图11所示的设计中，如步骤203’所示，数据优化模块利用数据驱动模型对有价值数据 标注标签，再利用标注数据集对车端模型进行训练从而提升其推理能力。
本申请实施例所称的“数据驱动模型”是指基于数据驱动的模型，例如可以是深度学习模型、传统机器学习模型等。
在一种可能的设计中，数据驱动模型是传统机器学习模型，可以采用支持向量机算法(SVM)、Adaboost算法、逻辑回归算法、隐马尔可夫算法、K近邻算法(KNN)、三层人工神经网络算法、贝叶斯算法、决策树算法等任意一种传统机器学习算法。
在一种可能的设计中，上述传统机器学习模型(如SVM或Adaboost)基于人为定义的方向梯度直方图(HOG)特征进行计算，实现对有价值数据标注标签的目的。
具体的，数据优化模块采用数据驱动模型对有价值数据进行自动化标注，获得有效、可靠的标注数据集，再利用标注数据集训练车端模型。
针对不同功能的车端模型，标签有不同的定义。例如，当车端模型为用于目标检测的模型，标签包括特定目标的检测框、类别、置信度等信息。
以车端模型为用于目标检测的模型为例，利用云端计算资源强大的运算能力和数据存储能力，在云端设置目标检测能力远高于车端模型的数据驱动模型，然后利用数据驱动模型识别有价值数据中的特定目标(如：行人、机动车、非机动车、交通标志牌、交通信号灯、车道线等)，由于数据驱动模型的识别能力远远高于车端模型，其识别结果相对车端模型的推理能力就可以作为标签用来训练车端模型。这种自动标注方式可节省大量的人工标注工作量，显著提升数据标注效率，加快模型迭代速度。
为了充分利用云端计算资源，获得能够有效训练车端模型的标注数据集，可将数据驱动模型设置为多个具有单任务、深层次特征网络结构的深度学习模型。其中，单任务特征是指单个模型仅用于执行单个任务，且模型与模型之间相互独立，并不共享参数，利用模型的单任务特征可最大程度挖掘个体目标召回与识别精度；深层次特征是指模型具有多个隐藏层，能够对输入特征进行多层次的抽象，更好的划分不同类型的数据，利用模型的深层次特征可提升个体目标在复杂路况下的目标召回与识别精度。
在上述用于目标检测的模型实施例中，可根据具体场景需要感知的特定目标设置多个单任务、深层次的数据驱动模型，例如，设置多个数据驱动模型，每个数据驱动模型分别用于检测行人、机动车、非机动车、交通标志牌、交通信号灯、车道线等。
(2)在如图12所示的设计中，如步骤203”所示，数据优化模块采用数学规划模型对有价值数据进行优化以获得期望值，再利用期望值对车端模型进行训练从而提升其推理能力。
具体的，数学规划模型采用的优化方法可以是线性规划、整数线性规划、非线性规划、多目标规划、动态规划等。
以车端模型为行为预测模型为例，如图6所示，行为预测模型对目标在未来某一时间段的预测轨迹为图中虚线，而当该时间段经过后，基于多传感器(视觉传感器、激光雷达、毫米波雷达等)感知融合的结果为图中实线，为了提升行为预测模型的推理能力(尽量与实际感知结果一致)，可将多传感器感知融合的结果作为期望值用来训练车端模型；然而基于多传感器感知融合的结果也是检测得到的，有可能与目标在真实世界中的表现有差异，例如，图6所示的实线由多个离散的轨迹点组成，其中的M点与其他各轨迹点连成的轨迹线相隔较远，属于跳变点，然而目标在真实世界中的轨迹一般都是平滑的，不会出现跳变点，这种情况就需要将M点剔除，将剩余各个轨迹点组成的轨迹作为期望值来训练车端模型。该过程中，剔除M点的过程即为对有价值数据优化的过程，可以采用线性规划、非线性规划对离散的轨迹点进行处理，具体，可以采用最小二乘法、模拟退火算法等能够达到数据拟合、数据降噪目的的算法。
再以车端模型为决策规划模型为例，假设期望值为人工驾驶员在某场景中的操作表现，然而具体的某个人工驾驶员的操作表现可能是不符合操作规范\交通规则的，这种情况就需要将其中不符合操作规范\交通规则的的操作数据剔除，实现对有价值数据的优化。这一优化过程可将符合操作规范\交通规则作为约束条件，有价值数据中不符合操作规范\交通规则的数据即为不满足约束条件的数据，在数据优化中需要剔除这类数据。
在一种可能的设计中，训练模块利用数据驱动模型得到的标注数据集来训练车端模型。
在另一种可能的设计中，训练模块利用数学规划模型对有价值数据进行优化后的结果来训练车端模型。
204，将车端计算资源正在使用的车端模型迭代为训练后的车端模型。
具体的，迭代模块确定训练模块训练后的车端模型对应的模型参数，并将模型参数下发给车端计算模块，车端计算模块利用下发的模型参数将正在使用的车端模型迭代为训练后的车端模型。
在一种可能的设计中，如图14中的步骤204A和204B所示，迭代模块会对训练后的车端模型进行 测试，当测试结果达到迭代要求(表明训练后的车端模型的推理能力明显优于正在使用的车端模型，可以通过为训练后的车端模型和正在使用中的车端模型分别设置用于表征其各自的推理能力的测试指标来判断测试结果是否达到迭代要求，例如可以根据训练后的车端模型和正在使用中的车端模型在测试中的推理能力表现设置测试指标，测试指标可以设置为与模型的推理能力呈正相关，即测试指标越好表明推理能力越强，当训练后的车端模型的测试指标优于正在使用的车端模型的测试指标时，就表明测试结果达到迭代要求)时，才会将模型参数下发给车端计算模块，完成车端模型迭代操作。
在数据优化模块具有数据驱动模型的实施例中，有可能存在车端模型在某些方面的推理能力优于数据驱动模型的情况，这种情况下，如图13中的步骤205和206所示，训练模块还可以利用优化结果对数据驱动模型进行训练，提升数据驱动模型在这类相比车端目标检测模型较为劣势的方面的推理能力。该实施例中，迭代模块将训练后的数据驱动模型对应的模型参数发送给数据优化模块，将正在使用的数据驱动模型迭代为训练后的数据驱动模型。
在一种可能的设计中，如图15中的步骤206A和206B所示，迭代模块还可以对训练后的数据驱动模型进行测试，当测试结果达到迭代要求(表明训练后的数据驱动模型的推理能力明显优于正在使用的数据驱动模型，可以通过为训练后的数据驱动模型和正在使用中的数据驱动模型分别设置用于表征其各自的推理能力的测试指标来判断测试结果是否达到迭代要求，例如可以根据训练后的数据驱动模型和正在使用中的数据驱动模型在测试中的推理能力表现设置测试指标，测试指标可以设置为与模型的推理能力呈正相关，即测试指标越好表明推理能力越强，当训练后的数据驱动模型的测试指标优于正在使用的数据驱动模型的测试指标时，就表明测试结果达到迭代要求)时，才会将模型参数下发给数据优化模块，完成数据驱动模型迭代操作。
本申请实施例提供的模型自动化迭代方法，首先自动采集对车端模型的性能提升有价值的数据，然后自动对有价值数据进行优化，最后利用优化结果训练车端模型并完成迭代：
(1)该方法采用车端推理--云端训练的模式，即在车端部署多任务、轻量级的车端模型，基于车端模型推理的结果自动地、有针对性地采集对车端模型的性能提升有价值的数据，再利用云端强大的计算能力和数据存储能力，实时地、自动地完成训练数据集生成、模型训练、模型迭代等一系列操作；这种车端推理--云端训练的模式充分发挥了云端的资源优势，提高了自动驾驶车辆模型迭代的效率；
(2)该方法在车端至云端通信资源有限的环境下，自动采集对车端模型的性能提升有价值的数据，该自动采集过程不仅效率高，而且涵盖了罕见、异常、突发的长尾场景，屏蔽了重复数据和垃圾数据，保证了所采集数据的有效性、多样性和完整性，为云端自动化完成模型训练和模型迭代提供了充足、高质、多样、有效、可靠的数据基础；
(3)该方法自动化地基于对车端模型的性能提升有价值的数据生成训练数据集，数据优化模块可利用单任务、深层次的数据驱动模型自动化完成数据标注获得训练数据集，也可利用数学规划模型快速、有效地得到满足要求的训练数据集，这种自动生成训练数据集的方式大大减少了人工标注工作，明显有助于解决因为标注效率低导致的模型迭代耗时长、速度慢的问题。
基于相同的发明思想，本申请实施例提供一种自动驾驶车辆。如图7所示，自动驾驶车辆ADV可以是利用自动驾驶技术实现的具有载人功能(如家用轿车、公共汽车等类型)、载货功能(如普通货车、厢式货车、甩挂车、封闭货车、罐式货车、平板货车、集装厢车、自卸货车、特殊结构货车等类型)、工具功能(如物流配送车、巡逻车、环卫车、吸尘车、洗地车、扫地机器人、割草机、高尔夫球车)、娱乐功能(如娱乐车、游乐场自动驾驶装置)或者特殊救援功能(如消防车、救护车等类型)的车辆。
图7示出根据本申请实施例的一种自动驾驶车辆ADV的结构。自动驾驶车辆ADV包括动力系统V-110、传感器系统V-120、致动系统V-130、外围设备系统V-140、车辆计算系统V-150。在一些可能的设计中，自动驾驶车辆ADV车辆可以包括更多、更少或不同的单元，并且每个单元可以包括更多、更少或不同的组件。在一些可能的设计中，图7所示单元和组件还可以以任意的数量进行组合或划分。
动力系统V-110可以被配置为为车辆提供运动动力。动力系统V-110包括引擎V-111、能量源V112、变速器V113和车轮V114中的一个或多个。
引擎V-111可以是内燃机、电机马达、蒸汽机和斯特林引擎的任何组合，也可以是其它马达和引擎。在一些可能的设计中，动力系统V-110可以包括多种类型的引擎和/或马达。例如，气电混合动力车可以包括汽油引擎和电动马达。
能量源V112可以是全部或部分地为引擎V-111提供动力的能量源V112。引擎V-111可以被配置为将能量源V112转换成机械能。能量源V112可以包括汽油、柴油、丙烷、其它基于压缩气体的燃料、乙醇、太阳能电池板、电池和其它电力源。能量源V112可以附加或可替换地包括燃料箱、电池、电容器和/或飞轮的任何组合。在一些可能的设计中，能量源V112也可以为车辆的其它单元提供能量。
变速器V113可以被配置为将机械动力从引擎V-111发送到车轮V114。为此，变速器V113可以包括变速箱、离合器、差速器、驱动轴和/或其它原件。在变速器V113包括驱动轴的实施例中，驱动轴可以包括被配置为耦合到车轮V114的一个或多个轮轴。
车轮V114可以被配置为任何形式，包括单轮、双轮、三轮、四轮、六轮等形式。其它车轮V114形式也是可能的，例如包括八个或更多车轮的形式。在任何情况下，车轮V114可以被配置为相对于其他车轮V114差速地旋转。在一些可能的设计中，车轮V114可以包括固定地附接到变速器V113的至少一个车轮，以及可以与路面面接触的、耦合到车辆的轮辋的至少一个轮胎。车轮V114可以包括金属和橡胶的任何组合，或者其它材料的组合。
动力系统V-110可以附加或可替换地包括除了前述组件之外的其它组件。
传感器系统V-120可以包括外部传感器V-121和内部传感器V-122。
外部传感器V-121可以包括被配置为感测车辆所处环境的信息的多个传感器，以及被配置为修改传感器的位置和/或方向的一个或多个致动器V1216。例如，外部传感器V-121可以包括位置传感器V1217、惯性传感器V1211、物体传感器V1212、图像传感器V1213中的一个或多个。
位置传感器V1217可以是估计车辆的地理位置的任何传感器，例如，全球定位系统GPS定位设备、载波相位差分RTK定位设备、北斗卫星定位系统定位设备、GLONASS定位系统定位设备、Galileo定位系统定位设备、全球导航卫星系统GNSS定位设备。位置传感器V1217可以包括估计车辆相对于地球的位置的收发器。
惯性传感器V1211可以是被配置为根据惯性加速度来感测车辆的位置和方向改变的任何传感器组合，例如惯性测量单元IMU。在一些可能的设计中，惯性传感器V1211可以包括加速计和陀螺仪。
物体传感器V1212可以是使用无线电信号或激光信号来感测车辆所处环境中的物体的任何传感器，例如雷达、激光测距仪、激光雷达。在一些可能的设计中，除了感测物体之外，雷达和激光雷达还可以附加地感测物体的速度和/或行驶方向。在一些可能的设计中，物体传感器V1212可以包括发射无线电信号或激光信号的发射器以及检测无线电信号或激光信号的检测器。
图像传感器V1213可以包括任何相机(例如静态相机、视频相机等)，用于拍摄车辆所处环境的图像。
此外，外部传感器V-121还可以包括其它的传感器，例如用于检测物体距离的任何传感器，例如，声呐V1214、超声波传感器V-1216等。
内部传感器V-122可以包括被配置为检测与车辆的行驶状态相应的信息的多个传感器。例如，内部传感器V-122可以包括车速传感器V-1221、加速度传感器V-1222以及横摆率传感器V-1223中的一个或多个。
车速传感器V-1221可以是检测车辆的速度的任何传感器。
加速度传感器V-1222可以是检测车辆的加速度的任何传感器。
横摆率传感器V-1223可以是检测车辆绕重心的铅垂轴的横摆率(旋转角速度)的任何传感器，例如，陀螺仪传感器。
在一些可能的设计中，为检测驾驶操作信息，内部传感器V-122还可以包括加速器踏板传感器V-1224、制动器踏板传感器V-1225以及方向盘传感器V-1226中的一个或多个。
加速器踏板传感器V-1224可以是检测加速器踏板的踩踏量的任何传感器，加速器踏板传感器V-1224例如设置于车辆的加速器踏板的轴部分。
制动器踏板传感器V-1225可以是检测制动器踏板的踩踏量的任何传感器，制动器踏板传感器V-1225例如设置于制动器踏板的轴部分。制动器踏板传感器V-1225也可以检测制动器踏板的操作力(对制动器踏板的踏力、主缸的压力等)。
方向盘传感器V-1226可以是检测方向盘的旋转状态的任何传感器，旋转状态的检测值例如是操舵转矩或舵角，方向盘传感器V-1226例如设置于车辆的转向轴。
此外，内部传感器V-122还可以包括其它的传感器，例如监测车辆内部各个组件的传感器(例 如氧气监测器、燃油表、引擎油温度计等)。
在一些示例中，传感器系统V-120可以实施为多个传感器组合，每个传感器组合被配置为安装在车辆的相应位置上(例如，顶部、底部、前侧、后侧、左侧、右侧等)。
致动系统V-130可以被配置为控制车辆的驾驶行为。致动系统V-130可以包括转向模块V-131、节气门模块V-132、制动模块V-133中的一个或多个。
转向模块V-131可以是控制车辆的转向转矩(或操舵转矩)的任何设备组合。
节气门模块V-132可以是通过调整发动机的空气供给量(节气门开度)来达到控制引擎V-111的操作速度和控制车辆的速度的任何设备组合。
制动模块V-133可以是使车辆减速的任何设备组合，例如，制动模块V-133可以利用摩擦力来使车轮V114减速。
外围设备系统V-140可以被配置为使车辆与外部传感器V-121、其它车辆、外部计算设备和/或用户进行交互。例如，外围设备系统V-140可以包括无线通信装置V-141、有线通信接口V-142、触屏显示器V-143、麦克风V-144和扬声器V-145中的一个或多个。
无线通信装置V-141可以被配置为直接地或无线地连接到动力系统V-110、传感器系统V-120、致动系统V-130、外围设备系统V-140和车辆计算系统V-150包括的一个或多个设备，以及直接地或无线地连接其它车辆、中控系统、枢纽服务区中的实体中的一种或多种。无线通信装置V-141可以包括基于无线通信技术通信的天线和芯片组，其中，无线通信技术可以包括全球移动通讯系统(Global System for Mobile Communications，GSM)，通用分组无线服务(General Packet Radio Service，GPRS)，码分多址接入(Code Division Multiple Access，CDMA)，宽带码分多址(Wideband Code Division Multiple Access，WCDMA)，时分码分多址(Time-Division Code Division Multiple Access，TD-SCDMA)，长期演进(Long Term Evolution，LTE)，蓝牙(Blue Tooth，BT)，全球导航卫星系统(Global Navigation Satellite System，GNSS)，调频(Frequency Modulation，FM)，近距离无线通信技术(Near Field Communication，NFC)，红外技术(Infrared，IR)。GNSS可以包括全球卫星定位系统(Global Positioning System，GPS)，全球导航卫星系统(Global Navigation Satellite System，GLONASS)，北斗卫星导航系统(Beidou Navigation Satellite System，BDS)，准天顶卫星系统(Quasi-zenith Satellite System，QZSS)和/或星基增强系统(Satellite Based Augmentation Systems，SBAS)。
有线通信接口V-142可以被配置为直接地连接动力系统V-110、传感器系统V-120、致动系统V-130、外围设备系统V-140和车辆计算系统V-150包括的一个或多个设备，以及直接地连接其它车辆、中控系统、枢纽服务区中的实体中的一种或多种。有线通信接口V-142可以包括集成电路(Inter-Integrated Circuit，I2C)接口，集成电路内置音频(Inter-Integrated Circuit Sound，I2S)接口，脉冲编码调制(Pulse Code Modulation，PCM)接口，通用异步收发传输器(Universal Asynchronous Receiver/Transmitter，UART)接口，移动产业处理器接口(Mobile Industry Processor Interface，MIPI)，通用输入输出(General-Purpose Input/Output，GPIO)接口，用户标识模块(Subscriber Identity Module，SIM)接口，和/或通用串行总线(Universal Serial Bus，USB)接口等。
触屏显示器V-143可以被用户用来向车辆输入命令。触屏显示器V-143可以被配置为通过电容感测、电阻感测或表面声波处理来感测用户手指的位置和/或位置的移动。触屏显示器V-143能够感测在平行或共面与触摸屏表面的方向、垂直与触摸屏表面的方向或者两个方向上的手指移动，并且还能够感测施加到触摸屏表面的压力水平。触屏显示器V-143可以由一个或多个半透明或透明的绝缘层和一个或多个半透明或透明的导电层形成。触屏显示器V-143也可以被配置为其它形式。
麦克风V-144可以被配置为用于接收声音信号(例如，语音命令或其它音频输入)并将声音信号转换为电信号。
扬声器V-145可以被配置为输出音频。
外围设备系统V-140可以进一步或可替换地包括其他组件。
车辆计算系统V-150可以包括处理器V-151和数据存储装置V-152。
处理器V-151可以被配置为用于运行存储于数据存储装置V-152中的指令以执行各种功能，这些功能包括但不限于如下所述的定位融合模块V-1501、感知模块V-1502、行驶状态确定模块V-1503、导航模块V-1504、决策模块V-1505、行驶控制模块V-1506、任务接收模块V-1507对应的功能。处理器V-151可以包括通用处理器(例如CPU、GPU)、专用处理器(例如专用集成电路(Application-specific integrated circuit，ASIC))、现场可编程门阵列(FPGA)、数字信号处理器(DSP)、集成电路、微控制器等一个或多个的组合。在处理器V-151包括多个处理器V-151的情况下，这些 处理器V-151能够单独或组合地工作。
数据存储装置V-152可以包括一个或多个易失性计算机可读存储介质和/或一个或多个非易失性计算机可读存储介质，诸如光学、磁性和/或有机存储介质。数据存储装置V-152可以包括只读存储器(ROM)、随机存取存储器(RAM)、闪速存储器、电可编程存储器(EPROM)、电可编程和可擦除存储器(EEPROM)、嵌入式多媒体卡(eMMC)、硬盘驱动器或任何易失性或非易失性介质等中的一个或多个的组合。数据存储装置V-152可以整体或部分地与处理器V-151集成。数据存储装置V-152可以被配置为存储可由处理器V-151运行以执行各种功能的指令，其中，这些功能包括但不限于如下所述的定位融合模块V-1501、感知模块V-1502、行驶状态确定模块V-1503、导航模块V-1504、决策模块V-1505、行驶控制模块V-1506、任务接收模块V-1507对应的功能。
定位融合模块V-1501可以被配置为接收来自传感器系统V-120感测到的环境数据、位置数据或其他类型的数据，通过对这些数据进行时间戳对齐、融合计算等处理，得到融合后的环境数据和车辆位置数据。定位融合模块V-1501可以包括例如卡尔曼滤波器、贝叶斯网络，以及实现其它功能的算法。
感知模块V-1502可以被配置为接收定位融合模块V-1501计算的融合后的环境数据，并对其进行计算机视觉处理以识别车辆所处环境中的物体和/或特征，该物体和/或特征包括例如车道线、行人、其他车辆、交通信号、基础交通设施等。感知模块V-1502可以使用物体识别算法、运动中恢复结构(Structure from Motion，SFM)算法、视频跟踪或其它计算机视觉技术。在一些可能的设计中，感知模块V-1502可以进一步地配置为对环境进行地图绘制、跟踪物体、估计物体的速度等。
行驶状态确定模块V-1503基于传感器系统V-120中的内部传感器V-122得到的数据识别车辆的行驶状态，例如包括车速、加速度或者横摆率。
任务接收模块V-1507可以被配置为接收任务，解析任务包含的装卸货地址、货物品类、装卸货时间等信息，并将这些信息发送给导航模块V-1504。
导航模块V-1504可以被配置为确定车辆的驾驶路径的任何单元。导航模块V-1504可以进一步地被配置为在车辆的操作时动态地更新驾驶路径。在一些可能的设计中，导航模块V-1504可以被配置为根据来自定位融合模块V-1501、定位传感器、物体传感器V1212、任务接收模块V-1507的处理结果和一个或多个预存的高精地图数据，为车辆确定行驶路径。
决策模块V-1505可以被配置为基于导航模块V-1504计算出的行驶路径、定位融合模块V-1501计算得到的车辆位置数据、以及感知模块V-1502识别出的车辆所处环境中的物体和/或特征，生成车辆的路点信息，路点信息中的路点是在行驶路径中车辆前进的轨迹点。
行驶控制模块V-1506可以被配置为接收决策模块V-1505产生的路点信息，并根据路点信息控制致动系统V-130，以使得车辆按照路点信息行驶。
数据存储装置V-152还可以被配置为存储其他的指令，包括将数据发送到动力系统V-110、传感器系统V-120、致动系统V-130和/或外围设备系统V-140中的一个或多个，从其中接收数据，与其交互，和/或对其进行控制的指令。数据存储装置V-152还可以被配置为存储其他的指令。例如，数据存储装置V-152可以存储用于控制变速器V113的操作以改善燃料效率的指令，可以存储用于控制图像传感器V1213拍摄环境图像的指令，可以存储用于根据物体传感器V1212感测的数据生成车辆所处环境的三维图像的指令，以及，可以存储用于将麦克风V-144转换得到的电信号识别成语音命令的指令。
数据存储装置V-152还可以被配置为存储其他的指令。除存储指令之外，数据存储装置V-152还可以被配置为存储多种信息，例如图像处理参数、训练数据、高精地图、路径信息等。在车辆以自动模式、半自动模式、手动模式运行的期间，这些信息可以被动力系统V-110、传感器系统V-120、致动系统V-130和外围设备系统V-140、车辆计算系统V-150中的一个或多个所使用。
车辆计算系统V-150可以通过系统总线、网络和/或其它连接机制通信连接到动力系统V-110、传感器系统V-120、致动系统V-130和外围设备系统V-140中的一个或多个。
车辆计算系统V-150可以通过数据线直接地或通过无线通信技术无线地连接外围设备系统V-140中的无线通信装置V-141，然后通过无线通信装置V-141无线地连接枢纽服务区和/或中控系统。
车辆计算系统V-150也可以是多个计算装置，这些计算装置分布式地控制车辆的个别组件或者个别系统。
车辆计算系统V-150可以附加地或可替换地包括其它的组件。
图7介绍了自动驾驶车辆100的功能框图，下面介绍自动驾驶车辆100中的车辆计算系统V-150。图8为本申请实施例提供的一种车辆计算系统V-150的结构示意图。
如图8所示，车辆计算系统V-150包括处理器E-100，处理器E-100和系统总线E-000耦合。处理器E-100可以是任何传统处理器，包括精简指令集计算(RISC)处理器、复杂指令集计算(CISC)处理器或上述的组合。可选的，处理器E-100可以是诸如专用集成电路(ASIC)的专用装置。处理器E-100可以是一个或者多个处理器，其中，每个处理器都可以包括一个或多个处理器核。
系统内存E-900和系统总线E-000耦合。运行在系统内存E-900的数据可以包括车辆计算系统V-150的操作系统E-901和应用程序E-904。
操作系统E-901包括壳(Shell)E-902和内核(kernel)E-903。壳E-902是介于用户和操作系统之内核E-903间的一种接口，是操作系统最外面的一层。壳E-902管理用户与操作系统之间的交互，等待用户的输入，向操作系统解释使用者的输入，并且处理各种各样的操作系统的输出结果。
内核E-903由操作系统E-901中用于管理存储器、文件、外设和系统资源的那些部分组成。直接与硬件交互，操作系统内核通常运行进程，并提供进程间的通信，提供CPU时间片管理、中断、内存管理、I/O管理等等。
应用程序E-904包括自动驾驶相关程序E-905，例如管理自动驾驶车辆100和路上障碍物交互的程序，控制自动驾驶装置的行车路线或者速度的程序，控制自动驾驶车辆100和路上其他自动驾驶装置交互的程序。应用程序E-904也存在于软件部署服务器的系统上。当需要执行应用程序E-904时，车辆计算系统V-150可以从软件部署服务器下载应用程序E-904。
系统总线E-000通过总线桥E-200和I/O总线E-300耦合。I/O总线E-300与I/O接口E-400耦合。I/O接口E-400连接有USB接口E-500和多种I/O设备进行通信，这些I/O设备例如是输入设备、媒体盘、收发器、摄像头、传感器等。其中，输入设备例如是键盘、鼠标、触摸屏等；媒体盘例如是CD-ROM、多媒体接口等；收发器用于发送和/或接受无线电通信信号；摄像头用于捕捉景田和动态数字视频图像；传感器可以是图7中传感系统包含的各类传感器，用于探测车辆计算系统V-150周围的环境，并将所感测的信息提供给车辆计算系统V-150。
硬盘驱动器E-800通过硬盘驱动器接口和系统总线E-000耦合。
显示适配器E-700与系统总线E-000耦合，用以驱动显示器。
车辆计算系统V-150可以通过网络接口E-600和软件部署服务器通信。网络接口E-600是硬件网络接口，例如网卡。网络可以是外部网络，例如因特网，也可以是内部网络，例如以太网或者虚拟私人网络(VPN)，还可以是无线网络，例如WiFi网络，蜂窝网络等。
车辆计算系统V-150可以包括车载执行设备，该车载执行设备可以包括一个或多个第一处理器、一个或多个第一存储器、以及存储在第一存储器上并可在第一处理器上运行的计算机指令。当第一处理器在运行第一存储器中的计算机指令时，执行本申请提供的各种实施例中车载执行设备对应的功能。其中，第一处理器可以被配置为处理器V-151中的一个或多个通用处理器(例如CPU、GPU)，一个或多个专用处理器(例如ASIC)，一个或多个现场可编程门阵列(FPGA)，一个或多个数字信号处理器(DSP)，一个或多个集成电路，和/或，一个或多个微控制器等。第一存储器可以被配置为数据存储装置V-152中的一个或多个只读存储器(ROM)，一个或多个随机存取存储器(RAM)，一个或多个闪速存储器，一个或多个电可编程存储器(EPROM)，一个或多个电可编程和可擦除存储器(EEPROM)，一个或多个嵌入式多媒体卡(eMMC)，和/或，一个或多个硬盘驱动器等。车载执行设备对应的功能可以实现为一种计算机程序产品，当该计算机程序产品在计算机上运行时，实现车载执行设备对应的功能。在一种可能的示例中，实现该对应功能的计算机程序产品可以存储在第一存储器中。
图9所示为自动驾驶车辆和车载执行设备的一种可能的示例，如图9所示，该自动驾驶车辆100中配置有车载执行设备50，该车载执行设备50包括第一处理器50A，第一存储器50B，以及存储在第一存储器上并可在第一处理器上运行的计算机指令。当第一处理器在运行第一存储器中的计算机指令时，执行如下步骤对应的方法：S91，通过车端模型获得推理结果；S92，根据车端模型推理的结果，采集对车端模型性能提升有价值的数据；S93，将正在使用的车端模型迭代为训练后的车端模型；其中，所述训练后的车端模型是由云端执行设备通过对所述对车端模型性能提升有价值的数据进行优化、并利用优化结果对车端模型进行训练得到的模型。在一些可能的设计中，车载执行设备50的具体实现形态还可以为其他具有相似存储器和处理器架构的电子设备，例如自动驾驶域控制器。
基于相同的发明思想，本申请实施例还提供一种云端执行设备，如图10所示，云端执行设备60可以包括一个或多个第二处理器60A、一个或多个第二存储器60B、以及存储在第二存储器上并可在第二处理器上运行的计算机指令。当第二处理器在运行第二存储器中的计算机指令时，执行本申请提供的各种实施例中云端执行设备对应的功能。其中，第二处理器可以被配置为一个或多个通用处理器(例如CPU、GPU)，一个或多个专用处理器(例如ASIC)，一个或多个现场可编程门阵列(FPGA)，一个或多个数字信号处理器(DSP)，一个或多个集成电路，和/或，一个或多个微控制器等。第二存储器可以被配置为一个或多个只读存储器(ROM)，一个或多个随机存取存储器(RAM)，一个或多个闪速存储器，一个或多个电可编程存储器(EPROM)，一个或多个电可编程和可擦除存储器(EEPROM)，一个或多个嵌入式多媒体卡(eMMC)，和/或，一个或多个硬盘驱动器等。云端执行设备对应的功能可以实现为一种计算机程序产品，当该计算机程序产品在计算机上运行时，实现云端执行设备对应的功能。在一种可能的示例中，实现该对应功能的计算机程序产品可以存储在第二存储器中。
图10所示为云端执行设备60的一种可能的示例，包括第二处理器60A，第二存储器60B，以及存储在第二存储器上并可在第二处理器上运行的计算机指令。当第二处理器在运行第二存储器中的计算机指令时，执行如下步骤对应的方法：S101，采集对车端模型性能提升有价值的数据；S102，对所述对车端模型性能提升有价值的数据进行优化，并利用优化结果训练车端模型；S103，将车载执行设备正在使用的车端模型迭代为训练后的车端模型。在一些可能的设计中，云端执行设备60的具体实现形态还可以为其他具有相似存储器和处理器架构的电子设备。
通过以上的实施方式的描述，所属领域的技术人员可以清楚地了解到，为描述的方便和简洁，仅以上述各功能模块的划分进行举例说明，实际应用中，可以根据需要而将上述功能分配由不同的功能模块完成，即将装置的内部结构划分成不同的功能模块，以完成以上描述的全部或者部分功能。
在本申请所提供的几个实施例中，应该理解到，所揭露的装置和方法，可以通过其它的方式实现。例如，以上所描述的装置实施例仅仅是示意性的，例如，所述模块或单元的划分，仅仅为一种逻辑功能划分，实际实现时可以有另外的划分方式，例如多个单元或组件可以结合或者可以集成到另一个装置，或一些特征可以忽略，或不执行。另一点，所显示或讨论的相互之间的耦合或直接耦合或通信连接可以是通过一些接口，装置或单元的间接耦合或通信连接，可以是电性，机械或其它的形式。
所述作为分离部件说明的单元可以是或者也可以不是物理上分开的，作为单元显示的部件可以是一个物理单元或多个物理单元，即可以位于一个地方，或者也可以分布到多个不同地方。可以根据实际的需要选择其中的部分或者全部单元来实现本实施例方案的目的。另外，在本申请各个实施例中的各功能单元可以集成在一个处理单元中，也可以是各个单元单独物理存在，也可以两个或两个以上单元集成在一个单元中。上述集成的单元既可以采用硬件的形式实现，也可以采用软件功能单元的形式实现。
所述集成的单元如果以软件功能单元的形式实现并作为独立的产品销售或使用时，可以存储在一个可读取存储介质中。基于这样的理解，本申请实施例的技术方案本质上或者说对现有技术做出贡献的部分或者该技术方案的全部或部分可以以软件产品的形式体现出来，该软件产品存储在一个存储介质中，包括若干指令用以使得一个设备(可以是单片机，芯片等)或处理器(processor)执行本申请各个实施例所述方法的全部或部分步骤。而前述的存储介质包括：U盘、移动硬盘、只读存储器(Read-Only Memory，ROM)、随机存取存储器(Random Access Memory，RAM)、磁碟或者光盘等各种可以存储程序代码的介质。
以上所述，仅为本申请的具体实施方式，但本申请的保护范围并不局限于此，任何在本申请揭露的技术范围内的变化或替换，都应涵盖在本申请的保护范围之内。因此，本申请的保护范围应以所述权利要求的保护范围为准。

Claims (42)

  1. 一种模型自动化迭代方法，其特征在于，包括：
    使用车端计算资源通过车端模型获得推理结果；
    根据车端模型推理的结果，采集对车端模型性能提升有价值的数据；
    使用云端计算资源，对所述对车端模型性能提升有价值的数据进行优化，并利用优化结果训练车端模型；
    将车端计算资源正在使用的车端模型迭代为训练后的车端模型。
  2. 根据权利要求1所述的方法，其特征在于，所述车端模型包括如下一种或多种：
    用于目标检测的模型；
    用于行为预测的模型；
    用于决策规划的模型。
  3. 根据权利要求1所述的方法，其特征在于，所述车端模型采用具有多任务、轻量级特征的网络结构。
  4. 根据权利要求1所述的方法，其特征在于，根据车端模型推理的结果，采集对车端模型性能提升有价值的数据，包括：
    根据车端模型推理的结果及其时空同步信息，确定并采集对车端模型性能提升有价值的数据；
    其中，所述时空同步信息包括与车端模型推理的结果在时间及空间上存在同步关系的环境数据、地图数据、车身状态数据、驾驶员操作数据中的一种或多种。
  5. 根据权利要求4所述的方法，其特征在于，根据车端模型推理的结果及其时空同步信息，确定并采集对车端模型性能提升有价值的数据，包括：
    利用车端模型推理的结果及其时空同步信息构建场景；
    确定已有场景库中缺少所述场景时，将车端模型推理的结果及其时空同步信息作为对车端模型性能提升有价值的数据进行采集。
  6. 根据权利要求5所述的方法，其特征在于，确定已有场景库中缺少所述场景时，将车端模型推理的结果及其时空同步信息作为对车端模型性能提升有价值的数据进行采集，包括：
    使用车端计算资源，按照预定的编码规则对所述场景进行编码；
    使用云端计算资源将场景编码与已有场景库对应的编码库进行比对；
    根据比对结果确定已有场景库的编码库中缺少所述场景编码时，命令车端计算资源将车端模型推理的结果及其时空同步信息作为对车端模型性能提升有价值的数据上传给云端计算资源。
  7. 根据权利要求5所述的方法，其特征在于，确定已有场景库中缺少所述场景，包括：
    确定已有场景库中缺少所述场景对应的类别；或者，
    确定已有场景库中存在所述场景对应的类别，但已有场景库中该类别下的数据量未达到预定数量。
  8. 根据权利要求4所述的方法，其特征在于，根据车端模型推理的结果及其时空同步信息，确定并采集对车端模型性能提升有价值的数据，包括：
    监测到车端模型推理的结果和/或时空同步信息存在异常时，将车端模型推理的结果及其时空同步信息作为对车端模型性能提升有价值的数据进行采集。
  9. 根据权利要求8所述的方法，其特征在于，监测到车端模型推理的结果和/或时空同步信息存在异常，包括下述至少一种：
    确定车端模型推理的结果和/或时空同步信息不属于常规场景；
    对基于不同算法逻辑得到的车端模型推理的结果进行一致性检验，确定检验结果未达到预定的一致性下限；
    确定车端模型推理的结果与期望值不匹配。
  10. 根据权利要求8所述的方法，其特征在于，监测到车端模型推理的结果和/或时空同步信息存在异常时，将车端模型推理的结果及其时空同步信息作为对车端模型性能提升有价值的数据进行采集，包括：
    使用车端计算资源监测到车端模型推理的结果和/或时空同步信息存在异常时，将车端模型推理的结果及其时空同步信息上传给云端计算资源。
  11. 根据权利要求4所述的方法，其特征在于，根据车端模型推理的结果及其时空同步信息，确定并采集对车端模型性能提升有价值的数据，包括：
    确定车端模型推理的结果与期望值相匹配，且匹配度达到预定的匹配阈值时，或者，对基于不同算法逻辑得到的车端模型推理的结果进行一致性检验，确定检验结果达到预定的一致性上限时；
    将车端模型推理的结果及其时空同步信息作为对车端模型性能提升有价值的数据进行采集。
  12. 根据权利要求11所述的方法，其特征在于，确定车端模型推理的结果与期望值相匹配，且匹配度达到预定匹配阈值时，将车端模型推理的结果及其时空同步信息作为对车端模型性能提升有价值的数据进行采集，包括：
    使用车端计算资源确定车端模型推理的结果与期望值相匹配，且匹配度达到预定匹配阈值时，将车端模型推理的结果及其时空同步信息上传给云端计算资源。
  13. 根据权利要求1所述的方法，其特征在于，对所述对车端模型性能提升有价值的数据进行优化，并利用优化结果训练车端模型，包括：
    使用云端计算资源通过数据驱动模型对所述对车端模型性能提升有价值的数据进行标注，并利用标注结果训练车端模型；其中，所述数据驱动模型为深度学习模型或传统机器学习模型。
  14. 根据权利要求1所述的方法，其特征在于，对所述对车端模型性能提升有价值的数据进行优化，并利用优化结果训练车端模型，包括：
    使用云端计算资源，利用数学规划模型对所述对车端模型性能提升有价值的数据进行优化，并利用优化结果训练车端模型。
  15. 根据权利要求13所述的方法，其特征在于，所述数据驱动模型包括包括如下一种或多种：
    用于目标检测的模型；
    用于行为预测的模型；
    用于决策规划的模型。
  16. 根据权利要求13所述的方法，其特征在于，所述数据驱动模型为具有单任务、深层次特征的网络结构的深度学习模型。
  17. 根据权利要求13所述的方法，其特征在于，所述方法还包括：
    利用优化结果训练数据驱动模型；
    将云端计算资源正在使用的数据驱动模型迭代为训练后的数据驱动模型。
  18. 根据权利要求1所述的方法，其特征在于，将车端计算资源正在使用的车端模型迭代为训练后的车端模型，包括：
    对训练后的车端模型进行测试；
    确定训练后的车端模型的测试指标高于车端计算资源正在使用的车端模型的测试指标时，将车端计算资源正在使用的车端模型迭代为训练后的车端模型。
  19. 根据权利要求16所述的方法，其特征在于，将云端计算资源正在使用的数据驱动模型迭代为训练后的数据驱动模型，包括：
    对训练后的数据驱动模型进行测试；
    确定训练后的数据驱动模型的测试指标高于云端计算资源正在使用的数据驱动模型的测试指标时，将云端计算资源正在使用的数据驱动模型迭代为训练后的数据驱动模型。
  20. 一种车端模型自动化迭代方法，其特征在于，包括：
    通过车端模型获得推理结果；
    与云端计算资源相配合，根据车端模型推理的结果，采集对车端模型性能提升有价值的数据；
    与云端计算资源相配合，将正在使用的车端模型迭代为训练后的车端模型；其中，所述训练后的车端模型是由云端计算资源通过对所述对车端模型性能提升有价值的数据进行优化、并利用优化结果对车端模型进行训练得到的模型。
  21. 根据权利要求20所述的方法，其特征在于，所述车端模型包括如下一种或多种：
    用于目标检测的模型；
    用于行为预测的模型；
    用于决策规划的模型。
  22. 根据权利要求20所述的方法，其特征在于，与云端计算资源相配合，根据车端模型推理的结果，采集对车端模型性能提升有价值的数据，包括：
    与云端计算资源相配合，根据车端模型推理的结果及其时空同步信息，确定并采集对车端模型性能提升有价值的数据；
    其中，所述时空同步信息包括与车端模型推理的结果在时间及空间上存在同步关系的环境数据、 地图数据、车身状态数据、驾驶员操作数据中的一种或多种。
  23. 根据权利要求22所述的方法，其特征在于，与云端计算资源相配合，根据车端模型推理的结果及其时空同步信息，确定并采集对车端模型性能提升有价值的数据，包括：
    利用车端模型推理的结果及其时空同步信息构建场景，并将所述场景上传给云端计算资源；
    接收到云端计算资源下发的命令时，将车端模型推理的结果及其时空同步信息作为对车端模型性能提升有价值的数据进行采集，并上传给云端计算资源，其中，云端计算资源在确定已有场景库中缺少所述场景时下发所述命令。
  24. 根据权利要求23所述的方法，其特征在于，将所述场景上传给云端计算资源，包括：
    按照预定的编码规则对所述场景进行编码，并将场景编码上传给云端计算资源；
    接收到云端计算资源下发的命令时，将车端模型推理的结果及其时空同步信息作为对车端模型性能提升有价值的数据进行采集，并上传给云端计算资源，其中，云端计算资源在确定已有场景库中缺少所述场景时下发所述命令，包括：
    接收到云端计算资源下发的命令时，将车端模型推理的结果及其时空同步信息上传给云端计算资源；其中，云端计算资源在将所述场景编码与已有场景库对应的编码库进行比对后，确定已有场景库中缺少所述场景编码时，发送所述命令。
  25. 根据权利要求22所述的方法，其特征在于，与云端计算资源相配合，根据车端模型推理的结果及其时空同步信息，确定对车端模型性能提升有价值的数据并进行采集，包括：
    监测到车端模型推理的结果和/或时空同步信息存在异常时，将车端模型推理的结果及其时空同步信息作为对车端模型性能提升有价值的数据进行采集，并上传给云端计算资源。
  26. 根据权利要求25所述的方法，其特征在于，监测到车端模型推理的结果和/或时空同步信息存在异常，包括下述至少一种：
    确定车端模型推理的结果和/或时空同步信息不属于常规场景；
    对基于不同算法逻辑得到的车端模型推理的结果进行一致性检验，确定检验结果未达到预定的一致性要求时；
    确定车端模型推理的结果与期望值不匹配。
  27. 根据权利要求25所述的方法，其特征在于，与云端计算资源相配合，根据车端模型推理的结果及其时空同步信息，确定对车端模型性能提升有价值的数据并进行采集，包括：
    确定车端模型推理的结果与期望值相匹配，且匹配度达到预定的匹配阈值时，或者，对基于不同算法逻辑得到的车端模型推理的结果进行一致性检验，确定检验结果达到预定的一致性要求时；
    将车端模型推理的结果及其时空同步信息作为对车端模型性能提升有价值的数据进行采集，并上传给云端计算资源。
  28. 一种云端模型自动化迭代方法，其特征在于，包括：
    与车端计算资源相配合，根据车端计算资源通过车端模型推理的结果，采集对车端模型性能提升有价值的数据；
    对所述对车端模型性能提升有价值的数据进行优化，并利用优化结果训练车端模型；
    与车端计算资源相配合，将车端计算资源正在使用的车端模型迭代为训练后的车端模型。
  29. 根据权利要求28所述的方法，其特征在于，与车端计算资源相配合，根据车端计算资源通过车端模型推理的结果，采集对车端模型性能提升有价值的数据，包括：
    与车端计算资源相配合，根据车端模型推理的结果及其时空同步信息，确定并采集对车端模型性能提升有价值的数据；
    其中，所述时空同步信息包括与车端模型推理的结果在时间及空间上存在同步关系的环境数据、地图数据、车身状态数据、驾驶员操作数据中的一种或多种。
  30. 根据权利要求29所述的方法，其特征在于，与车端计算资源相配合，根据车端模型推理的结果及其时空同步信息，确定并采集对车端模型性能提升有价值的数据，包括：
    接收车端计算资源上传的场景，其中，车端计算资源利用车端模型推理的结果及其时空同步信息构建所述场景；
    确定已有场景库中缺少所述场景时，命令车端计算资源将车端模型推理的结果及其时空同步信息作为对车端模型性能提升有价值的数据进行采集并上传。
  31. 根据权利要求29所述的方法，其特征在于，接收车端计算资源上传的场景，包括：
    接收车端计算资源上传的场景编码，所述场景编码是车端计算资源按照预定的编码规则对所述 场景进行编码得到的；
    确定已有场景库中缺少所述场景，包括：
    确定已有场景库对应的编码库中缺少所述场景编码。
  32. 根据权利要求29所述的方法，其特征在于，与车端计算资源相配合，根据车端模型推理的结果及其时空同步信息，确定并采集对车端模型性能提升有价值的数据，包括：
    接收车端计算资源上传的车端模型推理的结果及其时空同步信息，其中，车端计算资源在监测到车端模型推理的结果和/或时空同步信息存在异常时，将车端模型推理的结果及其时空同步信息作为所述对车端模型性能提升有价值的数据进行上传。
  33. 根据权利要求29所述的方法，其特征在于，与车端计算资源相配合，根据车端模型推理的结果及其时空同步信息，确定并采集对车端模型性能提升有价值的数据，包括：
    接收车端计算资源上传的车端模型推理的结果及其时空同步信息，其中，车端计算资源在确定车端模型推理的结果与期望值相匹配，且匹配度达到预定的匹配阈值时，或者，对基于不同算法逻辑得到的车端模型推理的结果进行一致性检验，确定检验结果达到预定的一致性上限时，将车端模型推理的结果及其时空同步信息作为所述对车端模型性能提升有价值的数据进行上传。
  34. 根据权利要求28所述的方法，其特征在于，对所述对车端模型性能提升有价值的数据进行优化，并利用优化结果训练车端模型，包括：
    通过数据驱动模型对所述对车端模型性能提升有价值的数据进行标注，并利用标注结果训练车端模型；其中，所述数据驱动模型为深度学习模型或传统机器学习模型。
  35. 根据权利要求28所述的方法，其特征在于，对所述对车端模型性能提升有价值的数据进行优化，并利用优化结果训练车端模型，包括：
    利用数学规划模型对所述对车端模型性能提升有价值的数据进行优化，并利用优化结果训练车端模型。
  36. 一种车端执行设备，其特征在于，包括：
    车端计算模块，配置有车端模型，通过车端模型推理得到结果；
    车端采集模块，被配置为与云端执行设备相配合，根据车端模型推理的结果，采集对车端模型性能提升有价值的数据；其中，所述云端执行设备对所述对车端模型性能提升有价值的数据进行优化，并利用优化结果训练车端模型；
    所述车端计算模块还被配置为与云端执行设备相配合，将配置的车端模型迭代为训练后的车端模型。
  37. 一种云端执行设备，其特征在于，包括：
    云端采集模块，被配置为与车端执行设备相配合，根据车端执行设备通过车端模型推理的结果，采集对车端模型性能提升有价值的数据；
    数据优化模块，被配置为对所述对车端模型性能提升有价值的数据进行优化；
    训练模块，被配置为利用优化结果训练车端模型；
    迭代模块，被配置为与车端执行设备相配合，将车端执行设备正在使用的车端模型迭代为训练后的车端模型。
  38. 一种电子设备，其特征在于，包括：至少一个处理器，以及与所述至少一个处理器通信连接的存储器，其中，所述存储器存储有可被所述至少一个处理器执行的指令，所述指令被所述至少一个处理器执行，以使所述至少一个处理器能够执行权利要求20-27中任意一项所述方法的步骤。
  39. 一种自动驾驶车辆，其特征在于，包括根据权利要求38所述的电子设备。
  40. 一种存储介质，其上存储有计算机程序，其特征在于，该程序被处理器执行时实现权利要求20-27中任意一项所述方法的步骤。
  41. 一种电子设备，其特征在于，包括：至少一个处理器，以及与所述至少一个处理器通信连接的存储器，其中，所述存储器存储有可被所述至少一个处理器执行的指令，所述指令被所述至少一个处理器执行，以使所述至少一个处理器能够执行权利要求28-35中任意一项所述方法的步骤。
  42. 一种存储介质，其上存储有计算机程序，其特征在于，该程序被处理器执行时实现权利要求28-35中任意一项所述方法的步骤。
PCT/CN2022/120034 2021-09-22 2022-09-20 一种模型自动化迭代方法、设备及存储介质 WO2023045936A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202111106600.2A CN114970303A (zh) 2021-09-22 2021-09-22 一种模型自动化迭代方法、设备及存储介质
CN202111106600.2 2021-09-22

Publications (1)

Publication Number Publication Date
WO2023045936A1 true WO2023045936A1 (zh) 2023-03-30

Family

ID=82974988

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/120034 WO2023045936A1 (zh) 2021-09-22 2022-09-20 一种模型自动化迭代方法、设备及存储介质

Country Status (2)

Country Link
CN (1) CN114970303A (zh)
WO (1) WO2023045936A1 (zh)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116977810A (zh) * 2023-09-25 2023-10-31 之江实验室 多模态后融合的长尾类别检测方法和系统

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114970303A (zh) * 2021-09-22 2022-08-30 北京智行者科技有限公司 一种模型自动化迭代方法、设备及存储介质

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110196593A (zh) * 2019-05-16 2019-09-03 济南浪潮高新科技投资发展有限公司 一种自动驾驶多场景环境检测及决策系统及方法
US20200348676A1 (en) * 2019-04-30 2020-11-05 Baidu Usa Llc Neural network approach for parameter learning to speed up planning for complex driving scenarios
CN111899594A (zh) * 2019-05-06 2020-11-06 百度（美国）有限责任公司 用于自动驾驶车辆的动态模型的自动化训练数据提取方法
CN112925657A (zh) * 2021-01-18 2021-06-08 国汽智控(北京)科技有限公司 一种车路云协同处理系统及方法
CN113264060A (zh) * 2021-05-28 2021-08-17 东风汽车集团股份有限公司 一种基于云端训练模型的自动驾驶节能系统及方法
CN114970303A (zh) * 2021-09-22 2022-08-30 北京智行者科技有限公司 一种模型自动化迭代方法、设备及存储介质

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20200348676A1 (en) * 2019-04-30 2020-11-05 Baidu Usa Llc Neural network approach for parameter learning to speed up planning for complex driving scenarios
CN111899594A (zh) * 2019-05-06 2020-11-06 百度（美国）有限责任公司 用于自动驾驶车辆的动态模型的自动化训练数据提取方法
CN110196593A (zh) * 2019-05-16 2019-09-03 济南浪潮高新科技投资发展有限公司 一种自动驾驶多场景环境检测及决策系统及方法
CN112925657A (zh) * 2021-01-18 2021-06-08 国汽智控(北京)科技有限公司 一种车路云协同处理系统及方法
CN113264060A (zh) * 2021-05-28 2021-08-17 东风汽车集团股份有限公司 一种基于云端训练模型的自动驾驶节能系统及方法
CN114970303A (zh) * 2021-09-22 2022-08-30 北京智行者科技有限公司 一种模型自动化迭代方法、设备及存储介质

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116977810A (zh) * 2023-09-25 2023-10-31 之江实验室 多模态后融合的长尾类别检测方法和系统
CN116977810B (zh) * 2023-09-25 2024-01-09 之江实验室 多模态后融合的长尾类别检测方法和系统

Also Published As

Publication number Publication date
CN114970303A (zh) 2022-08-30

Similar Documents

Publication Publication Date Title
WO2023045935A1 (zh) 一种目标检测模型自动化迭代方法、设备及存储介质
CN110550029B (zh) 障碍物避让方法及装置
WO2023045936A1 (zh) 一种模型自动化迭代方法、设备及存储介质
US11953333B2 (en) Systems and methods for autonomous vehicle performance evaluation
CN113168708B (zh) 车道线跟踪方法和装置
CN109211575B (zh) 无人驾驶汽车及其场地测试方法、装置及可读介质
CN113792566B (zh) 一种激光点云的处理方法及相关设备
WO2021212379A1 (zh) 车道线检测方法及装置
CN113879295A (zh) 轨迹预测方法及装置
CN113498529B (zh) 一种目标跟踪方法及其装置
US20220198107A1 (en) Simulations for evaluating driving behaviors of autonomous vehicles
CN114880842A (zh) 轨迹预测模型的自动化迭代方法、电子设备和存储介质
WO2021065626A1 (ja) 交通制御システム、交通制御方法及び制御装置
US11754719B2 (en) Object detection based on three-dimensional distance measurement sensor point cloud data
US20230048680A1 (en) Method and apparatus for passing through barrier gate crossbar by vehicle
CN112810603B (zh) 定位方法和相关产品
CN113859265B (zh) 一种驾驶过程中的提醒方法及设备
US20230031130A1 (en) Open door reconstruction for sensor simulation
CN113968242B (zh) 自动驾驶场景生成方法、装置及系统
CN113741384B (zh) 检测自动驾驶系统的方法和装置
US20210387628A1 (en) Extracting agent intent from log data for running log-based simulations for evaluating autonomous vehicle software
WO2022148068A1 (zh) 一种车辆检测方法和车辆检测装置
CN113799794B (zh) 车辆纵向运动参数的规划方法和装置
CN113022573B (zh) 道路结构检测方法及装置
CN115508841A (zh) 一种路沿检测的方法和装置

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22871982

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE