WO2025001786A1 - 数据处理方法、装置和车辆 - Google Patents

数据处理方法、装置和车辆 Download PDF

Info

Publication number
WO2025001786A1
WO2025001786A1 PCT/CN2024/097457 CN2024097457W WO2025001786A1 WO 2025001786 A1 WO2025001786 A1 WO 2025001786A1 CN 2024097457 W CN2024097457 W CN 2024097457W WO 2025001786 A1 WO2025001786 A1 WO 2025001786A1
Authority
WO
WIPO (PCT)
Prior art keywords
delay
vehicle
information
delay optimization
processing unit
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2024/097457
Other languages
English (en)
French (fr)
Inventor
管高扬
王强东
陈瑞宁
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shenzhen Yinwang Intelligent Technology Co Ltd
Original Assignee
Shenzhen Yinwang Intelligent Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shenzhen Yinwang Intelligent Technology Co Ltd filed Critical Shenzhen Yinwang Intelligent Technology Co Ltd
Priority to EP24830433.9A priority Critical patent/EP4737253A1/en
Publication of WO2025001786A1 publication Critical patent/WO2025001786A1/zh
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Classifications

    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W60/00—Drive control systems specially adapted for autonomous road vehicles
    • B60W60/001—Planning or execution of driving tasks
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W50/00—Details of control systems for road vehicle drive control not related to the control of a particular sub-unit, e.g. process diagnostic or vehicle driver interfaces
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W60/00—Drive control systems specially adapted for autonomous road vehicles
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04W—WIRELESS COMMUNICATION NETWORKS
    • H04W4/00—Services specially adapted for wireless communication networks; Facilities therefor
    • H04W4/30—Services specially adapted for particular environments, situations or purposes
    • H04W4/40—Services specially adapted for particular environments, situations or purposes for vehicles, e.g. vehicle-to-pedestrians [V2P]
    • H04W4/44—Services specially adapted for particular environments, situations or purposes for vehicles, e.g. vehicle-to-pedestrians [V2P] for communication between vehicles and infrastructures, e.g. vehicle-to-cloud [V2C] or vehicle-to-home [V2H]
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W50/00—Details of control systems for road vehicle drive control not related to the control of a particular sub-unit, e.g. process diagnostic or vehicle driver interfaces
    • B60W2050/0001—Details of the control system
    • B60W2050/0043—Signal treatments, identification of variables or parameters, parameter estimation or state estimation

Definitions

  • the present application relates to the field of intelligent driving, and more specifically, to a data processing method, device and vehicle.
  • a very important indicator for measuring safety is the end-to-end latency of the data flow link of the autonomous driving system, such as the latency from the acquisition of the original signal by sensors such as lidar and cameras to the issuance of a brake command to the chassis domain controller.
  • a lower end-to-end latency (such as less than 100ms) can greatly improve the safety factor of vehicles and pedestrians, and is one of the key goals for the later optimization of systems and applications.
  • the present application provides a data processing method, device and vehicle, which help to reduce end-to-end latency, thereby helping to ensure the driving safety of users.
  • the present application provides a data processing method, the method comprising: obtaining first environmental information around a vehicle and first status information of the vehicle; sending the first environmental information and the first status information to a cloud server; receiving first delay optimization configuration information sent by the cloud server, the cloud server storing a mapping relationship between the first environmental information, the first status information and the first delay optimization configuration information, the first delay optimization configuration information comprising a first delay optimization parameter when a processing unit in the vehicle processes data collected by a sensor; and processing the first data collected by the sensor according to the first delay optimization parameter.
  • the cloud server can send corresponding delay optimization configuration information to the vehicle based on the environmental information around the vehicle, the vehicle's status information and the mapping relationship.
  • the vehicle can obtain appropriate delay optimization configuration information for the current scenario from the cloud server, which helps to reduce the delay of the vehicle's processing unit when processing data collected by the sensor, thereby helping to reduce end-to-end delay, thereby helping to ensure the user's driving safety.
  • the mapping relationship stored in the above cloud server may be a mapping relationship between environment information, state information and latency optimization configuration information in different scenarios.
  • the vehicle can obtain appropriate delay optimization configuration information in different scenarios.
  • the cloud server determines the appropriate delay optimization configuration information for the current scenario of the vehicle through a mapping relationship, which can also reduce the waiting time for the vehicle to obtain the delay optimization configuration information.
  • the first latency optimization parameter includes one or more of a priority parameter of a task in a processing unit, a core binding parameter, and a scheduling parameter of a scheduler in an operating system.
  • the first delay optimization configuration information includes end-to-end delay optimization parameters, where the end-to-end delay is the end-to-end delay of the data flow link of the autonomous driving system in the vehicle, such as the delay from the collection of raw data by sensors (e.g., cameras, lidars, etc.) to the issuance of vehicle control commands to the vehicle controller (e.g., chassis controller).
  • the vehicle controller e.g., chassis controller
  • the above end-to-end delay may include the delay of the above processing unit processing the data collected by the sensor.
  • the first environmental information includes at least one of time, geographic location, climate, and road conditions.
  • the first status information includes at least one of the vehicle's hardware specifications, system status, and autonomous driving status.
  • the vehicle is a vehicle in an autonomous driving state.
  • the method also includes: when the delay of the processing unit processing the data collected by the sensor according to the first delay optimization parameter is greater than or equal to the delay threshold, obtaining second environmental information around the vehicle and second status information of the vehicle; sending the second environmental information and the second status information to the cloud server; receiving second delay optimization configuration information sent by the cloud server, the cloud server storing a mapping relationship between the second environmental information, the second status information and the second delay optimization configuration information, the second delay optimization configuration information including the second delay optimization parameter when the processing unit processes the data collected by the sensor; and processing the second data collected by the sensor according to the second delay optimization parameter.
  • the vehicle can send the second environment information around it and the second state information of the vehicle to the cloud server.
  • the cloud server can send updated delay optimization configuration information to the vehicle based on the mapping relationship.
  • the vehicle can adaptively adjust the delay optimization configuration information during driving to achieve the effect of automatic online performance optimization, which helps to reduce end-to-end delay, thereby helping to ensure the driving safety of users.
  • the vehicle adjusts the deployed delay optimization configuration information, there is no need for the participation of existing autonomous driving application developers, such as modifying code or recompiling.
  • the vehicle can dynamically perceive the performance degradation caused by scene changes during driving by extracting environmental information and vehicle status information within the current time window.
  • the cloud server determines the delay optimization configuration information suitable for the changed scene based on the changed scene of the vehicle.
  • the delay of the processing unit in processing the data collected by the sensor according to the first delay optimization parameter is greater than or equal to the first delay threshold or less than or equal to the second delay threshold, second environmental information around the vehicle and second state information of the vehicle are obtained, wherein the second delay threshold is less than the first delay threshold.
  • the first delay optimization configuration information includes the delay threshold or the delay range.
  • the vehicle can obtain the delay range through the first delay optimization configuration information sent by the cloud server, so as to monitor in real time the relationship between the delay of the processing unit processing the data collected by the sensor and the delay range.
  • the cloud server can be requested for new delay optimization configuration information again, so as to avoid the safety hazards caused by the deterioration of end-to-end delay.
  • the processing unit includes a main processing unit and a backup processing unit, and processes the first data collected by the sensor according to the first delay optimization parameter, including: deploying the first delay optimization parameter in the backup processing unit; within a first time period, the delay of the backup processing unit processing the first data according to the first delay optimization parameter is less than the delay of the main processing unit processing the first data, and the first delay optimization parameter is deployed in the main processing unit.
  • the validity of the first delay optimization parameter can be verified in advance without affecting the normal driving of the vehicle.
  • the first delay optimization parameter is deployed before the main processing unit, and the method also includes: within the first time period, determining that the vehicle control instructions output by the backup processing unit are the same as the vehicle control instructions output by the main processing unit.
  • the vehicle control instructions output by the backup processing unit within the first time period are the same as the vehicle control instructions output by the main processing unit, which helps to improve the effectiveness of verifying the first delay optimization configuration information.
  • the first environmental information includes environmental information of the vehicle at multiple moments in the future
  • the first state information includes state information of the vehicle at the multiple moments in the future
  • the first delay configuration information includes delay optimization parameters when the processing unit processes data collected by the sensor at the multiple moments in the future.
  • the vehicle can obtain the delay configuration parameters at multiple moments in the future in advance. This avoids the frequent interactions between the vehicle and the cloud server in the future, and helps reduce the waiting time for the vehicle to obtain delay optimization configuration information.
  • the method includes: the vehicle sends navigation information to a cloud server, and the cloud server determines the vehicle's environmental information and status information at multiple moments in the future based on the navigation information, thereby sending to the vehicle delay optimization parameters for the processing unit to process data collected by the sensor at multiple moments in the future.
  • a data processing method comprising: obtaining first environmental information around a vehicle and first state information of the vehicle; determining first delay optimization configuration information according to the first environmental information, the first state information, and a mapping relationship, the mapping The mapping relationship includes a mapping relationship between the first environment information, the first state information and the first delay optimization configuration information, and the first delay optimization configuration information includes a first delay optimization parameter when the processing unit in the vehicle processes the data collected by the sensor; according to the first delay optimization parameter, the first data collected by the sensor is processed.
  • the vehicle can determine the appropriate delay optimization configuration information for the current scenario according to the mapping relationship saved in the vehicle, which helps to reduce the delay of the vehicle's processing unit when processing data collected by sensors, thereby helping to reduce end-to-end delay, thereby helping to ensure the user's driving safety.
  • the method also includes: when the delay when the processing unit processes the data collected by the sensor according to the first delay optimization parameter is greater than or equal to the delay threshold, obtaining second environmental information around the vehicle and second state information of the vehicle; determining second delay optimization configuration information according to the second environmental information, the second state information and the mapping relationship, the mapping relationship including a mapping relationship between the second environmental information, the second state information and the second delay optimization configuration information, the second delay optimization configuration information including the second delay optimization parameter when the processing unit processes the data collected by the sensor; and processing the second data collected by the sensor according to the second delay optimization parameter.
  • the vehicle when the delay of the processing unit processing the data collected by the sensor is greater than or equal to the delay threshold during the driving process of the vehicle, the vehicle can re-determine the updated delay optimization configuration information based on the second environmental information around the vehicle, the second state information of the vehicle and the mapping relationship. In this way, the vehicle can adaptively adjust the delay optimization configuration information during driving to achieve the effect of automatic online performance optimization, thereby helping to reduce end-to-end delay, thereby helping to ensure the driving safety of users.
  • the processing unit includes a main processing unit and a backup processing unit, and processes the first data collected by the sensor according to the first delay optimization parameter, including: deploying the first delay optimization parameter in the backup processing unit; within a first time period, the delay of the backup processing unit processing the first data according to the first delay optimization parameter is less than the delay of the main processing unit processing the first data, and the first delay optimization parameter is deployed in the main processing unit.
  • the first delay optimization parameter is deployed before the main processing unit, and the method also includes: within the first time period, determining that the vehicle control instructions output by the backup processing unit are the same as the vehicle control instructions output by the main processing unit.
  • the first environmental information includes environmental information of the vehicle at multiple moments in the future
  • the first state information includes state information of the vehicle at the multiple moments in the future
  • the first delay configuration information includes delay optimization parameters when processing data collected by the sensor at the multiple moments in the future.
  • the present application provides a data processing method, the method comprising: obtaining first environmental information around a vehicle and first status information of the vehicle; sending first delay optimization configuration information to the vehicle based on the first environmental information, the first status information and a mapping relationship, the mapping relationship comprising a mapping relationship between environmental information, status information and delay optimization configuration information, the first delay optimization configuration information comprising a first delay optimization parameter when a processing unit in the vehicle processes data collected by a sensor.
  • the cloud server can send corresponding delay optimization configuration information to the vehicle based on the mapping relationship between the environmental information around the vehicle and the vehicle's status information.
  • the vehicle can obtain appropriate delay optimization configuration information for the current scenario from the cloud server, which helps to reduce the delay of the vehicle's processing unit when processing data collected by the sensor, thereby helping to reduce end-to-end delay, thereby helping to ensure the user's driving safety.
  • the first environmental information includes environmental information of the vehicle at multiple moments in the future
  • the first state information includes state information of the vehicle at the multiple moments in the future
  • the first delay optimization configuration information includes delay optimization parameters when processing data collected by the sensor at the multiple moments in the future.
  • the method further includes: according to the first environment information and the first state information, using a hierarchical navigable small world graphs (HNSW) algorithm to retrieve the mapping relationship to obtain the first delay optimization configuration information.
  • HNSW hierarchical navigable small world graphs
  • the cloud server can retrieve the mapping relationship based on the HNSW algorithm to obtain the first delay optimization configuration information, which helps to reduce the delay when the cloud server searches for the corresponding delay optimization configuration information through the information sent by the vehicle.
  • the method before obtaining the first environmental information around the vehicle and the first state information of the vehicle, the method also includes: obtaining third environmental information around the autonomous driving vehicle and third state information of the autonomous driving vehicle; determining the first delay when processing data collected by the sensor according to the third delay optimization parameter set by the offline simulation system, the third environmental information and the third state information; updating the third delay optimization parameter according to the first delay to obtain a fourth delay optimization parameter; and storing the correspondence between the third environmental information, the third state information and the fourth delay optimization parameter in the mapping relationship.
  • the initial delay optimization parameters are set through the offline simulation system to check the actual running time of the offline simulation system.
  • the end-to-end delay is calculated and optimized. This makes the end-to-end delay calculation more accurate and does not require additional simulation verification.
  • the method before obtaining the first environmental information around the vehicle and the first state information of the vehicle, the method also includes: obtaining third environmental information around the autonomous driving vehicle and third state information of the autonomous driving vehicle; extracting delay model data based on the third environmental information and the third state information data, the delay model data including one or more of the number of application threads, periodic data of the threads, time probability distribution of thread running, and dependency relationship of thread data; inputting the delay model data into a delay simulator to obtain a second delay; determining a fourth delay optimization parameter corresponding to the third environmental information and the third state information based on the second delay; and storing the correspondence between the third environmental information, the third state information, and the fourth delay optimization parameter in the mapping relationship.
  • the end-to-end delay can be simulated by a delay simulator, which can reduce the overhead of simulation delay estimation and perform large-scale parallel optimization with fast convergence speed.
  • the method before receiving the first environmental information and the first state information sent by the vehicle, the method also includes: obtaining third environmental information around the autonomous driving vehicle and third state information of the autonomous driving vehicle; inputting the third environmental information and the third state information into a prediction model to obtain a fourth delay optimization parameter, wherein the prediction model is trained by a training data set, and the training data set includes sample environmental information, sample state information, and the sample delay optimization parameter; and storing the correspondence between the third environmental information, the third state information, and the fourth delay optimization parameter in the mapping relationship.
  • obtaining third environment information around the autonomous driving vehicle and third state information of the autonomous driving vehicle includes: receiving data collected by sensors of the autonomous driving vehicle and the third state information; indexing the data collected by the sensors and the third state information according to time and location to obtain the third environment information and the third state information.
  • the data collected by the sensor and the third state information are indexed according to time and location to obtain the third environment information and the third state information, including: through a crowdsourcing algorithm or a group intelligence perception algorithm, the data collected by the sensor and the third state information are indexed according to time and location to obtain the third environment information and the third state information.
  • targeted scheduling strategies can be provided for massive autonomous driving scenarios (i.e., the combination of the vehicle's environmental information and the vehicle's status information).
  • massive autonomous driving scenarios i.e., the combination of the vehicle's environmental information and the vehicle's status information.
  • crowdsourcing algorithms or group intelligence perception algorithms there is no need to devote a large amount of manpower and material resources to solve the delay optimization configuration of different extreme scenarios (or extreme situations).
  • Each time the version is updated only the mapping relationship in the cloud server needs to be updated, without the need to trouble users to make modifications.
  • the present application provides a data processing device, which includes: an acquisition unit, used to acquire first environmental information around a vehicle and first status information of the vehicle; a sending unit, used to send the first environmental information and the first status information to a cloud server; a receiving unit, used to receive first delay optimization configuration information sent by the cloud server, the cloud server saves a mapping relationship between the first environmental information, the first status information and the first delay optimization configuration information, the first delay optimization configuration information includes a first delay optimization parameter when the processing unit in the vehicle processes data collected by a sensor; the processing unit is used to process the first data collected by the sensor according to the first delay optimization parameter.
  • the acquisition unit is used to obtain the second environmental information around the vehicle and the second state information of the vehicle when the delay of the processing unit processing the data collected by the sensor according to the first delay optimization parameter is greater than or equal to the delay threshold;
  • the sending unit is used to send the second environmental information and the second state information to the cloud server;
  • the receiving unit is used to receive the second delay optimization configuration information sent by the cloud server, the cloud server saves the mapping relationship between the second environmental information, the second state information and the second delay optimization configuration information, and the second delay optimization configuration information includes the second delay optimization parameter when the processing unit processes the data collected by the sensor;
  • the processing unit is used to process the second data collected by the sensor according to the second delay optimization parameter.
  • the processing unit includes a main processing unit and a backup processing unit, and the processing unit is used to deploy the first delay optimization parameter in the backup processing unit; within a first time period, the delay of the backup processing unit processing the first data according to the first delay optimization parameter is less than the delay of the main processing unit processing the first data, and the first delay optimization parameter is deployed in the main processing unit.
  • the device also includes: a determination unit, used to determine, within the first time period, that the vehicle control instruction output by the backup processing unit is the same as the vehicle control instruction output by the main processing unit.
  • the first environmental information includes environmental information of the vehicle at multiple moments in the future
  • the first state information includes state information of the vehicle at the multiple moments in the future
  • the first delay configuration information includes delay optimization parameters when the processing unit processes data collected by the sensor at the multiple moments in the future.
  • the first environmental information includes time, geographic location, climate and road conditions. and/or, the first status information includes at least one of the vehicle's hardware specifications, system status, and autonomous driving status.
  • the present application provides a data processing device, which includes: an acquisition unit for acquiring first environmental information around a vehicle and first state information of the vehicle; a determination unit for determining first delay optimization configuration information based on the first environmental information, the first state information and a mapping relationship, the mapping relationship including a mapping relationship between the first environmental information, the first state information and the first delay optimization configuration information, the first delay optimization configuration information including a first delay optimization parameter when a processing unit in the vehicle processes data collected by a sensor; the processing unit is used to process the first data collected by the sensor according to the first delay optimization parameter.
  • the acquisition unit is also used to obtain second environmental information around the vehicle and second state information of the vehicle when the delay of the processing unit processing the data collected by the sensor according to the first delay optimization parameter is greater than or equal to the delay threshold;
  • the determination unit is also used to determine second delay optimization configuration information based on the second environmental information, the second state information and the mapping relationship, the mapping relationship including the mapping relationship between the second environmental information, the second state information and the second delay optimization configuration information, and the second delay optimization configuration information including the second delay optimization parameter when the processing unit processes the data collected by the sensor;
  • the processing unit is also used to process the second data collected by the sensor according to the second delay optimization parameter.
  • the processing unit includes a main processing unit and a backup processing unit, and the processing unit is used to deploy the first delay optimization parameter in the backup processing unit; within a first time period, the delay of the backup processing unit processing the first data according to the first delay optimization parameter is less than the delay of the main processing unit processing the first data, and the first delay optimization parameter is deployed in the main processing unit.
  • the determination unit is further used to determine, within the first time period, that the vehicle control instruction output by the backup processing unit is the same as the vehicle control instruction output by the main processing unit.
  • the first environmental information includes environmental information of the vehicle at multiple moments in the future
  • the first state information includes state information of the vehicle at the multiple moments in the future
  • the first delay configuration information includes delay optimization parameters when processing data collected by the sensor at the multiple moments in the future.
  • the present application provides a data processing device, comprising: an acquisition unit, used to acquire first environmental information around a vehicle and first status information of the vehicle; a determination unit, used to determine first delay optimization configuration information based on the first environmental information, the first status information and a mapping relationship, the mapping relationship including a mapping relationship between environmental information, status information and delay optimization configuration information, the first delay optimization configuration information including a first delay optimization parameter when a processing unit in the vehicle processes data collected by a sensor; a sending unit, used to send the first delay optimization configuration information to the vehicle.
  • the first environmental information includes environmental information of the vehicle at multiple moments in the future
  • the first state information includes state information of the vehicle at the multiple moments in the future
  • the first delay optimization configuration information includes delay optimization parameters when processing data collected by the sensor at the multiple moments in the future.
  • the device also includes: an information retrieval unit, used to retrieve the mapping relationship according to the first environment information and the first state information using a hierarchical navigable small world HNSW algorithm to obtain the first delay optimization configuration information.
  • the device also includes a parameter updating unit and a storage unit
  • the acquisition unit is further used to obtain third environmental information around the autonomous driving vehicle and third state information of the autonomous driving vehicle before obtaining the first environmental information and the first state information sent by the vehicle
  • the determination unit is further used to determine the first delay when processing data collected by the sensor according to the third delay optimization parameter set by the offline simulation system, the third environmental information and the third state information
  • the parameter updating unit is used to update the third delay optimization parameter according to the first delay to obtain a fourth delay optimization parameter
  • the storage unit is used to store the correspondence between the third environmental information, the third state information and the fourth delay optimization parameter in the mapping relationship.
  • the device also includes a delay simulation unit, and the acquisition unit is further used to obtain third environmental information around the autonomous driving vehicle and third state information of the autonomous driving vehicle before obtaining the first environmental information and the first state information sent by the vehicle; the determination unit is further used to extract delay model data based on the third environmental information and the third state information data, and the delay model data includes one or more of the number of threads of the application, the periodic data of the threads, the time probability distribution of the thread running, and the dependency relationship of the thread data; the delay simulation unit is used to input the delay model data into a delay simulator to obtain a second delay; the determination unit is further used to determine a fourth delay optimization parameter corresponding to the third environmental information and the third state information based on the second delay; the storage unit is used to store the correspondence between the third environmental information, the third state information and the fourth delay optimization parameter in the mapping relationship.
  • the acquisition unit is used to: receive data collected by a sensor of the autonomous driving vehicle and the third state information; search the data collected by the sensor and the third state information according to time and location; The third environment information and the third state information are obtained.
  • the acquisition unit is used to: index the data collected by the sensor and the third state information according to time and location through a crowdsourcing algorithm or a group perception algorithm to obtain the third environment information and the third state information.
  • the present application provides a data processing device, which includes a processing unit and a storage unit, wherein the storage unit is used to store instructions, and the processing unit executes the instructions stored in the storage unit so that the device performs any possible data processing method in the first aspect or the second aspect.
  • the present application provides a data processing device, which includes a processing unit and a storage unit, wherein the storage unit is used to store instructions, and the processing unit executes the instructions stored in the storage unit so that the device executes any possible data processing method in the third aspect.
  • the present application provides a data processing system, which control system includes a sensor and a computing platform, and the computing platform includes any possible data processing device in the fourth aspect, or includes any possible data processing device in the fifth aspect, or includes any possible data processing device in the seventh aspect.
  • the present application provides a vehicle, which includes any possible data processing device in the fourth aspect, or includes any possible data processing device in the fifth aspect, or includes the data processing device described in the seventh aspect, or includes the data processing system described in the ninth aspect.
  • the present application provides a server, which includes any possible data processing device in the sixth aspect, or includes the data processing device described in the eighth aspect.
  • the present application provides a computer program product, comprising: a computer program code, when the computer program code is run on a computer, the computer executes any possible data processing method in the first aspect or the second aspect above.
  • the present application provides a computer program product, comprising: a computer program code, when the computer program code is run on a computer, the computer executes any possible data processing method in the third aspect above.
  • the above-mentioned computer program code can be stored in whole or in part on the first storage medium, wherein the first storage medium can be packaged together with the processor or separately packaged with the processor, and the embodiments of the present application do not specifically limit this.
  • the present application provides a computer-readable medium storing a program code.
  • the computer program code runs on a computer, the computer executes any possible data processing method in the first aspect or the second aspect.
  • the present application provides a computer-readable medium storing a program code.
  • the computer program code runs on a computer, the computer executes any possible data processing method in the third aspect.
  • the present application provides a chip, comprising a circuit for executing any possible data processing method in the first aspect or the second aspect.
  • the present application provides a chip, comprising a circuit for executing any possible data processing method in the third aspect.
  • FIG1 is a functional block diagram of a vehicle provided in an embodiment of the present application.
  • FIG2 is a schematic diagram showing the relationship between application nodes in the robot operating system ROS.
  • FIG3 is a schematic flow chart of a data processing method provided in an embodiment of the present application.
  • FIG. 4 is a schematic diagram of a system architecture provided in an embodiment of the present application.
  • FIG. 5 is another schematic diagram of the system architecture provided in an embodiment of the present application.
  • FIG. 6 is another schematic flow chart of the data processing method provided in an embodiment of the present application.
  • FIG. 7 is an example diagram of a fast retrieval layer data structure of a delay feature vector provided in an embodiment of the present application.
  • FIG. 8 is another schematic flowchart of the data processing method provided in an embodiment of the present application.
  • FIG. 9 is a schematic block diagram of a data processing device provided in an embodiment of the present application.
  • FIG. 10 is another schematic block diagram of a data processing device provided in an embodiment of the present application.
  • FIG. 11 is another schematic block diagram of a data processing device provided in an embodiment of the present application.
  • At least one of A and B is similar to "A and/or B", describing the association relationship of associated objects, indicating that three relationships can exist, for example, at least one of A and B can mean: A exists alone, A and B exist at the same time, and B exists alone.
  • prefixes such as “first” and “second” are used only to distinguish different description objects, and have no limiting effect on the position, order, priority, quantity or content of the described objects.
  • the use of prefixes such as ordinal numbers to distinguish description objects in the embodiments of the present application does not constitute a limitation on the described objects.
  • the meaning of "multiple" is two or more.
  • FIG1 is a functional block diagram of a vehicle 100 provided in an embodiment of the present application.
  • the vehicle 100 may include a perception system 120 and a computing platform 150, wherein the perception system 120 may include one or more sensors for sensing information about the environment around the vehicle 100.
  • the perception system 120 may include a positioning system, and the positioning system may be a global positioning system (GPS), a Beidou system, or other positioning systems.
  • the perception system 120 may also include one or more of an inertial measurement unit (IMU), a laser radar, a millimeter wave radar, an ultrasonic radar, and a camera device.
  • IMU inertial measurement unit
  • the computing platform 150 may include one or more processors, such as processors 151 to 15n (n is a positive integer).
  • the processor is a circuit with signal processing capability.
  • the processor may be a circuit with instruction reading and execution capability, such as a central processing unit (CPU), a microprocessor, a graphics processing unit (GPU) (which can be understood as a microprocessor), or a digital signal processor (DSP); in another implementation, the processor may implement certain functions through the logical relationship of a hardware circuit, and the logical relationship of the hardware circuit is fixed or reconfigurable, such as a processor that is a hardware circuit implemented by an application-specific integrated circuit (ASIC) or a programmable logic device (PLD), such as a field programmable gate array (FPGA).
  • ASIC application-specific integrated circuit
  • PLD programmable logic device
  • the process of the processor loading a configuration document to implement the hardware circuit configuration can be understood as the process of the processor loading instructions to implement the functions of some or all of the above units.
  • the processor can also be a hardware circuit designed for artificial intelligence, which can be understood as an ASIC, such as a neural network processing unit (NPU), a tensor processing unit (TPU), a deep learning processing unit (DPU), etc.
  • the computing platform 150 can also include a memory, which is used to store instructions, and some or all of the processors 151 to 15n can call instructions in the memory to implement corresponding functions.
  • the vehicles involved in the embodiments of the present application are vehicles in a broad sense, which may be means of transportation (such as commercial vehicles, passenger cars, motorcycles, flying cars, trains, etc.), industrial vehicles (such as forklifts, trailers, tractors, etc.), engineering vehicles (such as excavators, bulldozers, cranes, etc.), agricultural equipment (such as mowers, harvesters, etc.), amusement equipment, toy vehicles, etc.
  • the embodiments of the present application do not specifically limit the type of vehicles.
  • end-to-end latency of the data flow link of the autonomous driving system such as the latency from the acquisition of raw signals by sensors such as lidar and cameras to the issuance of braking and other instructions to the chassis domain controller.
  • the running time of each node in the data flow link will have a direct and significant impact.
  • a lower end-to-end latency e.g., less than 100ms
  • the end-to-end delay optimization methods commonly used by autonomous driving system engineers mostly use some limited scenarios to adjust and optimize the parameters of autonomous driving systems and applications.
  • the methods of parameter adjustment and optimization include manual tuning based on expert experience, direct tuning on the vehicle with online closed-loop feedback, and tuning using abstract models, simulators, and emulators combined with AI tuning algorithms.
  • these methods are basically configured for some common working conditions and universal environments, and it is difficult to make adaptive dynamic adjustments for different scenarios during vehicle driving, and the generalization is insufficient; in addition, the adjustment process is generally very time-consuming. Even with the computing resources of cloud servers, the entire process takes several hours.
  • the embodiments of the present application provide a data processing method, device and vehicle, wherein the vehicle can send environmental information around the vehicle and the current status information of the vehicle to a cloud server, and the cloud server can send delay optimization configuration information corresponding to the environmental information and status information to the vehicle based on the mapping relationship between the environmental information, status information and delay optimization configuration information.
  • the vehicle can process the data collected by the sensor through the delay optimization configuration information, which helps to reduce the end-to-end delay, thereby helping to ensure the safety of the vehicle and pedestrians.
  • Message passing A general term for a type of data communication method between processes or software components in a computer system. It abstracts and encapsulates the data to be communicated into "messages". The two or more parties involved in the communication can transfer messages between processes or components by calling primitives such as message sending and receiving, thereby completing data communication.
  • Autonomous driving intelligent operating system An operating system based on the portable operating system interface (POSIX) standard, suitable for the high-performance computing and high-bandwidth communication required for autonomous driving, providing vehicle environment perception, intelligent decision-making, path planning and other functions.
  • POSIX portable operating system interface
  • Application node In the autonomous driving intelligent operating system, an application with specific autonomous driving functions (such as perception, fusion and planning) encapsulated by communication middleware (such as robot operating system (ROS) and communication management (CM) in automotive open system architecture (AUTOSAR), etc.), usually a collection of single processes and multiple threads.
  • communication middleware such as robot operating system (ROS) and communication management (CM) in automotive open system architecture (AUTOSAR), etc.
  • ROS robot operating system
  • CM communication management
  • AUTOSAR automotive open system architecture
  • Two or more nodes realize the transmission of messages and service calls between processes or components by calling communication middleware.
  • Message channel In the autonomous driving intelligent operating system, the communication channel designated between application nodes, which consists of the sender's sending port, the receiver's receiving port, the channel name, the message format, etc.
  • FIG. 2 shows a schematic diagram of the relationship between application nodes in ROS.
  • ROS includes nodes 1, 2, 3 and 4, wherein node 1 can be used to obtain images collected by sensors, node 2 can be used to perform image segmentation on the images collected by sensors, node 3 can be used to identify the type of obstacles, the distance from the vehicle, the speed of obstacles, etc. obtained after image segmentation, and node 4 can be used to determine vehicle control instructions based on the recognition results of node 3 and send the vehicle control instructions to the chassis domain controller.
  • the latency of the above-mentioned autonomous driving application can be understood as the latency of a single node processing data, for example, the latency of node 1 processing data.
  • the above-mentioned end-to-end delay of autonomous driving can be understood as the delay of the node set in the ROS communication system processing data.
  • Fig. 3 shows a schematic flow chart of a data processing method 300 provided in an embodiment of the present application. As shown in Fig. 3, the method 300 can be executed by a vehicle and a cloud server, and the method 300 includes steps S310 to S340.
  • the vehicle obtains first environmental information around the vehicle and first state information of the vehicle.
  • the vehicle obtains first environmental information around the vehicle, including: the vehicle can determine the first environmental information based on data collected by a sensor (for example, a camera, a millimeter-wave radar, a lidar, etc.).
  • a sensor for example, a camera, a millimeter-wave radar, a lidar, etc.
  • Table 1 shows the environment information and status information acquired by the vehicle.
  • the above autonomous driving modes can be understood as the autonomous driving modes currently enabled by the vehicle, such as navigation cruise assistant (NCA), integrated cruise assistant (ICA), auto valet parking (AVP), etc.
  • Auxiliary functions may include autonomous emergency braking (AEB), lane departure warning (LDW), etc.
  • the environmental information and status information shown in Table 1 above are merely illustrative, and the vehicle may obtain more or less information than that shown in Table 1.
  • the environmental information may include data collected by the vehicle's sensors (e.g., one or more of a camera, a laser radar, a millimeter-wave radar, a GPS, and an IMU).
  • the vehicle may send the data collected by the sensor to a cloud server, which determines the environment in which the vehicle is located (e.g., geographic location, time, road conditions, etc.).
  • the vehicle sends the first environment information and the first status information to the cloud server.
  • the cloud server can receive the first environment information and the first status information sent by the vehicle.
  • the vehicle may send the first environmental information and the first status information to a cloud server; alternatively, the vehicle may also send the data collected by the sensor and the first status information to a cloud server, and the cloud server may determine the first environmental information based on the data collected by the sensor.
  • the vehicle may periodically send environmental information about the vehicle and status information about the vehicle to the cloud server.
  • the method 300 before the cloud server receives the first environmental information and the first status information sent by the vehicle, the method 300 also includes: the cloud server receives third status information from the autonomous driving vehicle and data collected by sensors in the autonomous driving vehicle; the cloud server indexes the third status information and the data collected by the sensor according to time and location to obtain the third environmental information and the third status information.
  • the cloud server indexes the third state information and the data collected by the sensor according to time and location to obtain the third environment information and the third state information, including: the cloud server indexes the third state information and the data collected by the sensor according to time and location through a crowdsourcing algorithm or a group intelligence perception algorithm to obtain the third environment information and the third state information.
  • the cloud server sends first delay optimization configuration information to the vehicle based on the first environmental information, the first status information and the mapping relationship, the mapping relationship includes a mapping relationship between environmental information, status information and delay optimization configuration information, and the first delay optimization configuration information includes a first delay optimization parameter when the vehicle's processing unit processes data collected by the sensor.
  • mapping relationship may also be referred to as a latency optimization configuration information retrieval library.
  • the cloud server may store a latency optimization configuration information retrieval library, which includes a mapping relationship between environment information, state information, and latency optimization configuration information.
  • the first environmental information includes environmental information of the vehicle at multiple moments in the future
  • the first state information includes state information of the vehicle at multiple moments in the future
  • the first delay configuration information includes delay optimization parameters when processing data collected by the sensor at multiple moments in the future.
  • the vehicle can send navigation information (for example, navigation information from the vehicle's current location to the destination) to a cloud server, and the cloud server can determine the environmental information around the vehicle at multiple moments in the future and the status information of the vehicle at multiple moments in the future based on the navigation information.
  • the cloud server can determine the delay optimization configuration information at multiple moments in the future based on the environmental information around the vehicle at multiple moments in the future, the status information of the vehicle at multiple moments in the future, and the mapping relationship, and send it to the vehicle. In this way, the vehicle can save the delay optimization configuration information at multiple moments in the future in advance.
  • the data collected by the sensor can be processed by selecting appropriate delay optimization configuration information from the delay optimization configuration information at multiple moments in the future. In this way, it helps to avoid frequent interactions between the vehicle and the cloud server.
  • the vehicle can also determine the environmental information around the vehicle at multiple moments in the future and the status information of the vehicle at multiple moments in the future based on the navigation information, and then send the environmental information around the vehicle at multiple moments in the future and the status information of the vehicle at multiple moments in the future to the cloud server.
  • the first delay optimization configuration information may include delay optimization parameters when at least some of the nodes among the nodes 2, 3 and 4 shown in FIG. 2 process data collected by sensors.
  • the latency optimization parameters include one or more of priority information of tasks in the processing unit, core binding information, and scheduling parameters of a scheduler in an operating system (eg, a Linux system).
  • an operating system eg, a Linux system
  • the method 300 also includes: when the vehicle determines that the delay for processing the data collected by the sensor is greater than or equal to the delay threshold based on the first delay optimization parameter, obtaining second environmental information around the vehicle and second state information of the vehicle; the vehicle sends the second environmental information and the second state information to the cloud server; the cloud server sends second delay optimization configuration information to the vehicle based on the second environmental information, the second state information and the mapping relationship, the second delay optimization configuration information including the second delay optimization parameter when the processing unit processes the data collected by the sensor; the vehicle processes the second data collected by the sensor based on the second delay optimization parameter.
  • the road type where the vehicle is located is a highway
  • the road speed limit is 120 kilometers per hour (km/h)
  • the CPU occupancy rate of the vehicle is 50%.
  • the vehicle can send this information to the cloud server, so that the cloud server determines the delay optimization configuration information 1 according to the mapping relationship and sends it to the vehicle.
  • the vehicle can process the data collected by the sensor according to the delay configuration information 1.
  • the road type where the vehicle is located is an urban road, the road speed limit is 80km/h and the CPU occupancy rate of the vehicle is 80%.
  • the delay threshold for example, 100ms
  • the vehicle can send the corresponding environmental information and state information at time T2 to the cloud server, so that the cloud server determines the delay optimization configuration information 2 according to the environmental information and state information at time T2 and the mapping relationship and sends it to the vehicle.
  • the vehicle can process the data collected by the sensor according to the delay configuration information 2.
  • the vehicle can request new delay optimization configuration information from the cloud server, so that the vehicle processes the data collected by the sensor through the new delay optimization configuration information, which helps to reduce the end-to-end delay in a timely manner and also helps to improve the safety of the vehicle and the driving safety of the user.
  • the method 300 before the cloud server sends the first delay optimization configuration information to the vehicle, the method 300 also includes: according to the first environment information and the first status information, using the HNSW algorithm to retrieve the mapping relationship to obtain the first delay optimization configuration information.
  • the method 300 before the cloud server receives the first environmental information and the first status information sent by the vehicle, the method 300 also includes: the cloud server obtains the third environmental information around the autonomous driving vehicle and the third status information of the autonomous driving vehicle; determines the first delay when processing the data collected by the sensor according to the third delay optimization parameter set by the offline simulation system, the third environmental information and the third status information; updates the third delay optimization parameter according to the first delay to obtain a fourth delay optimization parameter; and the cloud server saves the correspondence between the third environmental information, the third status information and the fourth delay optimization parameter in the mapping relationship.
  • the above-mentioned autonomous driving vehicle can be understood as a road test vehicle, and the third environment information and the third state information can be information collected by the road test vehicle.
  • the third delay optimization parameter may be a preset delay optimization parameter, or may be a delay optimization parameter optimized last time.
  • the preset delay optimization parameter indicates that the priorities of multiple tasks in the processing unit are the same.
  • the cloud server may store an offline simulation system, and the cloud server may set the autonomous driving application process priority and the parameters corresponding to the core binding (i.e., the third delay optimization parameter) on the offline simulation system, and input the third environment information and the third state information into the offline simulation system.
  • the cloud server may monitor the end-to-end delay during actual operation on the offline simulation system.
  • the cloud server may update the third delay optimization parameter based on the end-to-end delay during actual operation, thereby obtaining the fourth delay optimization parameter.
  • Cloud Service The device may save the corresponding relationship between the third environment information, the third state information and the fourth delay optimization parameter in the mapping relationship.
  • the fourth delay optimization parameter can be obtained by repeatedly iterating the heuristic AI tuning algorithm, the reinforcement learning algorithm, and the deep learning algorithm. In this way, the end-to-end delay calculation method is more accurate and no additional simulation verification is required before deployment on the cloud server.
  • the method 300 also includes: the cloud server obtains third environmental information around the autonomous driving vehicle and third state information of the autonomous driving vehicle; the cloud server extracts delay model data based on the third environmental information and the third state information data, and the delay model data includes one or more of the number of application threads, periodic data of the threads, time probability distribution of thread running, and dependency relationship of thread data; the cloud server inputs the delay model data into a delay simulator to obtain a second delay; the cloud server determines a fourth delay optimization parameter corresponding to the third environmental information and the third state information based on the second delay; the cloud server saves the correspondence between the third environmental information, the third state information and the fourth delay optimization parameter in the mapping relationship.
  • the cloud server can extract the delay model data of the autonomous driving system and application under the third environmental information through the third environmental information and the third state information sent by the autonomous driving vehicle.
  • the delay model data includes but is not limited to one or more of the number of application threads, the periodic data of the threads, the probability distribution of the thread running time, and the dependency of the thread data. Then, the end-to-end delay is directly calculated by the delay simulator, and the fourth delay optimization parameter is obtained by repeated iteration using a heuristic AI tuning algorithm, reinforcement learning or deep learning tuning algorithm.
  • the advantage of this method is that the overhead of simulation delay estimation is small, large-scale parallel optimization can be performed, and the convergence speed is fast.
  • the method 300 before the cloud server receives the first environmental information and the first status information sent by the vehicle, the method 300 also includes: the cloud server obtains the third environmental information around the autonomous driving vehicle and the third status information of the autonomous driving vehicle; the cloud server inputs the third environmental information and the third status information data into the prediction model to obtain a fourth delay optimization parameter; the cloud server saves the third environmental information, the third status information and the fourth delay optimization parameter in the mapping relationship.
  • the prediction model can be trained using a training data set, which includes the environment information of the vehicle, the state information of the vehicle, and the delay optimization configuration information corresponding to the environment information and the state information (task priority information, core binding information, scheduling parameters).
  • the trained prediction model can be saved in a cloud server.
  • the cloud server can input the road test data into the prediction model to obtain the corresponding delay optimization configuration information.
  • the cloud server can save the corresponding relationship between the environment information, the state information, and the delay optimization configuration information in the mapping relationship.
  • the cloud server obtains third environmental information around the autonomous driving vehicle and third state information of the autonomous driving vehicle, including: the cloud server receives the third state information from the autonomous driving vehicle and data collected by sensors in the autonomous driving vehicle; the cloud server indexes the third state information and the data collected by the sensors according to time and location to obtain the third environmental information and the third state information.
  • the cloud server indexes the third state information and the data collected by the sensor according to time and location to obtain the third environment information and the third state information, including: indexing the third state information and the data collected by the sensor according to time and location through a crowdsourcing algorithm or a group intelligence perception algorithm to obtain the third environment information and the third state information.
  • S340 The vehicle processes the first data collected by the sensor according to the first time delay optimization parameter.
  • the vehicle includes a main processing unit and a backup processing unit, and the vehicle processes the first data collected by the sensor according to the first delay optimization parameter, including: the vehicle deploys the first delay optimization parameter in the backup processing unit; within a first time period, the delay of the backup processing unit in processing the first data according to the first delay optimization parameter is less than the delay of the main processing unit in processing the first data, and the vehicle deploys the first delay optimization parameter in the main processing unit.
  • the main processing unit may be a main domain
  • the backup processing unit may be a secure backup domain
  • the main processing unit may be a main processor
  • the backup processing unit may be a backup processor
  • the delay of the backup processing unit in processing the first data according to the first delay optimization parameter is less than the delay of the main processing unit in processing the first data. It can be understood that within a preset time length, the average value of the delay of the backup processing unit in processing the first data according to the first delay optimization parameter is less than the average value of the delay of the main processing unit in processing the first data; and/or, the maximum value of the delay of the backup processing unit in processing the first data according to the first delay optimization parameter is less than the maximum value of the delay of the main processing unit in processing the first data.
  • the vehicle deploys the first delay optimization parameter before the main processing unit
  • the method 300 further includes: within the first time period, determining that the vehicle control instruction output by the backup processing unit is the same as the vehicle control instruction output by the main processing unit.
  • FIG4 is a schematic diagram of a system architecture provided by an embodiment of the present application.
  • the system architecture includes a vehicle 410 and a cloud server 420, wherein the vehicle 410 may include a software platform and a hardware platform, and the software platform includes an autonomous driving application 411 and an operating system and middleware. 412, the operating system and middleware 412 include a scene-aware delay monitoring module 4121 and a delay optimization request and configuration module 4122.
  • the hardware platform includes a CPU 413, a GPU 414, a memory 415, a network card 416, a sensor 417, a controller 418, and the like.
  • the cloud server 420 may include an offline scene simulation database 421, an offline scene simulation system 422, and a delay feature vector retrieval library 423.
  • the delay feature vector retrieval library may store the above mapping relationship.
  • the scene-aware delay monitoring module 4121 can be used to perceive the environmental information around the vehicle 410 and the end-to-end delay changes in the current scene, and upload the environmental information and the status information of the vehicle 410 to the cloud server 420.
  • the above scene-aware delay monitoring module 4121 can upload environmental information to the cloud server 420, or can also upload data collected by the sensor 417 to the cloud server 420.
  • the cloud server 420 can determine the environmental information around the vehicle 410 based on the data collected by the sensor 417.
  • the delay optimization request and configuration module 4122 can be used to send a request to the cloud server 420 to optimize the end-to-end delay in the current scenario, and to receive the delay optimization configuration information sent by the cloud server 420 and configure the delay optimization configuration information on the vehicle 410.
  • the offline scene simulation database 421 can be used to record environmental data and vehicle status information of different vehicles in different scenes such as different roads and weather, including but not limited to point cloud data collected by lidar, image data collected by cameras, point cloud data collected by millimeter wave radar, data collected by GPS and inertial sensors, etc., as well as operation logs of various hardware and software in the vehicle, which can be subsequently used for autonomous driving scene playback, simulation, etc.
  • the offline scenario simulation system 422 can be used to read the environmental information and status information in the offline scenario simulation database 421, and provide automatic driving scenario-level simulation and simulation functions for different times, climates, geographical locations and road conditions.
  • the delay feature vector retrieval library 423 can be used to record environmental information, status information and delay optimization configuration information calculated by the offline scenario simulation system 422 in different scenarios.
  • the environmental information includes but is not limited to time, geographical location, climate conditions, road conditions and structure, etc.
  • the status information includes but is not limited to hardware specifications, system status, automatic driving status, etc.
  • CPU 413 or GPU 414 can be used to provide necessary computing resources for autonomous driving systems and applications.
  • the network card 416 can be used to provide network communication resources for the autonomous driving system and applications.
  • Memory 415 can be used to provide storage resources for autonomous driving systems and applications.
  • Sensor 417 can be used to provide perception data resources for autonomous driving systems and applications.
  • Controller 418 can be used to provide chassis control capability services for the autonomous driving vehicle.
  • the autonomous driving application 411 obtains and controls the CPU resources, GPU resources, network card resources, memory resources, and data collected by sensors required for operation through the operating system and middleware 412, and controls the operation of the vehicle 410 through the controller 418.
  • the scene-aware delay monitoring module 4121 analyzes the environment information, state information and end-to-end delay information of the vehicle 410 from the sensor data and application processing data during the operation of the autonomous driving application 411, which can be stored in the memory 415 or directly sent to the cloud server 420 through the network card 416.
  • the cloud server 420 can record this information in the offline scene simulation database 421.
  • the delay optimization request and configuration module 4122 can obtain the end-to-end delay from the scene-aware delay monitoring module 4121.
  • the end-to-end delay needs to be optimized, it will send an optimization request to the cloud server 420 through the network card 416, and receive the delay optimization configuration information sent by the delay feature vector retrieval library 423, and then deploy it on the current autonomous driving operating system and application.
  • FIG5 shows another schematic diagram of the system architecture provided by an embodiment of the present application.
  • the scene-aware delay monitoring module 4121 may include a collection monitoring module 41211 and a dynamic configuration module 41212
  • the delay optimization request and configuration module 4122 may include a network communication module 41221 and a configuration deployment module 41222.
  • the cloud server 420 may also include a scheduling and orchestration automatic optimization system 424 and a delay feature vector retrieval module 425.
  • the acquisition monitoring module 41211 can use the scheduling, communication and other log information of the running process of the autonomous driving application and combine with the algorithm to obtain the environmental information around the vehicle 410 and the status information of the vehicle 410. Then, the information is sent to the cloud server 420 through the network communication module 41221. The cloud server 420 can record this information in the offline scene simulation database 421.
  • the dynamic configuration module 41212 can dynamically obtain the end-to-end delay of the vehicle 410 from the acquisition monitoring module 41211.
  • an optimization request can be sent to the cloud server 420 through the network communication module 41221.
  • the optimization request may include environmental information around the vehicle 410 and the current state information of the vehicle 410.
  • the configuration deployment module 41222 After the delay optimization configuration information is obtained from the cloud server 420, it is sent to the configuration deployment module 41222, which deploys it on the autonomous driving system and application, thereby optimizing the end-to-end delay.
  • the scheduling and choreography automatic optimization system 424 can be used to periodically pull data from the offline scenario simulation database 421 through the offline scenario simulation system 422 on the cloud server 420 to simulate different autonomous driving scenarios and observe the end-to-end delay of the key link data flow of autonomous driving, and then automatically optimize to find the delay optimization configuration information corresponding to different environmental information and state information, and combine the environmental information, state information and delay optimization configuration information into a single system.
  • the corresponding relationship of the configuration information is recorded in the delay feature vector retrieval library 423.
  • FIG6 shows a schematic flow chart of a data processing method 600 provided in an embodiment of the present application.
  • the method 600 may be executed by a cloud server 420, and the method 600 includes:
  • step S610 may include two stages:
  • the first stage is to automatically collect sensor data and status information during the operation of the autonomous driving vehicle (or road test vehicle) and upload it to the cloud server 420.
  • the second stage is to integrate the data uploaded by each autonomous driving vehicle (or road test vehicle) according to the geographical location and time in the cloud server 420 to form an offline scenario simulation database 421.
  • the cloud server 420 integrates the data uploaded by each autonomous driving vehicle according to geographic location and time through crowdsourcing or group intelligence perception technology to form an offline scene simulation database 421.
  • the data that the autonomous driving vehicle needs to collect in Phase 1 includes, but is not limited to, point cloud data from lidar, image data from cameras, point cloud data from millimeter-wave radars, data from sensors such as GPS and IMU, as well as information such as weather, temperature, humidity, light intensity, and visibility.
  • Phase 2 will use the time series and geographic location coordinates of the data from different sources in Phase 1 as the main index, and use group intelligence perception algorithms or crowdsourcing algorithms to build the scattered data into a unified offline scenario simulation database, which can replay and simulate various autonomous driving scenarios for future testing and optimization.
  • Step S620 can be based on the data in the offline scenario simulation database 421 of step S610, and simulate and calculate the delay optimization configuration information corresponding to the environmental information and status information under a certain time slice on the cloud server 420, and then save the environmental information, status information and corresponding delay optimization configuration information under the time slice in the delay feature vector retrieval library 423.
  • Step S620 can be carried out after the offline scene simulation database 421 is initially constructed, and can also be carried out synchronously when the offline scene simulation database 421 is subsequently updated.
  • the offline scenario simulation system 422 selects environmental information and state information at different time slices from the offline scenario simulation database 421 in a sampling or traversal manner, and simulates and replays the end-to-end delay of the autonomous driving system application in different scenarios through the offline scenario simulation system 422. Then, the delay optimization configuration information is determined through a machine learning algorithm.
  • steps S610 and S620 may be completed before the vehicle 410 travels.
  • the vehicle 410 may periodically upload the environment information around the vehicle 410 and the status information of the vehicle 410 to the cloud server 420.
  • the cloud server 420 may retrieve the delay optimization configuration information from the delay feature vector retrieval library 423 according to the environment information and status information sent by the vehicle 410 and send it to the vehicle 410.
  • the above step S630 may occur while the vehicle 410 is driving.
  • the latency optimization configuration information may include a latency range in addition to the task priority, core binding information, scheduling parameters, etc.
  • the vehicle 410 may resend the optimization request to the cloud server.
  • the hardware in the above hardware platform can be an autonomous driving hardware processing platform, where AI CPU and AI Core provide CPU and GPU computing resources respectively, Ethernet card provides Gigabit network card communication capability, universal flash storage (UFS) provides large-capacity storage resources, sensors provide point cloud data, image data, GPS and other data, and controllers provide chassis control capabilities.
  • AI CPU and AI Core provide CPU and GPU computing resources respectively
  • Ethernet card provides Gigabit network card communication capability
  • UFS universal flash storage
  • sensors provide point cloud data
  • image data GPS and other data
  • controllers provide chassis control capabilities.
  • the above operating system can be a Linux system.
  • the acquisition and monitoring module 41211 of the autonomous driving hardware processing platform collects and records a large amount of raw sensor data during the autonomous driving process of the road test vehicle, including but not limited to the point cloud coordinate data of the lidar, the image and video data of the camera, the spatiotemporal coordinate data of the GPS, the timing data of the IMU, etc. Due to the large amount of data, the data at this stage will be stored in the UFS through the CM communication module during the autonomous driving process. When the road test is completed, the information stored in the UFS is directly uploaded to the cloud server through the Ethernet card or the memory reader/writer external workstation. At the same time, the status information of the vehicle during the autonomous driving process will also be uploaded to the cloud server, including but not limited to the frame information, vehicle status, hardware SoC resource usage information, network status, system software operation status, vehicle self-driving status information, etc.
  • the cloud server After receiving different road test data from different vehicles, the cloud server analyzes the time, start and end locations, routes, etc. of each road test data, and then The collected sensor data and status information data are aligned according to time and GPS coordinates, and recorded as a spatiotemporal database of autonomous driving scene data, that is, an offline scene simulation database.
  • heuristic algorithms and neural network-based AI algorithms are also used to automatically parse the weather conditions, solar terms, and road congestion conditions of each road test data, and establish indexes other than time and GPS coordinates to facilitate rapid retrieval and use by subsequent offline scene simulation systems.
  • the offline scene simulation system 422 of the cloud server 420 can call data from the offline scene simulation database 421 constructed in step S610 to simulate different autonomous driving scenes at different times, geographical locations, weather, road conditions, etc., and connect to the software and hardware platform or the autonomous driving system and application to measure the accuracy of the running results of the autonomous driving application and the performance of the entire system.
  • the offline scene simulation system uses an enhanced AI algorithm, which can splice the scene data in different road test data into the same simulation scene, and can also add vehicles, obstacles and pedestrians that did not exist in the playback data through generative AI technology, and even change the features in the original scene, such as changing the weather, traffic flow, road surface properties, etc.
  • the offline scene simulation system extracts the environment information and state information within the time window at a fixed frequency (such as 1 Hz or 0.1 Hz) to form time serialization data.
  • the time serialization data may be as shown in Table 1 above.
  • the offline scene simulation system 422 constructs the environmental information and state information extracted in all time slices into a delay feature vector retrieval library 423.
  • These environmental information and state information can be independently used for future autonomous driving vehicle retrieval, and some of the data is scene-independent data, such as hardware specification feature data. Some data fluctuates greatly with time and scene changes, such as system state characteristics.
  • the delay feature vector retrieval library 423 it can be regularly maintained. For example, when each autonomous driving software or hardware iteration is updated, the above steps S610 and S620 need to be repeated.
  • the scheduling and scheduling automatic optimization system 424 based on the AI algorithm performs end-to-end delay optimization for each time slice of the simulation, with environmental information and state information as input and delay optimization configuration information for the time slice as output.
  • the scheduling and scheduling automatic optimization system 424 can update the output to the delay feature vector retrieval library 423, thereby establishing a correspondence between environmental information, state information, and delay optimization configuration information.
  • the delay optimization configuration information under the time slice can be determined by the two methods described below.
  • the above-mentioned offline simulation system 422 may include a vehicle dynamics simulation system and an automatic driving simulation system.
  • the state information extracted within the time slice is combined to extract and generate the delay model data of the autonomous driving system and application corresponding to the environmental information under the time slice.
  • the delay model data includes but is not limited to the number of application threads, the periodic data of the threads, the probability distribution of the thread running time, the dependency of the thread data, etc.
  • the end-to-end delay is directly obtained by inputting the delay model data into the delay simulator, and the delay optimization configuration information under the time slice is repeatedly calculated by combining the heuristic AI tuning algorithm, reinforcement learning or deep learning tuning algorithm.
  • the advantage of this second method is that the overhead of simulation delay estimation is small, large-scale parallel optimization can be performed, and the convergence speed is fast.
  • the embodiment of the present application is based on HNSW technology and combines the idea of a jump table to construct a delay feature vector fast retrieval layer for the original linked list. Because the delay feature vectors generated by the same road test data have strong locality, there are a large number of identical redundant data in the same time window and the same geographical location. Therefore, the delay feature vectors can be sorted according to the degree of change and the redundancy can be merged. For example, the hardware specifications, system versions and status are unchanged in the same road test. Sparse stratification is performed according to the density of the delay feature vector, starting with the sparse layer, and then querying the dense layer step by step if it meets the conditions. This can improve the retrieval speed of the delay optimization configuration information and reduce the response delay of the cloud server.
  • Figure 7 shows an example diagram of the delay feature vector fast retrieval layer data structure provided in the embodiment of the present application.
  • FIG8 shows a schematic flow chart of a data processing method 800 provided in an embodiment of the present application.
  • the method 800 may be executed by a vehicle, and the method 800 includes:
  • the vehicle determines whether the delay of the processing unit in processing the data collected by the sensor within the time window is greater than or equal to the delay threshold.
  • step S810 can be implemented by the following two methods:
  • the end-to-end delay mean or jitter exceeds the delay threshold within the preset time, it indicates that the external scene of the vehicle has changed and the delay optimization configuration information needs to be redeployed.
  • the advantages of method 2 are that it occupies fewer resources of the autonomous driving software and hardware platform, has low computing overhead, and has a fast response speed and high accuracy in redeploying the delay optimization configuration information when the scene changes.
  • the vehicle sends an optimization request to the cloud server, where the optimization request includes environmental information around the vehicle and status information of the vehicle.
  • the dynamic configuration module 41212 sends the environmental information around the vehicle and the status information of the vehicle to the cloud server, and waits for the cloud server to re-search the delay optimization configuration information suitable for the current scene in the delay feature vector retrieval library and send it to the vehicle.
  • the cloud server can reduce the search response time, and can use the massive computing resources in the cloud server to accurately find the appropriate delay optimization configuration information without occupying the local computing resources of the vehicle.
  • the version is updated, only the delay feature vector index library in the cloud server needs to be updated, and the vehicle is not aware of it.
  • the above is an example of a vehicle sending an optimization request to a cloud server when the time delay feature vector retrieval library is deployed on the cloud server, but the present application is not limited thereto.
  • the time delay feature vector retrieval library can also be deployed in a vehicle.
  • the cloud server can deploy the delay feature vector retrieval library on the vehicle's autonomous driving hardware processing platform through model lossy compression technology.
  • the dynamic configuration module 41212 can re-search the delay optimization configuration information suitable for the current scenario in the compressed delay feature vector retrieval library based on the environmental information around the vehicle and the vehicle's status information. In this way, the vehicle can obtain the updated delay optimization configuration information in a timely manner without being connected to the Internet, and can better comply with functional safety certification.
  • the vehicle receives delay optimization configuration information determined by the cloud server based on the environmental information and status information, and the delay optimization configuration information includes delay optimization configuration parameters.
  • S840 The vehicle deploys the delay optimization configuration parameters on the safety backup domain.
  • the above main domain and safety backup domain can process the data collected by the same sensor.
  • the dynamic configuration module obtains the latest delay optimization configuration information from the delay feature vector retrieval library of the cloud server, it sends it to the configuration deployment module 41222.
  • the configuration deployment module 41222 deploys the delay optimization configuration information in the safety backup domain of autonomous driving to observe its end-to-end delay optimization effect.
  • the safety backup domain and the main domain can both be on the autonomous driving hardware processing platform, and both deploy a full range of autonomous driving applications. If the vehicle control instructions output by the main domain and the safety backup domain are the same within a preset time length and the delay of the safety backup domain processing data collected by the sensor is better than the delay of the main domain processing data collected by the sensor, the configuration deployment module 41222 can deploy the delay optimization configuration information in the main domain.
  • Fig. 9 shows a schematic block diagram of a data processing device 900 provided in an embodiment of the present application.
  • the device 900 includes: an acquisition unit 910, which is used to acquire first environmental information around a vehicle and first state information of the vehicle; a sending unit 920, which is used to send the first environmental information and the first state information to a cloud server; a receiving unit 930, which receives first delay optimization configuration information sent by the cloud server, the cloud server stores a mapping relationship between the first environmental information, the first state information and the first delay optimization configuration information, and the first delay optimization configuration information includes a first delay optimization parameter when the processing unit in the vehicle processes the data collected by the sensor; the processing unit 940, which is used to process the first data collected by the sensor according to the first delay optimization parameter.
  • the acquisition unit 910 is used to obtain second environmental information around the vehicle and second state information of the vehicle when the delay when the processing unit processes the data collected by the sensor according to the first delay optimization parameter is greater than or equal to the delay threshold; the sending unit 920 is used to send the second environmental information and the second state information to the cloud server; the receiving unit 930 is used to receive second delay optimization configuration information sent by the cloud server, the cloud server stores a mapping relationship between the second environmental information, the second state information and the second delay optimization configuration information, and the second delay optimization configuration information includes the second delay optimization parameter when the processing unit processes the data collected by the sensor; the processing unit 940 is used to process the second data collected by the sensor according to the second delay optimization parameter.
  • the processing unit 940 includes a main processing unit and a backup processing unit, and the processing unit 940 is used to deploy the first delay optimization parameter in the backup processing unit; within a first time period, the delay when the backup processing unit processes the first data according to the first delay optimization parameter is less than the delay when the main processing unit processes the first data, and the first delay optimization parameter is deployed in the main processing unit.
  • the device 900 further includes: a determination unit, configured to determine, within the first time period, whether the vehicle control instruction output by the backup processing unit is the same as the vehicle control instruction output by the main processing unit.
  • the first environment information includes environment information of the vehicle at multiple moments in the future
  • the first state information includes the future The state information of the vehicle at multiple moments
  • the first delay configuration information includes the delay optimization parameters when the processing unit processes the data collected by the sensor at multiple moments in the future.
  • the first environmental information includes at least one of time, geographic location, climate and road conditions; and/or, the first status information includes at least one of the vehicle's hardware specifications, system status, and autonomous driving status.
  • the acquisition unit 910 may be the computing platform in Figure 1 or a processing circuit, processor or controller in the computing platform. Taking the acquisition unit 910 as the processor 121 in the computing platform as an example, the processor 121 may acquire the environment information around the vehicle and the state information of the vehicle.
  • the sending unit 920 and the receiving unit 930 may be communication devices in the vehicle 100 .
  • the processing unit 940 may be the computing platform in Figure 1 or a processing circuit, processor or controller in the computing platform. Taking the processing unit 940 as the processor 122 in the computing platform as an example, the processor 122 may process the data collected by the sensor according to the latency optimization configuration information sent by the cloud server.
  • the functions implemented by the above acquisition unit 910 and the functions implemented by the processing unit 940 can be implemented by different processors, or can also be implemented by the same processor, which is not limited in the embodiment of the present application.
  • Fig. 10 shows a schematic flow chart of a data processing device 1000 provided in an embodiment of the present application.
  • the device 1000 includes: an acquisition unit 1010, which is used to acquire first environmental information around a vehicle and first state information of the vehicle; a determination unit 1020, which is used to determine first delay optimization configuration information according to the first environmental information, the first state information and a mapping relationship, the mapping relationship including a mapping relationship between the first environmental information, the first state information and the first delay optimization configuration information, the first delay optimization configuration information including a first delay optimization parameter when a processing unit in the vehicle processes data collected by a sensor; the processing unit 1030, which is used to process the first data collected by the sensor according to the first delay optimization parameter.
  • the acquisition unit 1010 is also used to obtain second environmental information around the vehicle and second state information of the vehicle when the delay when the processing unit processes the data collected by the sensor according to the first delay optimization parameter is greater than or equal to the delay threshold; the determination unit 1020 is also used to determine second delay optimization configuration information based on the second environmental information, the second state information and the mapping relationship, the mapping relationship including the mapping relationship between the second environmental information, the second state information and the second delay optimization configuration information, and the second delay optimization configuration information including the second delay optimization parameter when the processing unit processes the data collected by the sensor; the processing unit 1030 is also used to process the second data collected by the sensor according to the second delay optimization parameter.
  • the processing unit 1030 includes a main processing unit and a backup processing unit, and the processing unit 1030 is used to deploy the first delay optimization parameter in the backup processing unit; within a first time period, the delay of the backup processing unit processing the first data according to the first delay optimization parameter is less than the delay of the main processing unit processing the first data, and the first delay optimization parameter is deployed in the main processing unit.
  • the determination unit 1020 is further used to determine, within the first time period, that the vehicle control instruction output by the backup processing unit is the same as the vehicle control instruction output by the main processing unit.
  • the first environmental information includes environmental information of the vehicle at multiple moments in the future
  • the first state information includes state information of the vehicle at multiple moments in the future
  • the first delay configuration information includes delay optimization parameters when processing data collected by the sensor at multiple moments in the future.
  • the acquisition unit 1010 may be the computing platform in Figure 1 or a processing circuit, processor or controller in the computing platform. Taking the acquisition unit 1010 as the processor 121 in the computing platform as an example, the processor 121 may acquire the environment information around the vehicle and the status information of the vehicle.
  • the determination unit 1020 may be the computing platform in FIG1 or a processing circuit, a processor, or a controller in the computing platform. Taking the determination unit 1020 as the processor 122 in the computing platform as an example, the processor 122 may store a mapping relationship between environment information, state information, and delay optimization configuration information. The processor 122 may determine appropriate delay optimization configuration information in the current scenario based on the acquired environment information, state information, and the mapping relationship.
  • the processing unit 1030 may be the computing platform in Figure 1 or a processing circuit, processor or controller in the computing platform. Taking the control unit 1030 as the processor 123 in the computing platform as an example, the processor 123 may delay the optimization configuration information and process the data collected by the sensor.
  • the functions implemented by the acquisition unit 1010, the functions implemented by the determination unit 1020, and the functions implemented by the processing unit 1030 may be implemented by different processors, or may be implemented by the same processor, or some functions may be implemented by the same processor. This embodiment of the application does not limit this.
  • FIG11 is a schematic block diagram of a data processing device 1100 provided in an embodiment of the present application.
  • the device 1100 includes: an acquisition unit 1110, configured to acquire first environmental information around a vehicle and first state information of the vehicle; a determination unit 1120, configured to determine the first state information of the vehicle based on the first state information of the vehicle; According to the first environmental information, the first state information and the mapping relationship, the first delay optimization configuration information is determined, the mapping relationship includes the mapping relationship between the environmental information, the state information and the delay optimization configuration information, and the first delay optimization configuration information includes the first delay optimization parameter when the processing unit in the vehicle processes the data collected by the sensor; the sending unit 1130 is used to send the first delay optimization configuration information to the vehicle.
  • the first environmental information includes environmental information of the vehicle at multiple moments in the future
  • the first state information includes state information of the vehicle at multiple moments in the future
  • the first delay optimization configuration information includes delay optimization parameters when processing data collected by the sensor at multiple moments in the future.
  • the device 1100 further includes: an information retrieval unit, configured to retrieve the mapping relationship by using a hierarchical navigable small-world HNSW algorithm according to the first environment information and the first state information to obtain the first delay optimization configuration information.
  • an information retrieval unit configured to retrieve the mapping relationship by using a hierarchical navigable small-world HNSW algorithm according to the first environment information and the first state information to obtain the first delay optimization configuration information.
  • the device 1100 also includes a parameter updating unit and a storage unit.
  • the acquisition unit 1110 is further used to obtain third environmental information around the autonomous driving vehicle and third state information of the autonomous driving vehicle before obtaining the first environmental information and first state information sent by the vehicle;
  • the determination unit 1120 is further used to determine the first delay when processing data collected by the sensor according to a third delay optimization parameter set by the offline simulation system, the third environmental information and the third state information;
  • the parameter updating unit is used to update the third delay optimization parameter according to the first delay to obtain a fourth delay optimization parameter;
  • the storage unit is used to store the correspondence between the third environmental information, the third state information and the fourth delay optimization parameter in the mapping relationship.
  • the device also includes a delay simulation unit, and the acquisition unit 1110 is further used to obtain third environmental information around the autonomous driving vehicle and third state information of the autonomous driving vehicle before obtaining the first environmental information and the first state information sent by the vehicle; the determination unit 1120 is further used to extract delay model data based on the third environmental information and the third state information data, and the delay model data includes one or more of the number of threads of the application, the periodic data of the threads, the time probability distribution of the thread running and the dependency relationship of the thread data; the delay simulation unit is used to input the delay model data into the delay simulator to obtain a second delay; the determination unit 1120 is further used to determine the fourth delay optimization parameter corresponding to the third environmental information and the third state information based on the second delay; the storage unit is used to store the correspondence between the third environmental information, the third state information and the fourth delay optimization parameter in the mapping relationship.
  • the acquisition unit 1110 is further used to obtain third environmental information around the autonomous driving vehicle and third state information of the autonomous driving vehicle before obtaining the first environmental information and the first state information
  • the acquisition unit 1110 is used to: receive data collected by the sensor of the autonomous driving vehicle and the third state information; index the data collected by the sensor and the third state information according to time and location to obtain the third environment information and the third state information.
  • the acquisition unit 1110 is used to: index the data collected by the sensor and the third state information according to time and location through a crowdsourcing algorithm or a group perception algorithm to obtain the third environment information and the third state information.
  • the division of the units in the above device is only a division of logical functions. In actual implementation, they can be fully or partially integrated into one physical entity, or they can be physically separated.
  • the units in the device can be implemented in the form of a processor calling software; for example, the device includes a processor, the processor is connected to a memory, and instructions are stored in the memory.
  • the processor calls the instructions stored in the memory to implement any of the above methods or realize the functions of the units of the device, wherein the processor is, for example, a general-purpose processor, such as a CPU or a microprocessor, and the memory is a memory in the device or a memory outside the device.
  • the units in the device can be implemented in the form of hardware circuits, and the functions of some or all of the units can be realized by designing the hardware circuits.
  • the hardware circuit can be understood as one or more processors; for example, in one implementation, the hardware circuit is an ASIC, and the functions of some or all of the above units are realized by designing the logical relationship of the components in the circuit; for another example, in another implementation, the hardware circuit can be implemented by PLD.
  • FPGA as an example, it can include a large number of logic gate circuits, and the connection relationship between the logic gate circuits is configured through the configuration file, so as to realize the functions of some or all of the above units. All units of the above device may be implemented entirely in the form of a processor calling software, or entirely in the form of a hardware circuit, or partially in the form of a processor calling software and the rest in the form of a hardware circuit.
  • Each unit in the above device can be one or more processors (or processing circuits) configured to implement the above method, such as: CPU, GPU, NPU, TPU, DPU, microprocessor, DSP, ASIC, FPGA, or a combination of at least two of these processor forms.
  • processors or processing circuits configured to implement the above method, such as: CPU, GPU, NPU, TPU, DPU, microprocessor, DSP, ASIC, FPGA, or a combination of at least two of these processor forms.
  • the SoC may include at least one processor for implementing any of the above methods or implementing the functions of each unit of the device.
  • the type of the at least one processor may be different, for example, including CPU and FPGA, CPU and artificial intelligence processor, CPU and GPU, etc.
  • An embodiment of the present application also provides a device, which includes a processing unit and a storage unit, wherein the storage unit is used to store instructions, and the processing unit executes the instructions stored in the storage unit so that the device executes the method or steps executed by the above embodiment.
  • the processing unit may be the processor 121 - 12n shown in FIG. 1 .
  • An embodiment of the present application further provides a data processing system, the path planning system comprising one or more sensors and a computing platform, wherein the computing platform comprises the above-mentioned data processing device 900 or data processing device 1000.
  • An embodiment of the present application further provides a vehicle, which may include the above-mentioned data processing device 900, or, include the above-mentioned data processing device 1000, or, include the above-mentioned data processing system.
  • the embodiment of the present application further provides a server, which includes the above-mentioned data processing device 1100.
  • An embodiment of the present application further provides a computer program product, which includes: a computer program code, and when the computer program code is run on a computer, the computer executes the above-mentioned data processing method.
  • the embodiment of the present application further provides a computer-readable medium, wherein the computer-readable medium stores a program code.
  • the computer program code is executed on a computer, the computer executes the above-mentioned data processing method.
  • each step of the above method can be completed by an integrated logic circuit of hardware in a processor or an instruction in the form of software.
  • the method disclosed in conjunction with the embodiment of the present application can be directly embodied as a hardware processor for execution, or a combination of hardware and software modules in a processor for execution.
  • the software module can be located in a storage medium mature in the art such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or a power-on erasable programmable memory, a register, etc.
  • the storage medium is located in a memory, and the processor reads the information in the memory and completes the steps of the above method in conjunction with its hardware. To avoid repetition, it is not described in detail here.
  • the memory may include a read-only memory and a random access memory, and provide instructions and data to the processor.
  • the size of the serial numbers of the above-mentioned processes does not mean the order of execution.
  • the execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.
  • the disclosed systems, devices and methods can be implemented in other ways.
  • the device embodiments described above are only schematic.
  • the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed.
  • Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.
  • the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
  • each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.
  • the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium.
  • the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art.
  • the computer software product is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present application.
  • the aforementioned storage media include: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk, and other media that can store program codes.

Landscapes

  • Engineering & Computer Science (AREA)
  • Automation & Control Theory (AREA)
  • Human Computer Interaction (AREA)
  • Transportation (AREA)
  • Mechanical Engineering (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Traffic Control Systems (AREA)

Abstract

本申请提供了一种数据处理方法、装置和车辆，可以应用于智能驾驶领域。该数据处理方法包括：获取车辆周围的第一环境信息以及该车辆的第一状态信息；向云端服务器发送该第一环境信息和该第一状态信息；接收该云端服务器发送的第一时延优化配置信息，该云端服务器保存有该第一环境信息、该第一状态信息和该第一时延优化配置信息的映射关系，该第一时延优化配置信息包括该车辆中的处理单元处理传感器采集的数据时的第一时延优化参数；根据该第一时延优化参数，处理该传感器采集的第一数据。本申请实施例可以应用于智能汽车或者新能源汽车中，有助于降低端到端时延，从而有助于保证用户的驾乘安全。

Description

数据处理方法、装置和车辆
本申请要求在2023年6月29日提交中国国家知识产权局、申请号为202310792509.3、发明名称为“数据处理方法、装置和车辆”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请涉及智能驾驶领域，并且更具体地，涉及一种数据处理方法、装置和车辆。
背景技术
近年来随着自动驾驶人工智能(artificial intelligence，AI)算法和技术的快速蓬勃发展，大量传统车厂和自动驾驶新势力纷纷下场打造面向L4的高阶全自动驾驶系统和应用。但是，人们对于自动驾驶技术落地存在一定担忧，其中最关心的问题之一的是其能否保障其他车辆和行人的安全。衡量安全性的一个非常重要指标就是自动驾驶系统的数据流链路的端到端时延，如从激光雷达、摄像头等传感器采集到原始信号开始，到向底盘域控制器发出刹车指令结束的时延。较低的端到端时延(如小于100ms)，可以大幅提高车辆和行人的安全系数，是系统和应用后期优化的关键目标之一。
因此，如何降低端到端时延成为了一个亟待解决的问题。
发明内容
本申请提供一种数据处理方法、装置和车辆，有助于降低端到端时延，从而有助于保证用户的驾乘安全。
第一方面，本申请提供了一种数据处理方法，该方法包括：获取车辆周围的第一环境信息以及该车辆的第一状态信息；向云端服务器发送该第一环境信息和该第一状态信息；接收该云端服务器发送的第一时延优化配置信息，该云端服务器保存有该第一环境信息、该第一状态信息和该第一时延优化配置信息的映射关系，该第一时延优化配置信息包括该车辆中的处理单元处理传感器采集的数据时的第一时延优化参数；根据该第一时延优化参数，处理该传感器采集的第一数据。
基于上述技术方案，云端服务器可以基于车辆周围的环境信息、车辆的状态信息和映射关系向车辆发送对应的时延优化配置信息。这样，车辆可以从云端服务器获取到当前所处场景下合适的时延优化配置信息，有助于降低车辆的处理单元在处理传感器采集的数据时的时延，从而有助于降低端到端时延，从而有助于保证用户的驾乘安全。
以上云端服务器中保存的映射关系可以是不同场景下环境信息、状态信息和时延优化配置信息之间的映射关系。
当前端到端时延优化配置基本都是线下根据工程师的经验，针对一些普适环境进行配置，难以针对车辆行驶过程中不同场景进行自适应动态调整，且当前端到端时延优化配置的调整过程非常耗时。本申请实施例中，通过在云端服务器提前部署针对不同场景下的时延优化配置信息，可以使得车辆在不同的场景下均可以获取到合适的时延优化配置信息。同时，云端服务器通过映射关系确定车辆当前所处场景下合适的时延优化配置信息的方式，也可以降低车辆获取时延优化配置信息时的等待时长。
在一些可能的实现方式中，该第一时延优化参数包括处理单元中任务的优先级参数、绑核参数和操作系统中调度器的调度参数中的一项或者多项。
在一些可能的实现方式中，该第一时延优化配置信息包括端到端时延优化参数，该端到端时延为车辆中的自动驾驶系统的数据流链路的端到端时延，如从传感器(例如，摄像头、激光雷达等)采集到原始数据开始，到向车辆的控制器(例如，底盘控制器)发出车控指令结束的时延。
以上端到端时延可以包括上述处理单元处理传感器采集的数据的时延。
在一些可能的实现方式中，该第一环境信息包括时间、地理位置、气候和路况中的至少一项。
在一些可能的实现方式中，该第一状态信息包括该车辆的硬件规格、系统状态、自动驾驶状态中的至少一项。
在一些可能的实现方式中，该车辆为处于自动驾驶状态中的车辆。
结合第一方面，在第一方面可能的实现方式中，该方法还包括：在该处理单元根据该第一时延优化参数处理该传感器采集的数据的时延大于或者等于时延阈值时，获取该车辆周围的第二环境信息以及该车辆的第二状态信息；向该云端服务器发送该第二环境信息和该第二状态信息；接收该云端服务器发送的第二时延优化配置信息，该云端服务器保存有该第二环境信息、该第二状态信息和该第二时延优化配置信息的映射关系，该第二时延优化配置信息包括该处理单元处理该传感器采集的数据时的第二时延优化参数；根据该第二时延优化参数，处理该传感器采集的第二数据。
基于上述技术方案，在车辆行驶过程中，处理单元处理传感器采集的数据的时延大于或者等于时延阈值时，车辆可以向云端服务器发送其周围的第二环境信息和车辆的第二状态信息。云端服务器可以基于映射关系向车辆发送更新的时延优化配置信息。这样，车辆可以自适应地在行驶过程中调整时延优化配置信息，实现在线性能自动优化的效果，有助于降低端到端时延，从而有助于保证用户的驾乘安全。同时，车辆在调整部署的时延优化配置信息时，无需现有自动驾驶应用开发者参与，如修改代码或者重新编译等。
车辆在行驶过程中可以动态感知由于场景变化带来的性能劣化，通过提取当前时间窗口内的环境信息和车辆的状态信息。由云端服务器基于车辆变化后的场景确定适合该变化后的场景的时延优化配置信息。
在一些可能的实现方式中，在该处理单元根据该第一时延优化参数处理该传感器采集的数据的时延大于或者等于第一时延阈值或者小于或者等于第二时延阈值时，获取该车辆周围的第二环境信息以及该车辆的第二状态信息，其中，该第二时延阈值小于该第一时延阈值。
或者，在该处理单元根据该第一时延优化参数处理该传感器采集的数据的时延不在时延范围内时，获取该车辆周围的第二环境信息以及该车辆的第二状态信息。
在一些可能的实现方式中，该第一时延优化配置信息中包括该时延阈值或者该时延范围。
基于上述技术方案，车辆可以通过云端服务器发送的第一时延优化配置信息获取到该时延范围，从而可以实时监测处理单元处理传感器采集的数据的时延与时延范围的关系。在该时延不在该时延范围内时，可以重新向云端服务器请求新的时延优化配置信息，从而可以避免由于端到端时延恶化而造成的安全隐患。
结合第一方面，在第一方面可能的实现方式中，该处理单元包括主处理单元和备份处理单元，该根据该第一时延优化参数，处理该传感器采集的第一数据，包括：将该第一时延优化参数部署在该备份处理单元；在第一时间段内，该备份处理单元根据该第一时延优化参数处理该第一数据的时延小于该主处理单元处理该第一数据的时延时，将该第一时延优化参数部署在该主处理单元。
基于上述技术方案，通过将第一时延优化参数先部署到备份处理单元，可以提前验证第一时延优化参数的有效性，同时不影响车辆的正常行驶。
结合第一方面，在第一方面可能的实现方式中，该将该第一时延优化参数部署在该主处理单元之前，该方法还包括：在该第一时间段内，确定该备份处理单元输出的车控指令与该主处理单元输出的车控指令相同。
基于上述技术方案，在将第一时延优化参数部署到主处理单元之前可以确定第一时间段内备份处理单元输出的车控指令与该主处理单元输出的车控指令相同，有助于提升对第一时延优化配置信息验证的有效性。
结合第一方面，在第一方面可能的实现方式中，该第一环境信息包括未来多个时刻下该车辆所处的环境信息，该第一状态信息包括该未来多个时刻下该车辆的状态信息；其中，该第一时延配置信息包括该未来多个时刻下该处理单元处理该传感器采集的数据时的时延优化参数。
基于上述技术方案，车辆可以提前获取未来多个时刻下的时延配置参数，这样，避免了车辆在未来一段时间内和云端服务器之间频繁交互的过程，有助于降低车辆获取时延优化配置信息时的等待时长。
在一些可能的实现方式中，该方法包括：车辆向云端服务器发送导航信息，由云端服务器根据该导航信息确定车辆在未来多个时刻下的环境信息和状态信息，从而向车辆发送未来多个时刻下该处理单元处理该传感器采集的数据时的时延优化参数。
第二方面，提供了一种数据处理方法，该方法包括：获取车辆周围的第一环境信息以及该车辆的第一状态信息；根据该第一环境信息、该第一状态信息以及映射关系，确定第一时延优化配置信息，该映 射关系包括该第一环境信息、该第一状态信息和该第一时延优化配置信息的映射关系，该第一时延优化配置信息包括该车辆中的处理单元处理传感器采集的数据时的第一时延优化参数；根据该第一时延优化参数，处理该传感器采集的第一数据。
基于上述技术方案，车辆可以根据车辆中保存的映射关系，确定当前所处场景下合适的时延优化配置信息，有助于降低车辆的处理单元在处理传感器采集的数据时的时延，从而有助于降低端到端时延，从而有助于保证用户的驾乘安全。
结合第二方面，在第二方面可能的实现方式中，该方法还包括：在该处理单元根据该第一时延优化参数处理该传感器采集的数据时的时延大于或者等于时延阈值时，获取该车辆周围的第二环境信息以及该车辆的第二状态信息；根据该第二环境信息、该第二状态信息以及该映射关系，确定第二时延优化配置信息，该映射关系包括该第二环境信息、该第二状态信息和该第二时延优化配置信息的映射关系，该第二时延优化配置信息包括该处理单元处理传感器采集的数据时的第二时延优化参数；根据该第二时延优化参数，处理该传感器采集的第二数据。
基于上述技术方案，在车辆行驶过程中，处理单元处理传感器采集的数据的时延大于或者等于时延阈值时，车辆可以重新根据车辆周围的第二环境信息、车辆的第二状态信息以及该映射关系，确定更新后的时延优化配置信息。这样，车辆可以自适应得在行驶过程中调整时延优化配置信息，实现在线性能自动优化的效果，从而有助于降低端到端时延，从而有助于保证用户的驾乘安全。
结合第二方面，在第二方面可能的实现方式中，该处理单元包括主处理单元和备份处理单元，该根据该第一时延优化参数，处理该传感器采集的第一数据，包括：将该第一时延优化参数部署在该备份处理单元；在第一时间段内，该备份处理单元根据该第一时延优化参数处理该第一数据的时延小于该主处理单元处理该第一数据的时延时，将该第一时延优化参数部署在该主处理单元。
结合第二方面，在第二方面可能的实现方式中，该将该第一时延优化参数部署在该主处理单元之前，该方法还包括：在该第一时间段内，确定该备份处理单元输出的车控指令与该主处理单元输出的车控指令相同。
结合第二方面，在第二方面可能的实现方式中，该第一环境信息包括未来多个时刻下该车辆所处的环境信息，该第一状态信息包括该未来多个时刻下该车辆的状态信息；其中，该第一时延配置信息包括该未来多个时刻下处理该传感器采集的数据时的时延优化参数。
第三方面，本申请提供了一种数据处理方法，该方法包括：获取车辆周围的第一环境信息以及该车辆的第一状态信息；根据该第一环境信息、第一状态信息和映射关系，向该车辆发送第一时延优化配置信息，该映射关系包括环境信息、状态信息和时延优化配置信息的映射关系，该第一时延优化配置信息包括该车辆中的处理单元处理传感器采集的数据时的第一时延优化参数。
基于上述技术方案，云端服务器可以基于车辆周围的环境信息、车辆的状态信息映射关系向车辆发送对应的时延优化配置信息。这样，车辆可以从云端服务器获取到当前所处场景下合适的时延优化配置信息，有助于降低车辆的处理单元在处理传感器采集的数据时的时延，从而有助于降低端到端时延，从而有助于保证用户的驾乘安全。
结合第三方面，在第三方面可能的实现方式中，该第一环境信息包括未来多个时刻下该车辆所处的环境信息，该第一状态信息包括该未来多个时刻下该车辆的状态信息；其中，该第一时延优化配置信息包括该未来多个时刻下处理该传感器采集的数据时的时延优化参数。
结合第三方面，在第三方面可能的实现方式中，该方法还包括：根据该第一环境信息、该第一状态信息，采用分层可导航小世界(hierarchical navigable small world graphs，HNSW)算法对该映射关系进行检索，得到该第一时延优化配置信息。
基于上述技术方案，云端服务器可以基于HNSW算法对该映射关系进行检索，从而得到该第一时延优化配置信息，有助于降低云端服务器通过车辆发送的信息查找其对应的时延优化配置信息时的时延。
结合第三方面，在第三方面可能的实现方式中，该获取车辆周围的第一环境信息以及该车辆的第一状态信息之前，该方法还包括：获取自动驾驶车辆周围的第三环境信息以及该自动驾驶车辆的第三状态信息；根据离线仿真系统设置的第三时延优化参数、该第三环境信息和该第三状态信息，确定处理传感器采集的数据时的第一时延；根据该第一时延，对该第三时延优化参数进行更新，得到第四时延优化参数；将该第三环境信息、该第三状态信息和该第四时延优化参数的对应关系，存储在该映射关系中。
基于上述技术方案，通过离线仿真系统设置初始时延优化参数，查看离线仿真系统上实际运行时的 端到端时延，并进行优化。这样得到的端到端时延计算方式更为准确，无需额外的仿真验证。
结合第三方面，在第三方面可能的实现方式中，该获取车辆周围的第一环境信息以及该车辆的第一状态信息之前，该方法还包括：获取自动驾驶车辆周围的第三环境信息以及该自动驾驶车辆的第三状态信息；根据该第三环境信息和该第三状态信息数据，提取时延模型数据，该时延模型数据包括应用的线程数、线程的周期性数据、线程运行的时间概率分布和线程数据的依赖关系中的一种或者多种；将该时延模型数据输入时延模拟器，得到第二时延；根据该第二时延，确定该第三环境信息和该第三状态信息对应的第四时延优化参数；将该第三环境信息、该第三状态信息和该第四时延优化参数的对应关系，存储在该映射关系中。
基于上述技术方案，通过时延模拟器模拟端到端时延，可以降低仿真估算时延的开销，也可以做大规模并行优化，收敛速度快。
在一些可能的实现方式中，该接收车辆发送的第一环境信息和第一状态信息之前，该方法还包括：获取自动驾驶车辆周围的第三环境信息以及该自动驾驶车辆的第三状态信息；将该第三环境信息以及该第三状态信息输入预测模型中，得到第四时延优化参数，该预测模型由训练数据集训练得到，该训练数据集包括样本环境信息、样本状态信息以及该样本时延优化参数；将该第三环境信息、该第三状态信息和该第四时延优化参数的对应关系，存储在该映射关系中。
在一些可能的实现方式中，该获取自动驾驶车辆周围的第三环境信息以及该自动驾驶车辆的第三状态信息，包括：接收来自该自动驾驶车辆的传感器采集的数据以及该第三状态信息；将该传感器采集的数据和该第三状态信息按照时间和位置进行索引，得到该第三环境信息和该第三状态信息。
在一些可能的实现方式中，该将该传感器采集的数据和该第三状态信息按照时间和位置进行索引，得到该第三环境信息和该第三状态信息，包括：通过众包算法或者群智感知算法，将该传感器采集的数据和该第三状态信息按照时间和位置进行索引，得到该第三环境信息和该第三状态信息。
基于上述技术方案，可以为海量自动驾驶场景(即车辆所处的环境信息和车辆的状态信息的组合)提供针对性的调度编排策略。利用众包算法或者群智感知算法，不需要专门大量人力物力去解决不同极端场景(或极端情况)的时延优化配置。每次版本更新时，只需更新云端服务器中的映射关系，无需麻烦用户做修改。
第四方面，本申请提供了一种数据处理装置，该装置包括：获取单元，用于获取车辆周围的第一环境信息以及该车辆的第一状态信息；发送单元，用于向云端服务器发送该第一环境信息和该第一状态信息；接收单元，接收该云端服务器发送的第一时延优化配置信息，该云端服务器保存有该第一环境信息、该第一状态信息和该第一时延优化配置信息的映射关系，该第一时延优化配置信息包括该车辆中的处理单元处理传感器采集的数据时的第一时延优化参数；该处理单元，用于根据该第一时延优化参数，处理该传感器采集的第一数据。
结合第四方面，在第四方面可能的实现方式中，该获取单元，用于在该处理单元根据该第一时延优化参数处理该传感器采集的数据的时延大于或者等于时延阈值时，获取该车辆周围的第二环境信息以及该车辆的第二状态信息；该发送单元，用于向该云端服务器发送该第二环境信息和该第二状态信息；该接收单元，用于接收该云端服务器发送的第二时延优化配置信息，该云端服务器保存有该第二环境信息、该第二状态信息和该第二时延优化配置信息的映射关系，该第二时延优化配置信息包括该处理单元处理该传感器采集的数据时的第二时延优化参数；该处理单元，用于根据该第二时延优化参数，处理该传感器采集的第二数据。
结合第四方面，在第四方面可能的实现方式中，该处理单元包括主处理单元和备份处理单元，该处理单元，用于将该第一时延优化参数部署在该备份处理单元；在第一时间段内，该备份处理单元根据该第一时延优化参数处理该第一数据的时延小于该主处理单元处理该第一数据的时延时，将该第一时延优化参数部署在该主处理单元。
结合第四方面，在第四方面可能的实现方式中，该装置还包括：确定单元，用于在该第一时间段内，确定该备份处理单元输出的车控指令与该主处理单元输出的车控指令相同。
结合第四方面，在第四方面可能的实现方式中，该第一环境信息包括未来多个时刻下该车辆所处的环境信息，该第一状态信息包括该未来多个时刻下该车辆的状态信息；其中，该第一时延配置信息包括该未来多个时刻下该处理单元处理该传感器采集的数据时的时延优化参数。
结合第四方面，在第四方面可能的实现方式中，该第一环境信息包括时间、地理位置、气候和路况 中的至少一项；和/或，该第一状态信息包括该车辆的硬件规格、系统状态、自动驾驶状态中的至少一项。
第五方面，本申请提供了一种数据处理装置，该装置包括：获取单元，用于获取车辆周围的第一环境信息以及该车辆的第一状态信息；确定单元，用于根据该第一环境信息、该第一状态信息以及映射关系，确定第一时延优化配置信息，该映射关系包括该第一环境信息、该第一状态信息和该第一时延优化配置信息的映射关系，该第一时延优化配置信息包括该车辆中的处理单元处理传感器采集的数据时的第一时延优化参数；该处理单元，用于根据该第一时延优化参数，处理该传感器采集的第一数据。
结合第五方面，在第五方面可能的实现方式中，该获取单元，还用于在该处理单元根据该第一时延优化参数处理该传感器采集的数据的时延大于或者等于时延阈值时，获取该车辆周围的第二环境信息以及该车辆的第二状态信息；该确定单元，还用于根据该第二环境信息、该第二状态信息以及该映射关系，确定第二时延优化配置信息，该映射关系包括该第二环境信息、该第二状态信息和该第二时延优化配置信息的映射关系，该第二时延优化配置信息包括该处理单元处理传感器采集的数据时的第二时延优化参数；该处理单元，还用于根据该第二时延优化参数，处理该传感器采集的第二数据。
结合第五方面，在第五方面可能的实现方式中，该处理单元包括主处理单元和备份处理单元，该处理单元，用于将该第一时延优化参数部署在该备份处理单元；在第一时间段内，该备份处理单元根据该第一时延优化参数处理该第一数据的时延小于该主处理单元处理该第一数据的时延时，将该第一时延优化参数部署在该主处理单元。
结合第五方面，在第五方面可能的实现方式中，该确定单元，还用于在该第一时间段内，确定该备份处理单元输出的车控指令与该主处理单元输出的车控指令相同。
结合第五方面，在第五方面可能的实现方式中，该第一环境信息包括未来多个时刻下该车辆所处的环境信息，该第一状态信息包括该未来多个时刻下该车辆的状态信息；其中，该第一时延配置信息包括该未来多个时刻下处理该传感器采集的数据时的时延优化参数。
第六方面，本申请提供了一种数据处理装置，该装置包括：获取单元，用于获取车辆周围的第一环境信息以及该车辆的第一状态信息；确定单元，用于根据该第一环境信息、第一状态信息和映射关系，确定第一时延优化配置信息，该映射关系包括环境信息、状态信息和时延优化配置信息的映射关系，该第一时延优化配置信息包括该车辆中的处理单元处理传感器采集的数据时的第一时延优化参数；发送单元，用于向该车辆发送该第一时延优化配置信息。
结合第六方面，在第六方面可能的实现方式中，该第一环境信息包括未来多个时刻下该车辆所处的环境信息，该第一状态信息包括该未来多个时刻下该车辆的状态信息；其中，该第一时延优化配置信息包括该未来多个时刻下处理该传感器采集的数据时的时延优化参数。
结合第六方面，在第六方面可能的实现方式中，该装置还包括：信息检索单元，用于根据该第一环境信息、该第一状态信息，采用分层可导航小世界HNSW算法对该映射关系进行检索，得到该第一时延优化配置信息。
结合第六方面，在第六方面可能的实现方式中，该装置还包括参数更新单元和存储单元，该获取单元，还用于在获取该车辆发送的第一环境信息和第一状态信息之前，获取自动驾驶车辆周围的第三环境信息以及该自动驾驶车辆的第三状态信息；该确定单元，还用于根据离线仿真系统设置的第三时延优化参数、该第三环境信息和该第三状态信息，确定处理传感器采集的数据时的第一时延；该参数更新单元，用于根据该第一时延，对该第三时延优化参数进行更新，得到第四时延优化参数；该存储单元，用于将该第三环境信息、该第三状态信息和该第四时延优化参数的对应关系，存储在该映射关系中。
结合第六方面，在第六方面可能的实现方式中，该装置还包括时延模拟单元，该获取单元，还用于在获取该车辆发送的第一环境信息和第一状态信息之前，获取自动驾驶车辆周围的第三环境信息以及该自动驾驶车辆的第三状态信息；该确定单元，还用于根据该第三环境信息和该第三状态信息数据，提取时延模型数据，该时延模型数据包括应用的线程数、线程的周期性数据、线程运行的时间概率分布和线程数据的依赖关系中的一种或者多种；该时延模拟单元，用于将该时延模型数据输入时延模拟器，得到第二时延；该确定单元，还用于根据该第二时延，确定该第三环境信息和该第三状态信息对应的第四时延优化参数；该存储单元，用于将该第三环境信息、该第三状态信息和该第四时延优化参数的对应关系，存储在该映射关系中。
结合第六方面，在第六方面可能的实现方式中，该获取单元，用于：接收来自该自动驾驶车辆的传感器采集的数据以及该第三状态信息；将该传感器采集的数据和该第三状态信息按照时间和位置进行索 引，得到该第三环境信息和该第三状态信息。
结合第六方面，在第六方面可能的实现方式中，该获取单元，用于：通过众包算法或者群智感知算法，将该传感器采集的数据和该第三状态信息按照时间和位置进行索引，得到该第三环境信息和该第三状态信息。
第七方面，本申请提供了一种数据处理装置，该装置包括处理单元和存储单元，其中存储单元用于存储指令，处理单元执行存储单元所存储的指令，以使该装置执行第一方面或者第二方面中任一种可能的数据处理方法。
第八方面，本申请提供了一种数据处理装置，该装置包括处理单元和存储单元，其中存储单元用于存储指令，处理单元执行存储单元所存储的指令，以使该装置执行第三方面中任一种可能的数据处理方法。
第九方面，本申请提供了一种数据处理系统，该控制系统包括传感器和计算平台，该计算平台包括第四方面中任一种可能的数据处理装置，或者，包括第五方面中任一种可能的数据处理装置，或者，包括第七方面中任一种可能的数据处理装置。
第十方面，本申请提供了一种车辆，该车辆包括第四方面中任一种可能的数据处理装置，或者，包括第五方面中任一种可能的数据处理装置，或者，包括第七方面所述的数据处理装置，或者，包括第九方面所述的数据处理系统。
第十一方面，本申请提供了一种服务器，该服务器包括第六方面中任一种可能的数据处理装置，或者，包括第八方面所述的数据处理装置。
第十二方面，本申请提供了一种计算机程序产品，所述计算机程序产品包括：计算机程序代码，当所述计算机程序代码在计算机上运行时，使得计算机执行上述第一方面或者第二方面中任一种可能的数据处理方法。
第十三方面，本申请提供了一种计算机程序产品，所述计算机程序产品包括：计算机程序代码，当所述计算机程序代码在计算机上运行时，使得计算机执行上述第三方面中任一种可能的数据处理方法。
需要说明的是，上述计算机程序代码可以全部或者部分存储在第一存储介质上，其中第一存储介质可以与处理器封装在一起的，也可以与处理器单独封装，本申请实施例对此不作具体限定。
第十四方面，本申请提供了一种计算机可读介质，所述计算机可读介质存储有程序代码，当所述计算机程序代码在计算机上运行时，使得计算机执行上述第一方面或者第二方面中任一种可能的数据处理方法。
第十五方面，本申请提供了一种计算机可读介质，所述计算机可读介质存储有程序代码，当所述计算机程序代码在计算机上运行时，使得计算机执行上述第三方面中任一种可能的数据处理方法。
第十六方面，本申请提供了一种芯片，该芯片包括电路，该电路用于执行上述第一方面或者第二方面中任一种可能的数据处理方法。
第十七方面，本申请提供了一种芯片，该芯片包括电路，该电路用于执行上述第三方面中任一种可能的数据处理方法。
附图说明
图1是本申请实施例提供的车辆的功能框图示意。
图2是机器人操作系统ROS中应用节点的关系示意图。
图3是本申请实施例提供的数据处理方法的示意性流程图。
图4是本申请实施例提供的系统架构的示意图。
图5是本申请实施例提供的系统架构的另一示意图。
图6是本申请实施例提供的数据处理方法的另一示意性流程图。
图7是本申请实施例提供的时延特征向量快速检索层数据结构的示例图。
图8是本申请实施例提供的数据处理方法的另一示意性流程图。
图9是本申请实施例提供的数据处理装置的示意性框图。
图10是本申请实施例提供的数据处理装置的另一示意性框图。
图11是本申请实施例提供的数据处理装置的另一示意性框图。
具体实施方式
下面将结合本申请实施例中的附图，对本申请实施例中的技术方案进行描述。其中，在本申请实施例的描述中，除非另有说明，“/”表示或的意思，例如，A/B可以表示A或B；本文中的“和/或”仅仅是一种描述关联对象的关联关系，表示可以存在三种关系，例如，A和/或B，可以表示：单独存在A，同时存在A和B，单独存在B这三种情况。“至少一项”是指一项或一项以上。例如，“A和B中的至少一项”，类似于“A和/或B”，描述关联对象的关联关系，表示可以存在三种关系，例如，A和B中的至少一项，可以表示：单独存在A，同时存在A和B，单独存在B这三种情况。
本申请实施例中采用诸如“第一”、“第二”的前缀词，仅仅为了区分不同的描述对象，对被描述对象的位置、顺序、优先级、数量或内容等没有限定作用。本申请实施例中对序数词等用于区分描述对象的前缀词的使用不对所描述对象构成限制，对所描述对象的陈述参见权利要求或实施例中上下文的描述，不应因为使用这种前缀词而构成多余的限制。此外，在本实施例的描述中，除非另有说明，“多个”的含义是两个或两个以上。
图1是本申请实施例提供的车辆100的一个功能框图示意。车辆100可以包括感知系统120和计算平台150，其中，感知系统120可以包括感测关于车辆100周边的环境的信息的一种或多种传感器。例如，感知系统120可以包括定位系统，定位系统可以是全球定位系统(global positioning system，GPS)，也可以是北斗系统或者其他定位系统。感知系统120还可以包括惯性测量单元(inertial measurement unit，IMU)、激光雷达、毫米波雷达、超声雷达以及摄像装置中的一种或者多种。
车辆100的部分或所有功能可以由计算平台150控制。计算平台150可包括一个或多个处理器，例如处理器151至15n(n为正整数)，处理器是一种具有信号的处理能力的电路，在一种实现中，处理器可以是具有指令读取与运行能力的电路，例如中央处理单元(central processing unit，CPU)、微处理器、图形处理器(graphics processing unit，GPU)(可以理解为一种微处理器)、或数字信号处理器(digital signal processor，DSP)等；在另一种实现中，处理器可以通过硬件电路的逻辑关系实现一定功能，该硬件电路的逻辑关系是固定的或可以重构的，例如处理器为专用集成电路(application-specific integrated circuit，ASIC)或可编程逻辑器件(programmable logic device，PLD)实现的硬件电路，例如现场可编程门阵列(field programmable gate array，FPGA)。在可重构的硬件电路中，处理器加载配置文档，实现硬件电路配置的过程，可以理解为处理器加载指令，以实现以上部分或全部单元的功能的过程。此外，处理器还可以是针对人工智能设计的硬件电路，其可以理解为一种ASIC，例如神经网络处理单元(neural network processing unit，NPU)、张量处理单元(tensor processing unit，TPU)、深度学习处理单元(deep learning processing unit，DPU)等。此外，计算平台150还可以包括存储器，存储器用于存储指令，处理器151至15n中的部分或全部处理器可以调用存储器中的指令，以实现相应的功能。
本申请实施例中涉及的车辆为广义概念上的车辆，可以是交通工具(如商用车、乘用车、摩托车、飞行车、火车等)，工业车辆(如：叉车、挂车、牵引车等)，工程车辆(如挖掘机、推土车、吊车等)，农用设备(如割草机、收割机等)，游乐设备，玩具车辆等，本申请实施例对车辆的类型不作具体限定。
如前所述，衡量安全性的一个非常重要指标就是自动驾驶系统的数据流链路的端到端时延，如从激光雷达、摄像头等传感器采集到原始信号开始，到向底盘域控制器发出刹车等指令结束的时延，数据流链路中每个节点的运行时间都会直接产生较大影响。较低的端到端时延(如小于100ms)，可以大幅提高车辆和行人的安全系数，是系统和应用后期优化的关键目标之一。
目前自动驾驶系统工程师常用的端到端时延优化手段，大多用部分有限的场景对自动驾驶系统和应用进行参数调整优化。参数调整优化的方法有专家经验手工调优，在线闭环反馈式的在车辆上直接调优，以及利用抽象模型、仿真器、模拟器结合AI调优算法进行调优等。但是，这些方法基本针对一些常见工况、普适环境进行配置，难以针对车辆行驶过程中不同场景进行自适应动态调整，泛化性不足；另外调整过程一般非常耗时，即使结合云端服务器的计算资源，整个过程也要好几个小时。
本申请实施例提供了一种数据处理方法、装置和车辆，车辆可以向云端服务器发送车辆周围的环境信息和车辆当前的状态信息，云端服务器可以根据环境信息、状态信息和时延优化配置信息之间的映射关系，向车辆发送与该环境信息和状态信息相对应的时延优化配置信息。车辆可以通过该时延优化配置信息处理传感器采集的数据，有助于降低端到端时延，从而有助于保证车辆和行人的安全。
在介绍本申请实施例提供的技术方案之前，首先介绍几个与本申请实施例相关的技术术语。
消息传递：计算机系统中，进程间或软件组件间的一类数据通信方法的统称。它将待通信的数据抽象并封装为“消息”，参与通信的双方或多方通过调用消息发送、接收等原语实现消息在进程或组件之间的传递，从而完成数据通信。
自动驾驶智能操作系统：基于可移植操作系统接口(portable operating system interface，POSIX)标准的操作系统，适用于自动驾驶所需要的高性能计算和高带宽通信的操作系统，提供车辆环境感知、智能决策和路径规划等功能。
应用节点：自动驾驶智能操作系统中，使用通信中间件(如机器人操作系统(robot operating system，ROS)、汽车开放系统架构(automotive open system architecture，AUTOSAR)中通信管理(communication management，CM)等)封装后的具有特定自动驾驶功能(如感知、融合和规划等)的应用，通常是单进程、多线程的集合。节点双方或多方通过调用通信中间件的方式实现消息、服务调用在进程或组件之间的传递。
消息通道：自动驾驶智能操作系统中，应用节点之间指定的通信通道，其组成包括发送者的发送端口、接收者的接收端口、通道名称、消息格式等。
当前很多自动驾驶应用的时延会受外部环境或场景的影响而波动，导致自动驾驶端到端时延变化，进而影响离线计算好的端到端时延优化配置的实际部署效果。自动驾驶算法不少是场景和数据驱动的，当自动驾驶车辆在不同环境运行时，会执行到不同的代码或模型逻辑。比如，高速、城区和地下泊车的场景变化，甚至道路上车流量变化或者车辆所处的天气变化时，应用实际执行的逻辑都有较大区别，所以端到端时延会受场景和外部环境影响，其基线和抖动情况在不同场景和环境会有较大差别。正是这种外部场景和环境的波动，会大幅影响离线配置好的单一时延优化配置效果，造成其在部分场景的效果大打折扣。甚至极端场景和环境可能有负效果，对自动驾驶系统和应用的安全性产生较大影响。
图2示出了一种ROS中应用节点的关系示意图。ROS中包括节点1、节点2、节点3和节点4，其中，节点1可以用于获取传感器采集的图像，节点2可以用于对传感器采集的图像进行图像分割，节点3可以用于针对图像分割后得到的结果识别其中的障碍物的类型、与自车之间的距离、障碍物的速度等，节点4可以用于根据节点3的识别结果确定车控指令并向底盘域控制器发送该车控指令。
上述自动驾驶应用的时延可以理解为单个节点处理数据的时延，例如，节点1处理数据的时延。
上述自动驾驶端到端时延可以理解为ROS通信系统中的节点集合处理数据的时延，例如，从节点1获取到传感器采集的图像，到节点4向底盘域控制器发送车控指令之间的时延。
图3示出了本申请实施例提供的数据处理方法300的示意性流程图。如图3所示，该方法300可以由车辆和云端服务器执行，该方法300包括步骤S310至步骤S340。
S310，车辆获取该车辆周围的第一环境信息以及该车辆的第一状态信息。
可选地，车辆获取该车辆周围的第一环境信息，包括：车辆可以根据传感器(例如，摄像头、毫米波雷达、激光雷达等)采集的数据，确定该第一环境信息。
示例性的，表1示出了车辆获取的环境信息和状态信息。
表1

以上自动驾驶模式可以理解为当前车辆开启的自动驾驶模式，例如，智驾导航辅助(navigation cruise assistant，NCA)、智能巡航辅助(integrated cruise assistant，ICA)、自动代客泊车(auto valet parking，AVP)等。辅助功能可以包括自动紧急制动(autonomous emergency braking，AEB)、车道偏离预警(lane departure warning，LDW)等。
上述表1所示的环境信息和状态信息仅仅是示意性的，车辆可以获取比表1所示的信息更多或者更少的信息。
可选地，该环境信息可以包括车辆的传感器(例如，摄像头、激光雷达、毫米波雷达、GPS、IMU中的一个或者多个)采集的数据。车辆可以将该传感器采集的数据发送给云端服务器，由云端服务器确定车辆所处的环境(例如，地理位置、时间、路况等)。
S320，车辆向云端服务器发送该第一环境信息和第一状态信息。
相应地，云端服务器可以接收车辆发送的该第一环境信息和该第一状态信息。
可选地，车辆可以根据传感器采集的数据确定该第一环境信息后，向云端服务器发送该第一环境信息和第一状态信息；或者，车辆可也可以向云端服务器发送传感器采集的数据以及该第一状态信息，由云端服务器根据该传感器采集的数据确定该第一环境信息。
可选地，车辆可以周期性得向云端服务器发送车辆周围的环境信息以及车辆的状态信息。
可选地，该云端服务器接收车辆发送的第一环境信息和第一状态信息之前，该方法300还包括：云端服务器接收来自自动驾驶车辆的第三状态信息以及该自动驾驶车辆中传感器采集的数据；云端服务器将该第三状态信息和该传感器采集的数据按照时间和位置进行索引，得到该第三环境信息和该第三状态信息。
可选地，云端服务器将该第三状态信息和该传感器采集的数据按照时间和位置进行索引，得到该第三环境信息和该第三状态信息，包括：云端服务器通过众包算法或者群智感知算法，将该第三状态信息和该传感器采集的数据按照时间和位置进行索引，得到该第三环境信息和该第三状态信息。
S330，云端服务器根据该第一环境信息、第一状态信息和映射关系，向该车辆发送第一时延优化配置信息，该映射关系包括环境信息、状态信息和时延优化配置信息的映射关系，该第一时延优化配置信息包括车辆的处理单元处理传感器采集的数据时的第一时延优化参数。
可选地，该映射关系还可以称之为时延优化配置信息检索库。云端服务器可以保存有时延优化配置信息检索库，该时延优化配置信息检索库中包括环境信息、状态信息和时延优化配置信息之间的映射关系。
可选地，该第一环境信息包括未来多个时刻下该车辆所处的环境信息，该第一状态信息包括该未来多个时刻下该车辆的状态信息；其中，该第一时延配置信息包括该未来多个时刻下处理该传感器采集的数据时的时延优化参数。
示例性的，车辆可以将导航信息(例如，从车辆当前位置至目的地的导航信息)发送给云端服务器，该云端服务器可以根据该导航信息，确定未来多个时刻下车辆周围的环境信息以及未来多个时刻下车辆的状态信息。云端服务器可以根据未来多个时刻下车辆周围的环境信息、及未来多个时刻下车辆的状态信息以及该映射关系，确定未来多个时刻下的时延优化配置信息并发送给车辆。这样，车辆可以提前保存未来多个时刻下的时延优化配置信息。在车辆周围的环境信息发生变化，和/或，车辆所处的状态信息发生变化时，可以通从未来多个时刻下的时延优化配置信息中选择合适的时延优化配置信息，处理传感器采集的数据。这样，有助于避免了车辆和云端服务器之间频繁的交互过程。
示例性的，车辆也可以根据该导航信息确定未来多个时刻下车辆周围的环境信息以及未来多个时刻下车辆的状态信息，从而向云端服务器发送未来多个时刻下车辆周围的环境信息以及未来多个时刻下车辆的状态信息。
上述第一时延优化配置信息可以包括图2所示的节点2、节点3和节点4中至少部分节点处理传感器采集的数据时的时延优化参数。
示例性的，该时延优化参数包括处理单元中任务的优先级信息、绑核信息、操作系统(例如，Linux系统)中调度器的调度参数中的一个或者多个。
可选地，该方法300还包括：在该车辆根据该第一时延优化参数确定处理该传感器采集的数据的时延大于或者等于时延阈值时，获取该车辆周围的第二环境信息以及该车辆的第二状态信息；车辆向该云端服务器发送该第二环境信息和该第二状态信息；云端服务器根据该第二环境信息、第二状态信息和该映射关系，向车辆发送第二时延优化配置信息，该第二时延优化配置信息包括该处理单元处理该传感器采集的数据时的第二时延优化参数；车辆根据该第二时延优化参数，处理该传感器采集的第二数据。
示例性的，在T1时刻，车辆所处的道路类型为高速公路、道路限速值为120公里每小时(km/h)且车辆的CPU占用率为50％，车辆可以将这些信息发送给云端服务器，从而由云端服务器根据映射关系确定时延优化配置信息1并发送给车辆。车辆可以根据该时延配置信息1，处理传感器采集的数据。
在T2时刻(例如，T1时刻之后的某个时刻)，车辆所处的道路类型为城区道路，道路限速值为80km/h且车辆的CPU占用率为80％，在车辆根据时延优化配置信息1处理传感器采集的数据的大于或者等于时延阈值(例如，100ms)时，车辆可以将T2时刻下对应的环境信息和状态信息发送给云端服务器，从而由云端服务器根据T2时刻下环境信息和状态信息，以及该映射关系确定时延优化配置信息2并发送给车辆。车辆可以根据该时延配置信息2，处理传感器采集的数据。由于环境信息和/或状态信息发生了改变，导致处理单元处理传感器采集数据的时延大于或者等于时延阈值时，车辆可以向云端服务器请求新的时延优化配置信息，从而使得车辆通过新的时延优化配置信息，处理传感器采集的数据，有助于及时降低端到端时延，也有助于提升车辆的安全性以及用户的驾乘安全。
可选地，该云端服务器向该车辆发送第一时延优化配置信息之前，该方法300还包括：根据该第一环境信息、该第一状态信息，采用HNSW算法对该映射关系进行检索，得到该第一时延优化配置信息。
可选地，该云端服务器接收车辆发送的第一环境信息和第一状态信息之前，该方法300还包括：该云端服务器获取自动驾驶车辆周围的第三环境信息以及该自动驾驶车辆的第三状态信息；根据离线仿真系统设置的第三时延优化参数、该第三环境信息和该第三状态信息，确定处理传感器采集的数据时的第一时延；根据该第一时延，对该第三时延优化参数进行更新，得到第四时延优化参数；该云端服务器将该第三环境信息、该第三状态信息和该第四时延优化参数的对应关系，保存在该映射关系中。
以上自动驾驶车辆可以理解为路测车辆，该第三环境信息和该第三状态信息可以为路测车辆采集的信息。
以上第三时延优化参数可以为预设的时延优化参数，或者，也可以为上一次优化后的时延优化参数。例如，该预设的时延优化参数指示处理单元中多个任务的优先级相同。
示例性的，云端服务器可以保存有离线仿真系统，云端服务器可以在离线仿真系统上设置自动驾驶应用进程优先级和绑核对应的参数(即第三时延优化参数)，并将该第三环境信息和该第三状态信息输入离线仿真系统。云端服务器可以监测离线仿真系统上实际运行时的端到端时延。云端服务器可以根据该实际运行时的端到端时延，对第三时延优化参数进行更新，从而给得到第四时延优化参数。云端服务 器可以将该第三环境信息、第三状态信息和该第四时延优化参数之间的对应关系保存在该映射关系中。
以上云端服务器根据该实际运行时的端到端时延对第三时延优化参数进行更新时，可以结合启发式AI调优算法、强化学习算法、深度学习算法反复迭代得到该第四时延优化参数。这样，端到端时延计算方法更为准确，部署在云端服务器前无需额外的仿真验证。
可选地，该云端服务器接收车辆发送的第一环境信息和第一状态信息之前，该方法300还包括：该云端服务器获取自动驾驶车辆周围的第三环境信息以及该自动驾驶车辆的第三状态信息；该云端服务器根据该第三环境信息和该第三状态信息数据，提取时延模型数据，该时延模型数据包括应用的线程数、线程的周期性数据、线程运行的时间概率分布和线程数据的依赖关系中的一种或者多种；该云端服务器将该时延模型数据输入时延模拟器，得到第二时延；该云端服务器根据该第二时延，确定该第三环境信息和该第三状态信息对应的第四时延优化参数；该云端服务器将该第三环境信息、该第三状态信息和该第四时延优化参数的对应关系，保存在该映射关系中。
示例性的，云端服务器可以通过自动驾驶车辆发送的第三环境信息和第三状态信息，提取在该第三环境信息下自动驾驶系统和应用的时延模型数据，时延模型数据包括但不限于应用的线程数、线程的周期性数据、线程运行时间的概率分布、线程数据的依赖关系中的一种或者多种。然后通过时延模拟器直接计算得到端到端时延，并用启发式AI调优算法、强化学习或者深度学习调优算法反复迭代得到该第四时延优化参数。该方法的优点是仿真估算时延的开销小，可以做大规模并行优化，收敛速度快。
可选地，该云端服务器接收车辆发送的第一环境信息和第一状态信息之前，该方法300还包括：该云端服务器获取自动驾驶车辆周围的第三环境信息以及该自动驾驶车辆的第三状态信息；该云端服务器将该第三环境信息和该第三状态信息数据输入预测模型，得到第四时延优化参数；该云端服务器将该第三环境信息、该第三状态信息和该第四时延优化参数，保存在该映射关系中。
示例性的，可以通过训练数据集对该预测模型进行训练，该训练数据集包括车辆所处的环境信息、车辆所处的状态信息以及该环境信息和状态信息对应的时延优化配置信息(任务的优先级信息、绑核信息、调度参数)。可以将训练好的预测模型保存在云端服务器。这样，在自动驾驶车辆(或者，路测车辆)将路测数据(包括路测车辆周围的环境信息以及路测车辆的状态信息)上传至云端服务器后，云端服务器可以将路测数据输入预测模型，从而得到对应的时延优化配置信息。云端服务器可以将环境信息、状态信息以及时延优化配置信息之间的对应关系保存在该映射关系中。
可选地，该云端服务器获取自动驾驶车辆周围的第三环境信息以及该自动驾驶车辆的第三状态信息，包括：该云端服务器接收来自该自动驾驶车辆的该第三状态信息以及该自动驾驶车辆中传感器采集的数据；该云端服务器将该第三状态信息和该传感器采集的数据按照时间和位置进行索引，得到该第三环境信息和该第三状态信息。
可选地，该云端服务器将该第三状态信息和该传感器采集的数据按照时间和位置进行索引，得到该第三环境信息和该第三状态信息，包括：通过众包算法或者群智感知算法，将该第三状态信息和该传感器采集的数据按照时间和位置进行索引，得到该第三环境信息和该第三状态信息。
S340，该车辆根据该第一时延优化参数，处理该传感器采集的第一数据。
可选地，该车辆包括主处理单元和备份处理单元，该车辆根据该第一时延优化参数，处理该传感器采集的第一数据，包括：该车辆将该第一时延优化参数部署在该备份处理单元；在第一时间段内，该备份处理单元根据该第一时延优化参数处理该第一数据的时延小于该主处理单元处理该第一数据的时延时，该车辆将该第一时延优化参数部署在该主处理单元。
示例性的，该主处理单元可以为主域，该备份处理单元可以为安全备份域。或者，该主处理单元可以为主处理器，该备份处理单元可以为备份处理器。
上述该备份处理单元根据该第一时延优化参数处理该第一数据的时延小于该主处理单元处理该第一数据的时延，可以理解为在预设时长内，该备份处理单元根据该第一时延优化参数处理该第一数据的时延的平均值小于该主处理单元处理该第一数据的时延的平均值；和/或，该备份处理单元根据该第一时延优化参数处理该第一数据的时延的最大值小于该主处理单元处理该第一数据的时延的最大值。
可选地，该车辆将该第一时延优化参数部署在该主处理单元之前，该方法300还包括：在该第一时间段内，确定该备份处理单元输出的车控指令与该主处理单元输出的车控指令相同。
图4示出了本申请实施例提供的系统架构的示意图。该系统架构中包括车辆410和云端服务器420，其中，车辆410可以包括软件平台和硬件平台，软件平台中包括自动驾驶应用411和操作系统和中间件 412，操作系统和中间件412中包括场景感知的时延监控模块4121和时延优化请求和配置模块4122。硬件平台包括CPU413、GPU414、存储器415、网卡416、传感器417和控制器418等。
云端服务器420可以包括离线场景仿真数据库421、离线场景仿真系统422和时延特征向量检索库423。其中，时延特征向量检索库可以保存有上述映射关系。
场景感知的时延监控模块4121，可以用于感知车辆410周围的环境信息，以及当前场景下的端到端时延变化情况，同时会将环境信息和车辆410的状态信息上传云端服务器420。
以上场景感知的时延监控模块4121可以向云端服务器420上传环境信息，或者，也可以向云端服务器420上传传感器417采集的数据。云端服务器420可以根据传感器417采集的数据来确定车辆410周围的环境信息。
时延优化请求和配置模块4122，可以用于向云端服务器420发送请求来优化当前场景下的端到端时延，以及接收云端服务器420发送的时延优化配置信息并将该时延优化配置信息配置在车辆410上。
离线场景仿真数据库421，可以用于记录不同车辆在不同道路、天气等不同场景中的环境数据以及车辆的状态信息，包括但不限于激光雷达采集的点云数据、摄像头采集的图像数据、毫米波雷达采集的点云数据、GPS和惯性传感器采集的数据等，以及车辆中各硬件和软件的运行日志，可以后续用于自动驾驶场景回放、模拟等。
离线场景仿真系统422，可以用于读取离线场景仿真数据库421中的环境信息和状态信息，提供不同时间、气候、地理位置和路况的自动驾驶场景级仿真和模拟功能。
时延特征向量检索库423，可以用于记录不同场景下的环境信息、状态信息以及离线场景仿真系统422计算得到的时延优化配置信息，环境信息包括但不限于时间、地理位置、气候条件、道路情况和结构等，状态信息包括但不限于硬件规格、系统状态、自动驾驶状态等。
CPU413或者GPU414，可以用于为自动驾驶系统和应用提供必要的算力资源。
网卡416，可以用于为自动驾驶系统和应用提供网络通信资源。
存储器415，可以用于为自动驾驶系统和应用提供存储资源。
传感器417，可以用于为自动驾驶系统和应用提供感知数据资源。
控制器418，可以用于为自动驾驶车辆提供底盘控制能力服务。
自动驾驶应用411通过操作系统和中间件412获取并操控运行时必要的CPU资源、GPU资源、网卡资源、存储器资源、传感器采集的数据，通过控制器418来操控车辆410的运行。
场景感知的时延监控模块4121从自动驾驶应用411运行过程中的传感器数据和应用处理数据，分析出车辆410所处的环境信息、状态信息和端到端时延信息，可存储在存储器415中，也可以直接通过网卡416发送到云端服务器420。云端服务器420可以将这些信息记录在离线场景仿真数据库421中。
时延优化请求和配置模块4122可以从场景感知的时延监控模块4121获取到端到端时延，需要对端到端时延优化时会通过网卡416向云端服务器420发送优化请求，并接收时延特征向量检索库423发送的时延优化配置信息，然后部署在当前自动驾驶操作系统和应用上。
图5示出了本申请实施例提供的系统架构的另一示意图。该系统架构中场景感知的时延监控模块4121可以包括采集监测模块41211和动态配置模块41212，时延优化请求和配置模块4122可以包括网络通信模块41221和配置部署模块41222。云端服务器420中还可以包括调度编排自动寻优系统424和时延特征向量检索模块425。
采集监测模块41211，可以使用自动驾驶应用运行过程的调度、通信等日志信息并结合算法来获取车辆410周围的环境信息、车辆410的状态信息。然后通过网络通信模块41221将这些信息发送给云端服务器420。云端服务器420可以将这些信息记录在离线场景仿真数据库421中。
动态配置模块41212，可以动态地从采集监测模块41211获取车辆410的端到端时延。在当前实际端到端时延比预期的恶化时，可以通过网络通信模块41221向云端服务器420发送优化请求，该优化请求中可以包括车辆410周围的环境信息和车辆410当前的状态信息。当从云端服务器420得到时延优化配置信息后发送给配置部署模块41222，由配置部署模块41222将其部署在自动驾驶系统和应用上，从而优化端到端时延。
调度编排自动寻优系统424，可以用于在云端服务器420上定期通过离线场景仿真系统422，从离线场景仿真数据库421中拉取数据模拟不同自动驾驶场景并观测自动驾驶关键链路数据流端到端时延，进而自动寻优找到不同环境信息和状态信息对应的时延优化配置信息，并将环境信息、状态信息和时延优 化配置信息的对应关系记录在时延特征向量检索库423中。
图6示出了本申请实施例提供的数据处理方法600的示意性流程图。该方法600可以由云端服务器420执行，该方法600包括：
S610，在云端服务器420构建基于时间和地理位置的离线仿真数据库421。
示例性的，步骤S610可以包括两个阶段：
阶段一是自动驾驶车辆(或者路测车辆)运行过程中自动采集传感器数据和状态信息并上传云端服务器420。
阶段二是在云端服务器420将各个自动驾驶车辆(或者路测车辆)上传的数据根据地理位置和时间整合，形成离线场景仿真数据库421。
示例性的，云端服务器420将各个自动驾驶车辆上传的数据根据地理位置和时间，通过众包或者群智感知技术整合，形成离线场景仿真数据库421。
阶段一中自动驾驶车辆需要采集的数据包括但不限于激光雷达的点云数据、摄像头的图像数据、毫米波雷达的点云数据、GPS、IMU等传感器的数据等，以及天气、温度、湿度、光强、可见度等信息。阶段二会将阶段一中不同来源的数据通过时间序列、地理位置坐标作为主索引，通过群智感知算法或众包算法将零散的数据构建为统一的离线场景仿真数据库，可回放、模拟各种自动驾驶场景，用于以后的测试和优化。
S620，通过离线仿真系统422和调度编排自动寻优系统424，获取某个时间切片下的车辆周围的环境信息、车辆的状态信息以及时延优化配置信息之间的对应关系，并将该对应关系保存在时延特征向量检索库423中。
步骤S620可以基于步骤S610的离线场景仿真数据库421中的数据，在云端服务器420仿真计算出某个时间切片下环境信息和状态信息对应的时延优化配置信息，然后将时间切片下的环境信息、状态信息以及对应的时延优化配置信息保存在时延特征向量检索库423中。
步骤S620可以在离线场景仿真数据库421初始构建后即可开展，后续离线场景仿真数据库421更新时也可同步进行。
示例性的，离线场景仿真系统422会采样式地或遍历式地从离线场景仿真数据库421中选取不同时间切片下的环境信息和状态信息，通过离线场景仿真系统422仿真回放不同场景下自动驾驶系统应用的端到端时延。然后通过机器学习算法确定时延优化配置信息。
以上步骤S610和步骤S620可以在车辆410行驶之前完成。
S630，接收车辆410发送的优化请求并向车辆410发送时延优化配置信息，该优化请求中包括车辆410周围的环境信息以及车辆410的状态信息。
示例性的，车辆410可以周期性得向云端服务器420上传车辆410周围的环境信息和车辆410的状态信息。云端服务器420可以根据车辆410发送的环境信息和状态信息，从时延特征向量检索库423检索得到时延优化配置信息并发送给车辆410。
以上步骤S630可以发生在车辆410行驶过程中。
可选地，该时延优化配置信息中除了包括任务的优先级、绑核信息、调度参数等以外，还可以包括时延范围。当车辆410根据该时延优化配置信息，处理传感器采集的数据的时延不在该时延范围时，车辆410可以重新向云端服务器发送优化请求。
以上硬件平台中的硬件可以是自动驾驶硬件处理平台，其中AI CPU和AI Core分别提供了CPU和GPU算力资源，以太网卡提供了千兆网卡通信能力，通用闪存存储(universal flash storage，UFS)提供了大容量存储资源，传感器提供了点云数据、图像数据、GPS等数据，控制器提供底盘控制能力。以上操作系统可以为Linux系统。
自动驾驶硬件处理平台的采集监测模块41211在路测车辆自动驾驶过程中采集并记录大量原始传感器数据，包括但不限于激光雷达的点云坐标数据、摄像头的图像和视频数据、GPS的时空坐标数据、IMU的时序数据等。由于数据量较大，此阶段数据会在自动驾驶过程中通过CM通信模块存在UFS中。等到路测结束时，直接将UFS中存储的信息通过以太网卡或存储器读写器外接工作站上传到云端服务器。同时也会将自动驾驶过程中的车辆的状态信息一起上传云端服务器，包括但不限于车架信息、车辆状态、硬件SoC资源使用信息、网络状态、系统软件运行状态、车辆自驾状态信息等。
云端服务器接收不同车辆的不同路测数据后，解析每趟路测数据的时间、起止位置、路线等，然后 将收集到的传感器数据和状态信息数据按照时间和GPS坐标位置对齐，记录为自动驾驶场景数据的时空数据库，即离线场景仿真数据库。同时，也会使用启发式算法和基于神经网络的AI算法自动解析出每趟路测数据的天气状况、节气信息、道路拥堵状况，建立除时间和GPS坐标以外的索引，方便后续离线场景仿真系统快速检索和使用。
云端服务器420的离线场景仿真系统422可以从步骤S610构建的离线场景仿真数据库421中调用数据，用于模拟不同时间、地理位置、天气、路况等不同自动驾驶场景，对接到软硬件平台或自动驾驶系统与应用上，可以测量自动驾驶应用运行结果准确性和整系统性能。可选地，考虑到真实世界采集并上传的场景数据(包括环境信息和状态信息)是有限的，离线场景仿真系统使用了增强AI算法，可以将不同路测数据中的场景数据拼接在同一个仿真场景内，也可以通过产生式AI技术在回放数据中增加原先不存在的车辆、障碍物和行人等，甚至改变原场景中的特征，比如改变天气、车流量、道路路面属性等。
在离线场景仿真过程中，离线场景仿真系统会以固定频率(如1Hz或0.1Hz)，提取时间窗口内的环境信息和状态信息，形成时间序列化数据。该时间序列化数据可以如上述表1所示。
离线场景仿真系统422将所有时间切片内提取到的环境信息和状态信息构建为时延特征向量检索库423。这些环境信息和状态信息可以独立供以后自动驾驶车辆检索使用，其中的部分数据是与场景无关的数据，如硬件规格特征数据。有的数据随时间和场景的变化，波动比较大，如系统状态特征。时延特征向量检索库423建立好以后可以进行定期维护，如每个自动驾驶软件或者硬件迭代更新时，需要重复上述步骤S610和S620。
与此同时，基于AI算法的调度编排自动寻优系统424为仿真的每个时间切片进行端到端时延优化，输入为环境信息和状态信息，输出为该时间切片的时延优化配置信息。调度编排自动寻优系统424可以将输出更新到时延特征向量检索库423中，从而建立起环境信息、状态信息和时延优化配置信息的对应关系。本申请实施例中，可以通过如下所述的两种方法来确定该时间切片下的时延优化配置信息。
方法一
直接在离线仿真系统422上设置自动驾驶应用中任务的优先级信息和绑核信息，看离线仿真系统422上实际运行时的端到端时延，结合启发式AI调优算法、强化学习或者深度学习调优算法反复迭代算出该时间切片下的时延优化配置信息。该方法一为任务的优先级和绑核的调整。该方法一的优点是端到端时延计算方法更加精准，部署前无需额外的仿真验证。
以上离线仿真系统422可以包括车辆的动力学仿真系统以及自动驾驶仿真系统。
方法二
先结合时间切片内提取的状态信息，提取生成该时间切片下环境信息对应的自动驾驶系统和应用的时延模型数据，时延模型数据包括但不限于应用的线程数、线程的周期性数据、线程运行时间的概率分布、线程数据的依赖关系等。通过将时延模型数据输入时延模拟器直接得到端到端时延，结合启发式AI调优算法、强化学习或者深度学习调优算法反复迭代算出该时间切片下的时延优化配置信息。该方法二的优点是仿真估算时延的开销小，可以做大规模并行优化，收敛速度快。
考虑到时延特征向量检索库数据量非常大，本申请实施例基于HNSW技术，结合了跳跃表的思想，为原始链表构建了时延特征向量快速检索层。因为同一趟路测数据产生的时延特征向量具有较强的局部性，在同一个时间窗口内和同一块地理位置内，有大量相同的冗余数据，因此可以将时延特征向量按照变化程度排序并合并掉冗余，例如硬件规格、系统版本和状态在同一趟路测是不变的。依照时延特征向量的稠密度进行稀疏分层，先从稀疏层查询起，符合条件的再逐级靠下向稠密层查询。这样可以提升对时延优化配置信息的检索速度，降低云端服务器的响应时延。图7示出了本申请实施例提供的时延特征向量快速检索层数据结构的示例图。
图8示出了本申请实施例提供的数据处理方法800的示意性流程图。该方法800可以由车辆执行，该方法800包括：
S810，车辆判断时间窗口内处理单元处理传感器采集的数据的时延是否大于或者等于时延阈值。
在该时间窗口内处理传感器采集的数据的时延大于或者等于时延阈值时，执行S820；否则，返回继续执行S810。
示例性的，上述步骤S810可以通过如下两种方法实现：
方法一
通过使用基于AI的算法，动态地对传感器感知的数据和车辆的状态信息进行推理，用预训练的模型 推理出从外部环境信息、车辆状态信息到车辆所处场景的变化。
方法二
通过监测车辆的端到端时延。当预设时长内，端到端时延均值或抖动超过时延阈值时，表明车辆的外部场景变化了，需要重新部署时延优化配置信息。方法二的优点是占用的自动驾驶软硬件平台资源少，计算开销小，且对场景改变时重新部署时延优化配置信息的响应速度快、准确率高。
S820，车辆向云端服务器发送优化请求，该优化请求包括车辆周围的环境信息和车辆的状态信息。
示例性的，动态配置模块41212将车辆周围的环境信息和车辆的状态信息发送给云端服务器，等待云端服务器在时延特征向量检索库中重新搜索适合当前场景的时延优化配置信息并发送给车辆。通过云端服务器可以降低搜索响应时间短，且可以利用云端服务器中海量计算资源，准确寻找合适的时延优化配置信息，且不占用车辆本地计算资源，版本更新时只需更新云端服务器中的时延特征向量索引库，车辆不感知。
以上是以时延特征向量检索库部署在云端服务器时，车辆向云端服务器发送优化请求为例进行说明的，本申请实施例并不限于此。该时延特征向量检索库也可以部署在车辆中。
示例性的，云端服务器可以通过模型有损压缩技术将时延特征向量检索库部署在车辆的自动驾驶硬件处理平台上。在车辆处于自动驾驶状态时，如果车辆的端到端时延大于或者等于时延阈值时，动态配置模块41212可以根据车辆周围的环境信息和车辆的状态信息，在压缩的时延特征向量检索库重新搜索适合当前场景的时延优化配置信息。这样，车辆可以在不联网的情况下及时获取到更新后的时延优化配置信息，可以更好的符合功能安全认证。
S830，车辆接收云端服务器根据该环境信息和状态信息确定的时延优化配置信息，该时延优化配置信息中包括时延优化配置参数。
S840，车辆将该时延优化配置参数部署在安全备份域上。
S850，在预设时长内，主域中处理传感器采集的数据的时延大于安全备份域处理传感器采集的数据的时延时，车辆将该时延优化配置信息部署在主域上。
以上主域和安全备份域可以对相同的传感器采集的数据进行处理。
示例性的，动态配置模块从云端服务器的时延特征向量检索库中获取最新的时延优化配置信息后，发给配置部署模块41222。配置部署模块41222将该时延优化配置信息部署在自动驾驶的安全备份域中，观察其端到端时延优化效果。该安全备份域和主域可以均处于自动驾驶硬件处理平台上，且均部署全量的自动驾驶应用。如果在预设时长内主域和安全备份域输出的车控指令相同且安全备份域处理传感器采集的数据的时延优于主域处理传感器采集的数据的时延，配置部署模块41222可以将时延优化配置信息部署在主域中。
图9示出了本申请实施例提供的数据处理装置900的示意性框图。如图9所示，该装置900包括：获取单元910，用于获取车辆周围的第一环境信息以及该车辆的第一状态信息；发送单元920，用于向云端服务器发送该第一环境信息和该第一状态信息；接收单元930，接收该云端服务器发送的第一时延优化配置信息，该云端服务器保存有该第一环境信息、该第一状态信息和该第一时延优化配置信息的映射关系，该第一时延优化配置信息包括该车辆中的处理单元处理传感器采集的数据时的第一时延优化参数；该处理单元940，用于根据该第一时延优化参数，处理该传感器采集的第一数据。
可选地，该获取单元910，用于在该处理单元根据该第一时延优化参数处理该传感器采集的数据时的时延大于或者等于时延阈值时，获取该车辆周围的第二环境信息以及该车辆的第二状态信息；该发送单元920，用于向该云端服务器发送该第二环境信息和该第二状态信息；该接收单元930，用于接收该云端服务器发送的第二时延优化配置信息，该云端服务器保存有该第二环境信息、该第二状态信息和该第二时延优化配置信息的映射关系，该第二时延优化配置信息包括该处理单元处理该传感器采集的数据时的第二时延优化参数；该处理单元940，用于根据该第二时延优化参数，处理该传感器采集的第二数据。
可选地，该处理单元940包括主处理单元和备份处理单元，该处理单元940，用于将该第一时延优化参数部署在该备份处理单元；在第一时间段内，该备份处理单元根据该第一时延优化参数处理该第一数据时的时延小于该主处理单元处理该第一数据时的时延时，将该第一时延优化参数部署在该主处理单元。
可选地，该装置900还包括：确定单元，用于在该第一时间段内，确定该备份处理单元输出的车控指令与该主处理单元输出的车控指令相同。
可选地，该第一环境信息包括未来多个时刻下该车辆所处的环境信息，该第一状态信息包括该未来 多个时刻下该车辆的状态信息；其中，该第一时延配置信息包括该未来多个时刻下该处理单元处理该传感器采集的数据时的时延优化参数。
可选地，该第一环境信息包括时间、地理位置、气候和路况中的至少一项；和/或，该第一状态信息包括该车辆的硬件规格、系统状态、自动驾驶状态中的至少一项。
例如，获取单元910可以是图1中的计算平台或者计算平台中的处理电路、处理器或者控制器。以获取单元910为计算平台中的处理器121为例，处理器121可以获取车辆周围的环境信息和车辆的状态信息。
又例如，发送单元920和接收单元930可以是车辆100中的通信装置。
又例如，处理单元940可以是图1中的计算平台或者计算平台中的处理电路、处理器或者控制器。以处理单元940为计算平台中的处理器122为例，处理器122可以根据云端服务器下发的时延优化配置信息，对传感器采集的数据进行处理。
以上获取单元910所实现的功能、处理单元940所实现的功能可以由不同的处理器实现，或者，还可以由相同的处理器实现，本申请实施例对此不作限定。
图10示出了本申请实施例提供了数据处理装置1000的示意性流程图。如图10所示，该装置1000包括：获取单元1010，用于获取车辆周围的第一环境信息以及该车辆的第一状态信息；确定单元1020，用于根据该第一环境信息、该第一状态信息以及映射关系，确定第一时延优化配置信息，该映射关系包括该第一环境信息、该第一状态信息和该第一时延优化配置信息的映射关系，该第一时延优化配置信息包括该车辆中的处理单元处理传感器采集的数据时的第一时延优化参数；该处理单元1030，用于根据该第一时延优化参数，处理该传感器采集的第一数据。
可选地，该获取单元1010，还用于在该处理单元根据该第一时延优化参数处理该传感器采集的数据时的时延大于或者等于时延阈值时，获取该车辆周围的第二环境信息以及该车辆的第二状态信息；该确定单元1020，还用于根据该第二环境信息、该第二状态信息以及该映射关系，确定第二时延优化配置信息，该映射关系包括该第二环境信息、该第二状态信息和该第二时延优化配置信息的映射关系，该第二时延优化配置信息包括该处理单元处理传感器采集的数据时的第二时延优化参数；该处理单元1030，还用于根据该第二时延优化参数，处理该传感器采集的第二数据。
可选地，该处理单元1030包括主处理单元和备份处理单元，该处理单元1030，用于将该第一时延优化参数部署在该备份处理单元；在第一时间段内，该备份处理单元根据该第一时延优化参数处理该第一数据的时延小于该主处理单元处理该第一数据的时延时，将该第一时延优化参数部署在该主处理单元。
可选地，该确定单元1020，还用于在该第一时间段内，确定该备份处理单元输出的车控指令与该主处理单元输出的车控指令相同。
可选地，该第一环境信息包括未来多个时刻下该车辆所处的环境信息，该第一状态信息包括该未来多个时刻下该车辆的状态信息；其中，该第一时延配置信息包括该未来多个时刻下处理该传感器采集的数据时的时延优化参数。
例如，获取单元1010可以是图1中的计算平台或者计算平台中的处理电路、处理器或者控制器。以获取单元1010为计算平台中的处理器121为例，处理器121可以获取车辆周围的环境信息以及该车辆的状态信息。
又例如，确定单元1020可以是图1中的计算平台或者计算平台中的处理电路、处理器或者控制器。以确定单元1020为计算平台中的处理器122为例，处理器122可以保存有环境信息、状态信息和时延优化配置信息的映射关系。处理器122可以根据获取的环境信息、状态信息和该映射关系，确定当前场景下合适的时延优化配置信息。
又例如，处理单元1030可以是图1中的计算平台或者计算平台中的处理电路、处理器或者控制器。以控制单元1030为计算平台中的处理器123为例，处理器123可以时延优化配置信息，对传感器采集的数据进行处理。
以上获取单元1010所实现的功能、确定单元1020所实现的功能和处理单元1030所实现的功能可以由不同的处理器实现，或者，还可以由相同的处理器实现，或者，也可以是部分功能由相同的处理器实现。本申请实施例对此不作限定。
图11示出了本申请实施例提供了数据处理装置1100的示意性框图。如图11所示，该装置1100包括：获取单元1110，用于获取车辆周围的第一环境信息以及该车辆的第一状态信息；确定单元1120，用于根 据该第一环境信息、第一状态信息和映射关系，确定第一时延优化配置信息，该映射关系包括环境信息、状态信息和时延优化配置信息的映射关系，该第一时延优化配置信息包括该车辆中的处理单元处理传感器采集的数据时的第一时延优化参数；发送单元1130，用于向该车辆发送该第一时延优化配置信息。
可选地，该第一环境信息包括未来多个时刻下该车辆所处的环境信息，该第一状态信息包括该未来多个时刻下该车辆的状态信息；其中，该第一时延优化配置信息包括该未来多个时刻下处理该传感器采集的数据时的时延优化参数。
可选地，该装置1100还包括：信息检索单元，用于根据该第一环境信息、该第一状态信息，采用分层可导航小世界HNSW算法对该映射关系进行检索，得到该第一时延优化配置信息。
可选地，该装置1100还包括参数更新单元和存储单元，该获取单元1110，还用于在获取该车辆发送的第一环境信息和第一状态信息之前，获取自动驾驶车辆周围的第三环境信息以及该自动驾驶车辆的第三状态信息；该确定单元1120，还用于根据离线仿真系统设置的第三时延优化参数、该第三环境信息和该第三状态信息，确定处理传感器采集的数据时的第一时延；该参数更新单元，用于根据该第一时延，对该第三时延优化参数进行更新，得到第四时延优化参数；该存储单元，用于将该第三环境信息、该第三状态信息和该第四时延优化参数的对应关系，存储在该映射关系中。
可选地，该装置还包括时延模拟单元，该获取单元1110，还用于在获取该车辆发送的第一环境信息和第一状态信息之前，获取自动驾驶车辆周围的第三环境信息以及该自动驾驶车辆的第三状态信息；该确定单元1120，还用于根据该第三环境信息和该第三状态信息数据，提取时延模型数据，该时延模型数据包括应用的线程数、线程的周期性数据、线程运行的时间概率分布和线程数据的依赖关系中的一种或者多种；该时延模拟单元，用于将该时延模型数据输入时延模拟器，得到第二时延；该确定单元1120，还用于根据该第二时延，确定该第三环境信息和该第三状态信息对应的第四时延优化参数；该存储单元，用于将该第三环境信息、该第三状态信息和该第四时延优化参数的对应关系，存储在该映射关系中。
可选地，该获取单元1110，用于：接收来自该自动驾驶车辆的传感器采集的数据以及该第三状态信息；将该传感器采集的数据和该第三状态信息按照时间和位置进行索引，得到该第三环境信息和该第三状态信息。
可选地，该获取单元1110，用于：通过众包算法或者群智感知算法，将该传感器采集的数据和该第三状态信息按照时间和位置进行索引，得到该第三环境信息和该第三状态信息。
应理解以上装置中各单元的划分仅是一种逻辑功能的划分，实际实现时可以全部或部分集成到一个物理实体上，也可以物理上分开。此外，装置中的单元可以以处理器调用软件的形式实现；例如装置包括处理器，处理器与存储器连接，存储器中存储有指令，处理器调用存储器中存储的指令，以实现以上任一种方法或实现该装置各单元的功能，其中处理器例如为通用处理器，例如CPU或微处理器，存储器为装置内的存储器或装置外的存储器。或者，装置中的单元可以以硬件电路的形式实现，可以通过对硬件电路的设计实现部分或全部单元的功能，该硬件电路可以理解为一个或多个处理器；例如，在一种实现中，该硬件电路为ASIC，通过对电路内元件逻辑关系的设计，实现以上部分或全部单元的功能；再如，在另一种实现中，该硬件电路为可以通过PLD实现，以FPGA为例，其可以包括大量逻辑门电路，通过配置文件来配置逻辑门电路之间的连接关系，从而实现以上部分或全部单元的功能。以上装置的所有单元可以全部通过处理器调用软件的形式实现，或全部通过硬件电路的形式实现，或部分通过处理器调用软件的形式实现，剩余部分通过硬件电路的形式实现。
以上装置中的各单元可以是被配置成实施以上方法的一个或多个处理器(或处理电路)，例如：CPU、GPU、NPU、TPU、DPU、微处理器、DSP、ASIC、FPGA，或这些处理器形式中至少两种的组合。
此外，以上装置中的各单元可以全部或部分可以集成在一起，或者可以独立实现。在一种实现中，这些单元集成在一起，以SoC的形式实现。该SoC中可以包括至少一个处理器，用于实现以上任一种方法或实现该装置各单元的功能，该至少一个处理器的种类可以不同，例如包括CPU和FPGA，CPU和人工智能处理器，CPU和GPU等。
本申请实施例还提供了一种装置，该装置包括处理单元和存储单元，其中存储单元用于存储指令，处理单元执行存储单元所存储的指令，以使该装置执行上述实施例执行的方法或者步骤。
可选地，若该装置位于车辆中，上述处理单元可以是图1所示的处理器121-12n。
本申请实施例还提供了一种数据处理系统，该路径规划系统包括一个或者多个传感器和计算平台，其中，该计算平台包括上述数据处理装置900或者数据处理装置1000。
本申请实施例还提供了一种车辆，该车辆可以包括上述数据处理装置900，或者，包括上述数据处理装置1000，或者，包括上述数据处理系统。
本申请实施例还提供了一种服务器，该服务器包括上述数据处理装置1100。
本申请实施例还提供了一种计算机程序产品，所述计算机程序产品包括：计算机程序代码，当所述计算机程序代码在计算机上运行时，使得计算机执行上述数据处理方法。
本申请实施例还提供了一种计算机可读介质，所述计算机可读介质存储有程序代码，当所述计算机程序代码在计算机上运行时，使得计算机执行上述数据处理方法。
在实现过程中，上述方法的各步骤可以通过处理器中的硬件的集成逻辑电路或者软件形式的指令完成。结合本申请实施例所公开的方法可以直接体现为硬件处理器执行完成，或者用处理器中的硬件及软件模块组合执行完成。软件模块可以位于随机存储器，闪存、只读存储器，可编程只读存储器或者上电可擦写可编程存储器、寄存器等本领域成熟的存储介质中。该存储介质位于存储器，处理器读取存储器中的信息，结合其硬件完成上述方法的步骤。为避免重复，这里不再详细描述。
应理解，本申请实施例中，该存储器可以包括只读存储器和随机存取存储器，并向处理器提供指令和数据。
还应理解，在本申请的各种实施例中，上述各过程的序号的大小并不意味着执行顺序的先后，各过程的执行顺序应以其功能和内在逻辑确定，而不应对本申请实施例的实施过程构成任何限定。
本领域普通技术人员可以意识到，结合本文中所公开的实施例描述的各示例的单元及算法步骤，能够以电子硬件、或者计算机软件和电子硬件的结合来实现。这些功能究竟以硬件还是软件方式来执行，取决于技术方案的特定应用和设计约束条件。专业技术人员可以对每个特定的应用来使用不同方法来实现所描述的功能，但是这种实现不应认为超出本申请的范围。
所属领域的技术人员可以清楚地了解到，为描述的方便和简洁，上述描述的系统、装置和单元的具体工作过程，可以参考前述方法实施例中的对应过程，在此不再赘述。
在本申请所提供的几个实施例中，应该理解到，所揭露的系统、装置和方法，可以通过其它的方式实现。例如，以上所描述的装置实施例仅仅是示意性的，例如，所述单元的划分，仅仅为一种逻辑功能划分，实际实现时可以有另外的划分方式，例如多个单元或组件可以结合或者可以集成到另一个系统，或一些特征可以忽略，或不执行。另一点，所显示或讨论的相互之间的耦合或直接耦合或通信连接可以是通过一些接口，装置或单元的间接耦合或通信连接，可以是电性，机械或其它的形式。
所述作为分离部件说明的单元可以是或者也可以不是物理上分开的，作为单元显示的部件可以是或者也可以不是物理单元，即可以位于一个地方，或者也可以分布到多个网络单元上。可以根据实际的需要选择其中的部分或者全部单元来实现本实施例方案的目的。
另外，在本申请各个实施例中的各功能单元可以集成在一个处理单元中，也可以是各个单元单独物理存在，也可以两个或两个以上单元集成在一个单元中。
所述功能如果以软件功能单元的形式实现并作为独立的产品销售或使用时，可以存储在一个计算机可读取存储介质中。基于这样的理解，本申请的技术方案本质上或者说对现有技术做出贡献的部分或者该技术方案的部分可以以软件产品的形式体现出来，该计算机软件产品存储在一个存储介质中，包括若干指令用以使得一台计算机设备(可以是个人计算机，服务器，或者网络设备等)执行本申请各个实施例所述方法的全部或部分步骤。而前述的存储介质包括：U盘、移动硬盘、只读存储器(read-only memory，ROM)、随机存取存储器(random access memory，RAM)、磁碟或者光盘等各种可以存储程序代码的介质。
以上所述，仅为本申请的具体实施方式，但本申请的保护范围并不局限于此，任何熟悉本技术领域的技术人员在本申请揭露的技术范围内，可轻易想到变化或替换，都应涵盖。在本申请的保护范围之内。因此，本申请的保护范围应以所述权利要求的保护范围为准。

Claims (27)

  1. 一种数据处理方法，其特征在于，包括：
    获取车辆周围的第一环境信息以及所述车辆的第一状态信息；
    向云端服务器发送所述第一环境信息和所述第一状态信息；
    接收所述云端服务器发送的第一时延优化配置信息，所述云端服务器保存有所述第一环境信息、所述第一状态信息和所述第一时延优化配置信息的映射关系，所述第一时延优化配置信息包括所述车辆中的处理单元处理传感器采集的数据时的第一时延优化参数；
    根据所述第一时延优化参数，处理所述传感器采集的第一数据。
  2. 根据权利要求1所述的方法，其特征在于，所述方法还包括：
    在所述处理单元根据所述第一时延优化参数处理所述传感器采集的数据的时延大于或者等于时延阈值时，获取所述车辆周围的第二环境信息以及所述车辆的第二状态信息；
    向所述云端服务器发送所述第二环境信息和所述第二状态信息；
    接收所述云端服务器发送的第二时延优化配置信息，所述云端服务器保存有所述第二环境信息、所述第二状态信息和所述第二时延优化配置信息的映射关系，所述第二时延优化配置信息包括所述处理单元处理所述传感器采集的数据时的第二时延优化参数；
    根据所述第二时延优化参数，处理所述传感器采集的第二数据。
  3. 根据权利要求1或2所述的方法，其特征在于，所述处理单元包括主处理单元和备份处理单元，所述根据所述第一时延优化参数，处理所述传感器采集的第一数据，包括：
    将所述第一时延优化参数部署在所述备份处理单元；
    在第一时间段内，所述备份处理单元根据所述第一时延优化参数处理所述第一数据的时延小于所述主处理单元处理所述第一数据的时延时，将所述第一时延优化参数部署在所述主处理单元。
  4. 根据权利要求3所述的方法，其特征在于，所述将所述第一时延优化参数部署在所述主处理单元之前，所述方法还包括：
    在所述第一时间段内，确定所述备份处理单元输出的车控指令与所述主处理单元输出的车控指令相同。
  5. 根据权利要求1至4中任一项所述的方法，其特征在于，所述第一环境信息包括未来多个时刻下所述车辆所处的环境信息，所述第一状态信息包括所述未来多个时刻下所述车辆的状态信息；
    其中，所述第一时延配置信息包括所述未来多个时刻下所述处理单元处理所述传感器采集的数据时的时延优化参数。
  6. 根据权利要求1至5中任一项所述的方法，其特征在于，所述第一环境信息包括时间、地理位置、气候和路况中的至少一项；和/或，
    所述第一状态信息包括所述车辆的硬件规格、系统状态、自动驾驶状态中的至少一项。
  7. 一种数据处理方法，其特征在于，包括：
    获取车辆周围的第一环境信息以及所述车辆的第一状态信息；
    根据所述第一环境信息、第一状态信息和映射关系，向所述车辆发送第一时延优化配置信息，所述映射关系包括环境信息、状态信息和时延优化配置信息的映射关系，所述第一时延优化配置信息包括所述车辆中的处理单元处理传感器采集的数据时的第一时延优化参数。
  8. 根据权利要求7所述的方法，其特征在于，所述第一环境信息包括未来多个时刻下所述车辆所处的环境信息，所述第一状态信息包括所述未来多个时刻下所述车辆的状态信息；
    其中，所述第一时延优化配置信息包括所述未来多个时刻下处理所述传感器采集的数据时的时延优化参数。
  9. 根据权利要求7或8所述的方法，其特征在于，所述方法还包括：
    根据所述第一环境信息、所述第一状态信息，采用分层可导航小世界HNSW算法对所述映射关系进行检索，得到所述第一时延优化配置信息。
  10. 根据权利要求7至9中任一项所述的方法，其特征在于，所述获取车辆周围的第一环境信息以及所述车辆的第一状态信息之前，所述方法还包括：
    获取自动驾驶车辆周围的第三环境信息以及所述自动驾驶车辆的第三状态信息；
    根据离线仿真系统设置的第三时延优化参数、所述第三环境信息和所述第三状态信息，确定处理传感器采集的数据时的第一时延；
    根据所述第一时延，对所述第三时延优化参数进行更新，得到第四时延优化参数；
    将所述第三环境信息、所述第三状态信息和所述第四时延优化参数的对应关系，存储在所述映射关系中。
  11. 根据权利要求7至9中任一项所述的方法，其特征在于，所述获取车辆周围的第一环境信息以及所述车辆的第一状态信息之前，所述方法还包括：
    获取自动驾驶车辆周围的第三环境信息以及所述自动驾驶车辆的第三状态信息；
    根据所述第三环境信息和所述第三状态信息，提取时延模型数据，所述时延模型数据包括应用的线程数、线程的周期性数据、线程运行的时间概率分布和线程数据的依赖关系中的一种或者多种；
    将所述时延模型数据输入时延模拟器，得到第二时延；
    根据所述第二时延，确定所述第三环境信息和所述第三状态信息对应的第四时延优化参数；
    将所述第三环境信息、所述第三状态信息和所述第四时延优化参数的对应关系，存储在所述映射关系中。
  12. 一种数据处理装置，其特征在于，包括：
    获取单元，用于获取车辆周围的第一环境信息以及所述车辆的第一状态信息；
    发送单元，用于向云端服务器发送所述第一环境信息和所述第一状态信息；
    接收单元，用于接收所述云端服务器发送的第一时延优化配置信息，所述云端服务器保存有所述第一环境信息、所述第一状态信息和所述第一时延优化配置信息的映射关系，所述第一时延优化配置信息包括所述车辆中的处理单元处理传感器采集的数据时的第一时延优化参数；
    所述处理单元，用于根据所述第一时延优化参数，处理所述传感器采集的第一数据。
  13. 根据权利要求12所述的装置，其特征在于，
    所述获取单元，还用于在所述处理单元根据所述第一时延优化参数处理所述传感器采集的数据的时延大于或者等于时延阈值时，获取所述车辆周围的第二环境信息以及所述车辆的第二状态信息；
    所述发送单元，还用于向所述云端服务器发送所述第二环境信息和所述第二状态信息；
    所述接收单元，还用于接收所述云端服务器发送的第二时延优化配置信息，所述云端服务器保存有所述第二环境信息、所述第二状态信息和所述第二时延优化配置信息的映射关系，所述第二时延优化配置信息包括所述处理单元处理所述传感器采集的数据时的第二时延优化参数；
    所述处理单元，还用于根据所述第二时延优化参数，处理所述传感器采集的第二数据。
  14. 根据权利要求12或13所述的装置，其特征在于，所述处理单元包括主处理单元和备份处理单元，所述处理单元，用于：
    将所述第一时延优化参数部署在所述备份处理单元；
    在第一时间段内，所述备份处理单元根据所述第一时延优化参数处理所述第一数据的时延小于所述主处理单元处理所述第一数据的时延时，将所述第一时延优化参数部署在所述主处理单元。
  15. 根据权利要求14所述的装置，其特征在于，所述装置还包括：
    确定单元，用于在所述处理单元将所述第一时延优化参数部署在所述主处理单元之前，确定在所述第一时间段内所述备份处理单元输出的车控指令与所述主处理单元输出的车控指令相同。
  16. 根据权利要求12至15中任一项所述的装置，其特征在于，所述第一环境信息包括未来多个时刻下所述车辆所处的环境信息，所述第一状态信息包括所述未来多个时刻下所述车辆的状态信息；
    其中，所述第一时延配置信息包括所述未来多个时刻下所述处理单元处理所述传感器采集的数据时的时延优化参数。
  17. 根据权利要求12至16中任一项所述的装置，其特征在于，所述第一环境信息包括时间、地理位置、气候和路况中的至少一项；和/或，
    所述第一状态信息包括所述车辆的硬件规格、系统状态、自动驾驶状态中的至少一项。
  18. 一种数据处理装置，其特征在于，包括：
    获取单元，用于获取车辆周围的第一环境信息以及所述车辆的第一状态信息；
    确定单元，用于根据所述第一环境信息、第一状态信息和映射关系，确定第一时延优化配置信息， 所述映射关系包括环境信息、状态信息和时延优化配置信息的映射关系，所述第一时延优化配置信息包括所述车辆中的处理单元处理传感器采集的数据时的第一时延优化参数；
    发送单元，用于向所述车辆发送所述第一时延优化配置信息。
  19. 根据权利要求18所述的装置，其特征在于，所述第一环境信息包括未来多个时刻下所述车辆所处的环境信息，所述第一状态信息包括所述未来多个时刻下所述车辆的状态信息；
    其中，所述第一时延优化配置信息包括所述未来多个时刻下处理所述传感器采集的数据时的时延优化参数。
  20. 根据权利要求18或19所述的装置，其特征在于，所述装置还包括：
    信息检索单元，用于根据所述第一环境信息、所述第一状态信息，采用分层可导航小世界HNSW算法对所述映射关系进行检索，得到所述第一时延优化配置信息。
  21. 根据权利要求18至20中任一项所述的装置，其特征在于，所述装置还包括参数更新单元和存储单元，
    所述获取单元，还用于获取自动驾驶车辆周围的第三环境信息以及所述自动驾驶车辆的第三状态信息；
    所述确定单元，还用于根据离线仿真系统设置的第三时延优化参数、所述第三环境信息和所述第三状态信息，确定处理传感器采集的数据的第一时延；
    所述参数更新单元，还用于根据所述第一时延，对所述第三时延优化参数进行更新，得到第四时延优化参数；
    所述储存单元，还用于将所述第三环境信息、所述第三状态信息和所述第四时延优化参数的对应关系，存储在所述映射关系中。
  22. 根据权利要求18至20中任一项所述的装置，其特征在于，所述装置还包括时延模拟单元和存储单元，
    所述获取单元，还用于获取自动驾驶车辆周围的第三环境信息以及所述自动驾驶车辆的第三状态信息；
    所述确定单元，还用于根据所述第三环境信息和所述第三状态信息，提取时延模型数据，所述时延模型数据包括应用的线程数、线程的周期性数据、线程运行的时间概率分布和线程数据的依赖关系中的一种或者多种；
    所述时延模拟单元，用于将所述时延模型数据输入时延模拟器，得到第二时延；
    所述确定单元，还用于根据所述第二时延，确定所述第三环境信息和所述第三状态信息对应的第四时延优化参数；
    所述存储单元，用于将所述第三环境信息、所述第三状态信息和所述第四时延优化参数的对应关系，存储在所述映射关系中。
  23. 一种数据处理装置，其特征在于，包括：
    存储器，用于存储计算机程序；
    处理器，用于执行所述存储器中存储的计算机程序，以使得所述装置执行如权利要求1至6中任一项所述的方法。
  24. 一种数据处理装置，其特征在于，包括：
    存储器，用于存储计算机程序；
    处理器，用于执行所述存储器中存储的计算机程序，以使得所述装置执行如权利要求7至11中任一项所述的方法。
  25. 一种车辆，其特征在于，包括如权利要求12至17中任一项所述的装置，或者，包括如权利要求23所述的装置。
  26. 一种服务器，其特征在于，包括如权利要求18至22中任一项所述的装置，或者，包括如权利要求24所述的装置。
  27. 一种计算机可读存储介质，其特征在于，其上存储有计算机程序，所述计算机程序被计算机执行时，以使得实现如权利要求1至6中任一项所述的方法，或者，实现如权利要求7至11中任一项所述的方法。
PCT/CN2024/097457 2023-06-29 2024-06-05 数据处理方法、装置和车辆 Ceased WO2025001786A1 (zh)

Priority Applications (1)

Application Number Priority Date Filing Date Title
EP24830433.9A EP4737253A1 (en) 2023-06-29 2024-06-05 Data processing method and apparatus, and vehicle

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202310792509.3A CN119261945A (zh) 2023-06-29 2023-06-29 数据处理方法、装置和车辆
CN202310792509.3 2023-06-29

Publications (1)

Publication Number Publication Date
WO2025001786A1 true WO2025001786A1 (zh) 2025-01-02

Family

ID=93937342

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2024/097457 Ceased WO2025001786A1 (zh) 2023-06-29 2024-06-05 数据处理方法、装置和车辆

Country Status (3)

Country Link
EP (1) EP4737253A1 (zh)
CN (1) CN119261945A (zh)
WO (1) WO2025001786A1 (zh)

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114077622A (zh) * 2020-08-10 2022-02-22 Wejo有限公司 处理车辆事件数据以用于行程分析的系统和方法
CN114157660A (zh) * 2020-09-04 2022-03-08 中移(上海)信息通信科技有限公司 数据传输的方法、装置、设备及存储介质
CN115086375A (zh) * 2022-06-14 2022-09-20 中国科学院微电子研究所 网联车辆运动状态信息延时补偿方法、装置、系统和介质
CN115767634A (zh) * 2022-09-23 2023-03-07 吉林大学 一种基于联邦学习的车联网时延敏感型应用卸载方法
US20230126317A1 (en) * 2021-10-25 2023-04-27 Wejo Limited System and method for processing vehicle event data for improved journey trace determination

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114077622A (zh) * 2020-08-10 2022-02-22 Wejo有限公司 处理车辆事件数据以用于行程分析的系统和方法
CN114157660A (zh) * 2020-09-04 2022-03-08 中移(上海)信息通信科技有限公司 数据传输的方法、装置、设备及存储介质
US20230126317A1 (en) * 2021-10-25 2023-04-27 Wejo Limited System and method for processing vehicle event data for improved journey trace determination
CN115086375A (zh) * 2022-06-14 2022-09-20 中国科学院微电子研究所 网联车辆运动状态信息延时补偿方法、装置、系统和介质
CN115767634A (zh) * 2022-09-23 2023-03-07 吉林大学 一种基于联邦学习的车联网时延敏感型应用卸载方法

Also Published As

Publication number Publication date
EP4737253A1 (en) 2026-05-06
CN119261945A (zh) 2025-01-07

Similar Documents

Publication Publication Date Title
JP7059362B2 (ja) 地図データ構築方法、車両端末、およびサーバ
US12097873B2 (en) Dynamic sensor data augmentation via deep learning loop
CN110196593B (zh) 一种自动驾驶多场景环境检测及决策系统及方法
CN113962141A (zh) 一种目标检测模型自动化迭代方法、设备及存储介质
CN114818230A (zh) 一种仿真测试方法、装置及系统
US20230202507A1 (en) Control system for autonomous vehicle simulator
US20240232460A1 (en) Machine learning for autonomous vehicles using parameterizable features
US11908095B2 (en) 2-D image reconstruction in a 3-D simulation
US12061537B2 (en) Testing software changes and determining a repeatability of software tests
US20230347925A1 (en) Agent and scenario modeling extracted via an mbse classification on a large number of real-world data samples
US11809790B2 (en) Architecture for distributed system simulation timing alignment
CN115421399B (zh) 一种面向自动驾驶场景的数字孪生系统和构建方法
CN115792867A (zh) 激光雷达仿真方法和装置
CN119512720A (zh) 一种资源调度方法、模型生成方法以及装置
CN118211145A (zh) 基于蒙特卡洛树的车辆动作决策方法、装置、电子设备、介质
US20240060788A1 (en) Map simulation services
US20240169111A1 (en) Parameterizing hot starts to maximize test repeatability
US20230185992A1 (en) Managing states of a simulated environment
US20220318450A1 (en) Lidar Atmospheric Effects in Simulation
WO2025001786A1 (zh) 数据处理方法、装置和车辆
CN118657044B (zh) 对自动驾驶模型进行训练的方法、装置和电子设备
WO2022067295A1 (en) Architecture for distributed system simulation timing alignment
US12222930B2 (en) Selective logging and wheel of time for vehicle network logger system
US20250046125A1 (en) Direct sensor logging for vehicle network logger system
EP4369042A1 (en) Systems and techniques for processing lidar data

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 24830433

Country of ref document: EP

Kind code of ref document: A1

WWE Wipo information: entry into national phase

Ref document number: 2024830433

Country of ref document: EP

NENP Non-entry into the national phase

Ref country code: DE

ENP Entry into the national phase

Ref document number: 2024830433

Country of ref document: EP

Effective date: 20260129

ENP Entry into the national phase

Ref document number: 2024830433

Country of ref document: EP

Effective date: 20260129