CN115438008A

CN115438008A - Data processing method, device and storage medium

Info

Publication number: CN115438008A
Application number: CN202110613845.8A
Authority: CN
Inventors: 孙叠; 朱杰; 杨晓宇
Original assignee: Huawei Technologies Co Ltd
Current assignee: Huawei Technologies Co Ltd
Priority date: 2021-06-02
Filing date: 2021-06-02
Publication date: 2022-12-06

Abstract

The application relates to a data processing method, a device and a storage medium, wherein the method comprises the following steps: the method comprises the steps of obtaining first data collected by a sensor, determining a first frame rate according to a frame rate interval and an index value, wherein the index value is used for indicating the state of a target vehicle, the first frame rate is within the frame rate interval, determining second data according to the first frame rate and the first data, and information of the second data is used for indicating a server to execute a first operation. According to the embodiment of the application, the frame rate of the uploaded data can be dynamically changed according to different states indicated by the index value of the target vehicle, so that redundant data can be eliminated, and the data uploading speed is increased.

Description

Data processing method, device and storage medium

Technical Field

The present application relates to the field of data processing technologies, and in particular, to a data processing method, an apparatus, and a storage medium.

Background

In the field of autonomous driving, where the vehicle needs to collect data by its own sensors, the sensors in an autonomous vehicle are expected to produce data volumes of 80GB per hour, but with high value data rates as low as 10%, where there is a large amount of redundant data that is less effective in analyzing the driving conditions of the vehicle, but increases the cost of storage and playback.

At present, when data processing is performed on sensor data, the related art usually collects the full amount of data in real time or only when an algorithm fails, so that a large amount of redundant data exists during data processing, and the data processing efficiency is low.

Disclosure of Invention

In view of the above, a data processing method, apparatus and storage medium are provided.

In a first aspect, an embodiment of the present application provides a data processing method, including: acquiring first data acquired by a sensor; determining a first frame rate according to a frame rate interval and an index value, wherein the index value is used for indicating the state of a target vehicle, and the first frame rate is within the frame rate interval; and determining second data according to the first frame rate and the first data, wherein the information of the second data is used for instructing a server to execute a first operation.

According to the embodiment of the application, aiming at first data collected by a sensor, a first frame rate of uploaded data can be dynamically determined according to different states indicated by index values of target vehicles, and second data used for indicating a server to execute first operation is determined according to the first frame rate and the first data, so that redundant data in the first data can be eliminated, the value of the uploaded second data is improved, the data uploading speed is increased, and the operation requirements can be met while the data redundancy is reduced by enabling the first frame rate to be within a frame rate interval.

According to the first aspect, in a possible implementation manner, determining the first frame rate according to the frame rate interval and the index value includes: when all index values in the index values exceed a threshold value corresponding to each index value, the first frame rate is an upper limit value of the frame rate section; or when part of the index values exceed the threshold value corresponding to each index value, the first frame rate is larger than the lower limit value of the frame rate interval and smaller than the upper limit value of the frame rate interval; or when all index values do not exceed the threshold value corresponding to each index value, the first frame rate is the lower limit value of the frame rate section.

According to the embodiment of the application, the index value is used for indicating the state of the target vehicle, the more the number of the index values exceeding the threshold value is, the more complex the driving scene is, the larger the first frame rate is required to complete the first operation, and the less the number of the index values exceeding the threshold value is, the simpler or more conventional the driving scene is, the smaller the data amount is required to complete the first operation, the smaller the first frame rate is required.

The first frame rate is respectively an upper limit value and a lower limit value of a frame rate interval when the first frame rate is related to the number of index values exceeding a threshold value, and the index values all exceed the threshold value and do not exceed the threshold value, and the first frame rate is located between the upper limit value and the lower limit value when part of the index values exceed the threshold value, so that the first frame rate is ensured to be within the frame rate interval, and the first frame rate can be changed according to the number of the index values exceeding the threshold value, so that the first frame rate is suitable for the change of the state of a target vehicle, and the operation requirement is met while the data redundancy is reduced.

According to the first aspect, in one possible implementation, the method further includes: acquiring a first message; and processing the second data according to the first message to obtain third data, wherein the size of the third data is smaller than or equal to that of the second data, and the information of the third data is used for indicating the server to perform the first operation.

According to the embodiment of the application, the vehicle can further process the data according to the indication of the first message by acquiring the first message, so that the data volume uploaded to the server is further reduced, and the fluency and timeliness of the first operation are guaranteed.

According to the first aspect, in a possible implementation manner, the first message indicates that a time difference between a second time and a first time exceeds a predetermined threshold, where the first time is a time when the server receives the second data or the third data, and the second time is a time when the server starts to perform the first operation according to the received second data or the third data.

According to the embodiment of the application, the server can keep the synchronism of receiving the data and executing the first operation, and the timeliness of the data is improved.

According to the first aspect, in one possible implementation, the first operation includes one or more of: data playback, data annotation, scene mining and data simulation.

According to the embodiment of the application, the type of the first operation executed on the uploaded data can be flexibly selected according to needs by including multiple types of first operations.

In a second aspect, an embodiment of the present application provides a data processing method, including: acquiring second data sent by a target vehicle, wherein the second data is determined by the target vehicle according to a first frame rate and first data acquired by a sensor, the first frame rate is determined according to a frame rate interval and an index value, the index value is used for indicating the state of the target vehicle, and the first frame rate is within the frame rate interval; and performing a first operation according to the indication of the information of the second data.

According to the embodiment of the application, the frame rate of the second data acquired by the server can be adjusted according to the state of the target vehicle by determining the second data acquired by the target vehicle according to the first frame rate and the first data acquired by the sensor, so that the value of the data acquired by the server is improved, the first frame rate is within the frame rate interval, the data redundancy is reduced, the operation requirement of the server when the server executes the first operation is met, and the speed of the server for acquiring the data can be adapted to the speed of the server when the server executes the operation.

According to the second aspect, in a possible implementation manner, the determining the first frame rate according to the frame rate interval and the index value includes: when all index values in the index values exceed a threshold value corresponding to each index value, the first frame rate is an upper limit value of the frame rate section; or when part of the index values exceed the threshold value corresponding to each index value, the first frame rate is larger than the lower limit value of the frame rate interval and smaller than the upper limit value of the frame rate interval; or when all index values do not exceed the threshold value corresponding to each index value, the first frame rate is the lower limit value of the frame rate section.

According to the embodiment of the application, the index value is used for indicating the state of the target vehicle, the more the number of the index values exceeding the threshold value is, the more complex the driving scene is, the larger the first frame rate is required to complete the first operation, and the less the number of the index values exceeding the threshold value is, the simpler or more conventional the driving scene is, the smaller the data amount is required to complete the first operation, the smaller the first frame rate is required. The first frame rate is respectively an upper limit value and a lower limit value of a frame rate interval when the first frame rate is related to the number of index values exceeding a threshold value, and the index values all exceed the threshold value and do not exceed the threshold value, and the first frame rate is located between the upper limit value and the lower limit value when part of the index values exceed the threshold value, so that the first frame rate is ensured to be within the frame rate interval, and the first frame rate can be changed according to the number of the index values exceeding the threshold value, so that the first frame rate is suitable for the change of the state of a target vehicle, and the operation requirement is met while the data redundancy is reduced.

According to the second aspect, in a possible implementation manner, performing a first operation according to the indication of the information of the second data includes: obtaining third data according to the indication of the information of the second data, wherein the size of the third data is smaller than or equal to that of the second data; and executing a first operation according to the indication of the information of the third data.

According to the embodiment of the application, the server can further process the data, so that the data quantity acquired by the server is further reduced, and the fluency and the timeliness of the server in the first operation are guaranteed.

According to the second aspect, in one possible implementation, the method further includes: and generating a first message, wherein the first message indicates that the time difference between a second time and a first time exceeds a preset threshold, the first time is the time when the second data is received, and the second time is the time when the first operation is started according to the second data.

According to the embodiment of the application, the first message is generated, when the time difference between the time when the server starts to execute the first operation and the time when the server receives the data is too large, the first message instructs the vehicle to process the second data so as to reduce the data volume of the second data, so that the server keeps the synchronization of receiving the data and executing the first operation, and the fluency and the timeliness of the server in executing the first operation are improved.

According to the second aspect, in one possible implementation, the method further includes: and sending the first message to the target vehicle, wherein the first message is used for instructing the target vehicle to process the first data.

According to the embodiment of the application, the server can inform the target vehicle of processing the data by sending the first message, so that the fluency and timeliness of the first operation performed by the server are improved.

According to the second aspect, in one possible implementation, the first operation includes one or more of: data playback, data annotation, scene mining and data simulation.

According to the embodiment of the application, the server can flexibly select the type of the first operation executed on the second data according to the requirement by including multiple types of first operations.

In a third aspect, an embodiment of the present application provides a data processing apparatus, including: the first acquisition module is used for acquiring first data acquired by the sensor; the system comprises a first determination module, a second determination module and a third determination module, wherein the first determination module is used for determining a first frame rate according to a frame rate interval and an index value, the index value is used for indicating the state of a target vehicle, and the first frame rate is within the frame rate interval; and the second determining module is used for determining second data according to the first frame rate and the first data, and the information of the second data is used for indicating the server to execute the first operation.

According to the third aspect, in a possible implementation manner, determining the first frame rate according to the frame rate interval and the index value includes: when all index values in the index values exceed a threshold value corresponding to each index value, the first frame rate is an upper limit value of the frame rate section; or when part of the index values exceed the threshold value corresponding to each index value, the first frame rate is larger than the lower limit value of the frame rate interval and smaller than the upper limit value of the frame rate interval; or when all index values do not exceed the threshold value corresponding to each index value, the first frame rate is the lower limit value of the frame rate section.

According to the third aspect, in a possible implementation manner, the apparatus further includes: the third acquisition module is used for acquiring the first message; and the processing module is used for processing the second data according to the first message to obtain third data, wherein the size of the third data is smaller than or equal to that of the second data, and the information of the third data is used for indicating the server to perform the first operation.

According to the third aspect, in a possible implementation manner, the first message indicates that a time difference between a second time and a first time exceeds a predetermined threshold, where the first time is a time when the server receives the second data or the third data, and the second time is a time when the server starts to perform the first operation according to the received second data or the third data.

According to the third aspect, in one possible implementation, the first operation includes one or more of: data playback, data annotation, scene mining and data simulation.

In a fourth aspect, an embodiment of the present application provides a data processing apparatus, including: the system comprises a second acquisition module, a first processing module and a second processing module, wherein the second acquisition module is used for acquiring second data sent by a target vehicle, the second data is determined by the target vehicle according to a first frame rate and first data acquired by a sensor, the first frame rate is determined according to a frame rate interval and an index value, the index value is used for indicating the state of the target vehicle, and the first frame rate is within the frame rate interval; and the operation module is used for performing first operation according to the indication of the information of the second data.

According to the fourth aspect, in a possible implementation manner, the determining the first frame rate according to the frame rate interval and the index value includes: when all index values in the index values exceed a threshold value corresponding to each index value, the first frame rate is an upper limit value of the frame rate section; or when part of the index values exceed the threshold value corresponding to each index value, the first frame rate is larger than the lower limit value of the frame rate interval and smaller than the upper limit value of the frame rate interval; or when all index values do not exceed the threshold value corresponding to each index value, the first frame rate is the lower limit value of the frame rate section.

According to the fourth aspect, in a possible implementation manner, performing a first operation according to the indication of the information of the second data includes: obtaining third data according to the indication of the information of the second data, wherein the size of the third data is smaller than or equal to that of the second data; and executing a first operation according to the indication of the information of the third data.

According to a fourth aspect, in a possible implementation manner, the apparatus further includes: and the generating module is used for generating a first message, wherein the first message indicates that the time difference between a second moment and a first moment exceeds a preset threshold, the first moment is the moment when the second data is received, and the second moment is the moment when the first operation is started to be executed according to the second data.

According to a fourth aspect, in a possible implementation manner, the apparatus further includes: and the sending module is used for sending the first message to the target vehicle, and the first message is used for indicating the target vehicle to process the first data.

According to the fourth aspect, in one possible implementation, the first operation includes one or more of: data playback, data annotation, scene mining and data simulation.

In a fifth aspect, an embodiment of the present application provides a data processing apparatus, including: a processor and a memory; the memory is used for storing programs; the processor is configured to execute the program stored in the memory, so as to enable the apparatus to implement the data processing method of the first aspect or one or more of the multiple possible implementation manners of the first aspect, or to implement the data processing method of the second aspect or one or more of the multiple possible implementation manners of the second aspect.

In a sixth aspect, an embodiment of the present application provides a computer-readable storage medium, including: computer instructions that, when executed, implement a data processing method according to one or more of the first aspect or the plurality of possible implementations of the first aspect, or implement a data processing method according to one or more of the second aspect or the plurality of possible implementations of the second aspect.

In a seventh aspect, an embodiment of the present application provides a terminal device, where the terminal device may execute the data processing method of the first aspect or one or more of the multiple possible implementation manners of the first aspect, or execute the data processing method of the second aspect or one or more of the multiple possible implementation manners of the second aspect.

In an eighth aspect, an embodiment of the present application provides a computer program product, which includes instructions that, when executed on a computer, cause the computer to perform a data processing method of the first aspect or one or more of the multiple possible implementations of the first aspect, or perform a data processing method of the second aspect or one or more of the multiple possible implementations of the second aspect.

In a ninth aspect, an embodiment of the present application provides an in-vehicle computing device, which includes a processor, and the processor is configured to execute the data processing method of the first aspect or one or more of the multiple possible implementation manners of the first aspect.

In a tenth aspect, an embodiment of the present application provides a server, which includes a processor, and the processor is configured to execute the data processing method of the second aspect or one or more of the multiple possible implementations of the second aspect.

In an eleventh aspect, an embodiment of the present application provides a chip system, where the chip system includes at least one processor, and is configured to support a data processing method for implementing one or more of the first aspect or the multiple possible implementations of the first aspect, or a data processing method for implementing one or more of the second aspect or the multiple possible implementations of the second aspect, for example, to receive or process data and/or information involved in the foregoing methods.

According to the eleventh aspect, in a possible implementation manner, the system-on-chip further includes a memory for storing program instructions and data, the memory is located inside the processor or outside the processor, and the system-on-chip may be formed by a chip, and may also include a chip and other discrete devices.

In a twelfth aspect, an embodiment of the present application provides a data processing apparatus, where the data processing apparatus includes at least one processor and a communication interface, where the communication interface is configured to send and/or receive data, and the at least one processor is configured to invoke a computer program stored in at least one memory, so as to enable the data processing apparatus to implement the data processing method of the first aspect or one or more of the multiple possible implementation manners of the first aspect, or implement the data processing method of the second aspect or one or more of the multiple possible implementation manners of the second aspect.

Drawings

FIG. 1 shows an architectural diagram of a data processing system according to an embodiment of the present application.

Fig. 2 shows a flow chart of a data processing method according to an embodiment of the application.

Fig. 3 shows a flow chart of a data processing method according to an embodiment of the application.

Fig. 4 shows a flow chart of an acquisition phase of a data processing method according to an embodiment of the application.

Fig. 5 shows a schematic diagram of an acquisition phase according to an embodiment of the application.

Fig. 6 shows a flow chart of a data processing method according to an embodiment of the present application.

Fig. 7 shows a flow chart of an execution operation phase of a data processing method according to an embodiment of the present application.

Fig. 8 shows a flow chart of a data processing method according to an embodiment of the present application.

Fig. 9 shows a schematic diagram of a data playback operation according to an embodiment of the present application.

Fig. 10 shows a block diagram of a data processing apparatus according to an embodiment of the present application.

Fig. 11 shows a block diagram of a data processing apparatus according to an embodiment of the present application.

Fig. 12 shows a block diagram of a data processing apparatus according to an embodiment of the present application.

Detailed Description

Various exemplary embodiments, features and aspects of the present application will be described in detail below with reference to the accompanying drawings. In the drawings, like reference numbers can indicate functionally identical or similar elements. While the various aspects of the embodiments are presented in drawings, the drawings are not necessarily drawn to scale unless specifically indicated.

The word "exemplary" is used exclusively herein to mean "serving as an example, embodiment, or illustration. Any embodiment described herein as "exemplary" is not necessarily to be construed as preferred or advantageous over other embodiments.

Furthermore, in the following detailed description, numerous specific details are set forth in order to provide a better understanding of the present application. It will be understood by those skilled in the art that the present application may be practiced without some of these specific details. In some instances, methods, means, elements and circuits that are well known to those skilled in the art have not been described in detail so as not to obscure the present application.

In order to solve the technical problem, the present application provides a data processing method, which can implement efficient and high-quality uploading of data to a server by a vehicle and real-time execution of a first operation on the received data by the server, and can be applied to a data processing system.

Fig. 1 shows an architectural diagram of a data processing system according to an embodiment of the present application. For the convenience of understanding the embodiments of the present application, first, a data processing system applicable to the embodiments of the present application will be described in detail by taking the data processing system shown in fig. 1 as an example. It should be noted that the solutions in the embodiments of the present application may also be applied to other data processing systems, and the corresponding names may also be replaced with names of corresponding functions in other data processing systems. Communication systems, such as vehicle to any object (V2X) communication systems, device-to-device (D2D) communication systems, and internet of vehicles communication systems, may be included in the data processing system. The data processing system may be a data processing system based on one or more of the following communication systems: the 4th generation (4 g) mobile communication system, such as a Long Term Evolution (LTE) system, a fifth generation (5 g) mobile communication system, such as a New Radio (NR) system, and a future mobile communication system, such as a sixth generation (6 g) mobile communication system, etc.

As shown in fig. 1, the data processing system may include a vehicle and a server. The server may be a device located in the cloud or local of the data processing system and having a data processing capability, and may be a physical device such as a host, a server, or the like, or may be a virtual device such as a virtual machine, a container, or the like. It should be noted that, for convenience of description, the server is referred to herein, and in the implementation process, the server may be the server, or may be another device with data processing capability, or a module (e.g., a chip or an integrated circuit) in the device. The server may be a server for collecting data such as geographic position information, vehicle track, vehicle surrounding information, and the like collected by the vehicle, and has a wired or wireless transceiving function, wherein the wired or wireless transceiving function may be provided in a chip (system) or other components or assemblies of the server.

The vehicle can access the data processing system and has a wired or wireless transceiving function, wherein the wired or wireless transceiving function can be arranged on a vehicle-mounted terminal, a vehicle-mounted module, a vehicle-mounted unit, a chip (system) or other components or assemblies of the vehicle. The vehicle can also be referred to as a user equipment, an access terminal, a subscriber unit, a subscriber station, a mobile station, a remote terminal, a mobile device, a user terminal, a wireless communication device, a user agent. The vehicle in the embodiment of the present application may be a wireless terminal in self-driving (self-driving), a wireless terminal in transportation safety (transportation safety), a wireless terminal in smart city (smart city), a vehicle-mounted terminal, an RSU having a terminal function, or the like. The embodiments of the present application do not limit this.

As shown in fig. 1, the vehicle may be provided with at least one sensor, such as a vehicle-mounted radar (e.g., a millimeter wave radar, an infrared radar, a laser radar, a doppler radar, etc.), a light quantity sensor, a rainfall sensor, an audio/video sensor (e.g., a camera, a driving recorder), a vehicle posture sensor (e.g., a gyroscope), a speed sensor (e.g., a doppler radar), an Inertial Measurement Unit (IMU), etc., and the sensor may be disposed on one vehicle or on a plurality of vehicles, and may collect data such as image data, point cloud data, perception data, prediction data, planning data, control data, positioning data, chassis data, etc., stored according to frames through the at least one sensor disposed on the vehicle.

In the embodiment of the present application, the server and the vehicle may communicate in a wireless connection manner, for example, may communicate in a wireless connection manner such as wireless fidelity (wifi) and bluetooth, or communicate in a wired connection manner, for example, may communicate in a wired connection manner such as an optical fiber and a cable, for example, during a test, the server may be carried on the vehicle and communicate with the vehicle in a wired connection manner, which is not limited in this respect, as long as it can implement interaction of data and information.

In a possible implementation manner, the data processing method according to the embodiment of the present application may be used in a scenario where a vehicle performs automatic driving, where the vehicle may send all data or a part of the data, which is collected by a sensor and indicates a state of a target vehicle, to a server according to a dynamically determined first frame rate, and the server may perform at least one operation of data playback, data tagging, scene mining, data simulation, and the like by using the data, where the vehicle and/or the server may further process the data to improve efficiency when performing the operations of the data playback, the data tagging, the scene mining, the data simulation, and the like, and the implementation of the process may refer to the following method embodiment, and details are not repeated here.

It should be understood that fig. 1 is a simplified schematic diagram that is merely used for convenience of understanding, and that other servers and/or other vehicles may also be included in the data processing system, none of which is shown in fig. 1, and the data processing method according to the embodiment of the present application may also be used in other scenarios, which are not limited in this application.

Fig. 2 shows a flowchart of a data processing method according to an embodiment of the present application. The method may be performed by the vehicle, as shown in fig. 2, and may include the following steps S201 to S203:

step S201, acquiring first data acquired by a sensor;

one or more sensors may be provided, and the number and the installation manner of the sensors are not limited in the embodiments of the present application. For example, the sensor may be disposed only on a target vehicle among the plurality of vehicles, may be disposed on another vehicle than the target vehicle among the plurality of vehicles, or may be disposed on another facility than the vehicle, such as a fixed facility on both sides of a road, and the first data collected by the sensor may be data for the target vehicle, such as a speed, an acceleration, and the like of the target vehicle. For example, with a vehicle a as a target vehicle, first data of the vehicle a may be collected by a sensor on the vehicle a; the first data of the vehicle A can be collected through a sensor on the vehicle B and then sent to the vehicle A or sent to a server; or the first data of the vehicle A is collected through a sensor on the fixed facility and then is sent to the vehicle A; alternatively, the first data of the vehicle a may be collected by at least two of a sensor on the vehicle a, a sensor on the vehicle B, and a sensor on a fixed facility, and summarized to the vehicle a, which is not limited in this application. The vehicle executing the method of the embodiment of the present application may be the target vehicle, or may be another vehicle other than the target vehicle.

For example, the first data collected by the sensor may be one or more of the above image data, point cloud data, sensing data, prediction data, planning data, control data, positioning data, chassis data, and the like, which may be collected by one or more sensors, respectively, for example, the image data may be collected by an audio/video sensor (e.g., a camera), the point cloud data may be collected by a laser radar, the sensing data may be collected by one or more sensors such as a laser radar, a camera, a millimeter wave radar, and the like, and the frame rate at which the sensor collects the data may be determined according to a rated collection frame rate of the sensor, which may be determined when the sensor leaves a factory.

Step S202, determining a first frame rate according to a frame rate interval and an index value, wherein the index value is used for indicating the state of a target vehicle, and the first frame rate is within the frame rate interval;

the index value may represent a value of an index, and the index includes various types of indexes that may be used to represent a state of the target vehicle. The first frame rate may be a frequency corresponding to data uploaded when the vehicle currently uploads the data to the server. The frame rate interval may represent a range of a set first frame rate, and according to the determined frame rate interval, a maximum value of the first frame rate of the vehicle upload data does not exceed an upper limit value of the frame rate interval, and a minimum value of the first frame rate is not lower than a lower limit value of the frame rate interval.

Step S203, determining second data according to the first frame rate and the first data, where information of the second data is used to instruct a server to execute a first operation.

The information of the second data may refer to data content of the second data or information carried in the second data, and the first operation may be performed under the instruction of the information of the second data, for example, when the second data is image data, the information of the second data may include a pixel value, image acquisition time, and the like in the image data, and the first operation such as playback of an image and the like may be performed under the instruction of the information of the second data, and when the second data is positioning data, the information of the second data may include coordinate values and the like in the positioning data, and the first operation such as position labeling may be performed under the instruction of the information of the second data. The specific content of the information of the second data is not limited in the present application.

It can be seen that, since the first frame rates corresponding to the uploaded second data at different times may be different, the data amount of the second data uploaded by the vehicle at different times may be different, for example, in a complex driving scenario (e.g., a scenario where the target vehicle is in a sharp turn or at an intersection), the data amount of the uploaded second data is larger, and in a normal driving scenario (e.g., a scenario where the target vehicle is in a constant speed or a straight road), the data amount of the uploaded second data is smaller.

According to the embodiment of the application, aiming at the first data collected by the sensor, the first frame rate of the uploaded data can be dynamically determined according to different states indicated by the index value of the target vehicle, and the second data used for indicating the server to execute the first operation is determined according to the first frame rate and the first data, so that redundant data in the first data can be eliminated, the value of the uploaded second data is improved, the data uploading speed is increased, and the first frame rate is within the frame rate interval, so that the data redundancy is reduced, and meanwhile, the operation requirement is met.

Fig. 3 shows a flowchart of a data processing method according to an embodiment of the present application. The method may be executed by the server, and as shown in fig. 3, may include the following steps S301 to S302:

step S301, acquiring second data sent by a target vehicle, wherein the second data is determined by the target vehicle according to a first frame rate and first data acquired by a sensor, the first frame rate is determined according to a frame rate interval and an index value, the index value is used for indicating the state of the target vehicle, and the first frame rate is within the frame rate interval;

step S302, a first operation is carried out according to the indication of the information of the second data.

According to the embodiment of the application, the frame rate of the second data acquired by the server can be adjusted according to the state of the target vehicle by determining the acquired second data for the target vehicle according to the first frame rate and the first data acquired by the sensor, so that the value of the data acquired by the server is improved, the first frame rate is within the frame rate interval, the data redundancy is reduced, meanwhile, the operation requirement of the server when executing the first operation is met, and the speed of the server for acquiring the data can be adapted to the speed of the server when executing the operation.

In one possible implementation, the first operation includes one or more of the following: data playback, data annotation, scene mining and data simulation.

In a possible implementation manner, in step S202, determining the first frame rate according to the frame rate interval and the index value may include: when all index values in the index values exceed a threshold value corresponding to each index value, the first frame rate is an upper limit value of the frame rate section; or when part of the index values exceed the threshold value corresponding to each index value, the first frame rate is larger than the lower limit value of the frame rate interval and smaller than the upper limit value of the frame rate interval; or when all index values in the index values do not exceed the threshold value corresponding to each index value, the first frame rate is the lower limit value of the frame rate section.

Here, the method shown in the following equation (1) to equation (5) may be referred to when determining the first frame rate. Those skilled in the art will appreciate that the manner of determining the first frame rate is not limited to the examples provided below, as long as it is satisfied that the first frame rate varies with the state of the target vehicle so that the data amount of the second data determined according to the first frame rate can satisfy the requirement for the first operation in the state of the target vehicle, and for example, the first frame rate may also be increased with an increase in the number of indices for which the index value exceeds the threshold value by other means.

The following further describes a flow of the data processing method provided by the embodiment of the present application by taking a vehicle and a server as examples, with reference to fig. 4 to 9.

Fig. 4 shows a flow chart of an acquisition phase of a data processing method according to an embodiment of the application. Wherein the exemplary description of step S401 may refer to step S201 above, and steps S402-S404 may be taken as examples of step S202 above. Exemplary description of step S406 may be found above for step S203.

As shown in fig. 4, the flow of the acquisition phase may include:

step S401, the vehicle acquires first data collected by a sensor.

In step S402, the vehicle determines an index value and a frame rate section.

Wherein, by the relationship between the index value and the threshold value, it is possible to determine whether or not the state of the target vehicle is in the state indicated by the index. The state of the target vehicle may include a state of the target vehicle itself, a state of an environment in which the target vehicle is located, a state of a road in which the target vehicle is located, and the like. The indexes corresponding to the self state of the target vehicle can comprise rapid acceleration/rapid deceleration, high acceleration change rate, sharp turning, turning around, backing up, merging into a lane, leaving from the lane and the like; indexes corresponding to the environmental state of the target vehicle can include weather difference (rainy days, snowy days, sand storms and haze), weak illumination intensity (night and tunnels), slippery road surfaces (ice and water), traffic jam, more pedestrians and the like; the indexes corresponding to the road state where the target vehicle is located may include complex intersections (crossroads, Y-intersections), uphill slopes, downhill slopes, curves, zebra crossings, country roads, mountain roads, closed parks, abnormal scenes, and the like. In embodiments of the present application, the vehicle-determined indicator may include one or more of the above. By combining various indexes, any complex scene can be customized. For example, if the index values of the three indexes of sharp turning, wet road surface and country road satisfy a predetermined relationship (e.g., are greater than a threshold value) with the corresponding threshold values, the target vehicle may be instructed to make a sharp turn on the wet road surface of the country road. Table 1 shows an example of the correspondence relationship of the index to the target vehicle state indicated by the index to the basis of determining the index value and the threshold value.

TABLE 1

It should be noted that only a part of the indexes are shown in table 1, and the number of the indexes determined by the vehicle may be larger or smaller, may include a part of the indexes shown in table 1, and may include other indexes besides the indexes shown in table 1. The embodiments of the present application do not limit this.

For example, as shown in table 1, "determining the index value and the basis of the threshold value" in table 1 may be a manner of determining that the target vehicle is in the target vehicle state indicated by the index, for example, corresponding to the index "rapid acceleration/rapid deceleration", the index value may be the acceleration of the target vehicle, a threshold value of the acceleration may be set, in the case where the current index value is greater than the acceleration threshold value, it is determined that the current target vehicle is in the "rapid acceleration/rapid deceleration" state, further, different threshold values of the acceleration corresponding to different speeds may be set, and in the case where the current index value is greater than the acceleration threshold value at the current speed, it is determined that the current target vehicle is in the "rapid acceleration/rapid deceleration" state; the index value may be an acceleration change rate of the target vehicle corresponding to the index "the acceleration change rate is large", the acceleration change rate may be determined by acceleration, a threshold value of the acceleration change rate may be set, and in a case where the current index value is greater than the acceleration change rate threshold value (or a corresponding acceleration change rate threshold value at the current speed), it is determined that the current target vehicle is in a state of "the acceleration change rate is large"; corresponding to the index "sharp turn", the index value may be a lateral speed of the target vehicle, a threshold value of the lateral speed may be set, and in the case where the current index value is greater than the threshold value of the lateral speed, it is determined that the current target vehicle is in a state of "sharp turn"; corresponding to the index "curve", the index value may be a curve wheel axis angle of the target vehicle, a threshold value of the curve wheel axis angle may be set, and in the case where the current index value is greater than the threshold value of the curve wheel axis angle, it is determined that the current target vehicle is in a state of "curve"; the index value may be the number of perception targets (pedestrians) of the target vehicle, and a number threshold value of the number of perception targets (pedestrians) may be set, and in the case where the current index value is larger than the number threshold value, it is determined that the current target vehicle is in the state of "pedestrian-rich".

The index value may be determined according to the first data collected by the sensor, for example, the index value (acceleration value) of "rapid acceleration/rapid deceleration" and "high acceleration change rate" may be determined according to the speed collected by the speed sensor, "the index value (lateral speed value) of" sharp turn "may be determined according to the lateral speed collected by the speed sensor," the index value (curve wheel axis angle) of "curve" may be determined according to the curve wheel axis angle collected by the vehicle attitude sensor; the "pedestrian-rich" index value (the number of pedestrians) may be determined from the image data acquired by the camera, and the number of pedestrians may be identified from the image data.

One or more of the index, the frame rate section, and the threshold may be preset in the vehicle; or may be preset in other devices and obtained by the vehicle from other devices; or the system can be preset in a server and then deployed to a vehicle through a network; and may be input into the vehicle by the user as desired, which is not limited in this application. Correspondingly, one or more of the index, the frame rate section and the threshold may also be modified, for example, in the case of being preset in the vehicle, the index, the frame rate section or the threshold in the vehicle may be reset; if the index, the frame rate section or the threshold value in the other equipment is preset in the other equipment, the modified index, the frame rate section or the threshold value can be acquired from the other equipment by the vehicle; under the condition of presetting in the server, the index, the frame rate interval or the threshold value in the server can be reset and redeployed to the vehicle through the network; in the case of direct input into the vehicle by the user, a new index, a frame rate interval, or a threshold value may be directly input by the user to be modified.

And S403, determining the weight corresponding to each index by the vehicle according to the dynamic acquisition strategy.

The dynamic acquisition policy may refer to a manner of determining the first frame rate.

In a possible implementation manner, the weight corresponding to each index may be determined first, and then the first frame rate may be determined according to the weight corresponding to each index. One exemplary way of determining the weight corresponding to each index value may refer to the following formulas (1) to (3).

Wherein, the formula (1) can be used to obtain the information content C of the corresponding index, wherein the larger the value of C, the larger the objective function of the corresponding index in all indexes is, r can represent the standardized matrix of the judgment matrix a, r _ij Can represent the elements of the ith row and the jth column in the standardized matrix r, n is the total number of indexes, and in the judgment matrix A, each row can correspond to one index, and each column can also correspond to one index, A _ij The importance degree of the index corresponding to the ith row relative to the index corresponding to the jth column can be represented, the judgment matrix A can be set when the indexes are set, and the method for obtaining the judgment matrix A, C, is not limited in the application _j The information amount, δ, of the index corresponding to the jth column in A _j The standard deviation in a corresponding to column j is shown.

Equation (2) can be used to obtain the objective weight W of the corresponding index _critic ，W _criticj Can be based on the information content C corresponding to the j column in A _j The ratio of the sum of the information amounts in all columns (1-n columns) is determined, and can represent the objective weight of the index corresponding to the j-th column in a.

Equation (3) may be used to obtain the weight W, W of the corresponding index _i The weight of the index corresponding to the ith column in A, and the weight W of indexes 1-n _i The sum of 1,W _ahp Can represent the weight vector of A, W _ahpi Can represent the weight value, W, of the corresponding ith index in the weight vector _critici The objective weight (obtained by equations (1) and (2)) of the index corresponding to the ith column in a can be expressed. Table 2 shows an example of the determination matrix a and the weights corresponding to the indices calculated by the formulas (1) to (3).

TABLE 2

	Index 1	Index 2	Index 3	Index 4	Index 5	Weight of
							Index 1	1	1/2	4	3	3	0.26
Index 2	2	1	7	5	5	0.48
							Index 3	1/4	1/7	1	1/2	1/3	0.06
Index 4	1/3	1/5	2	1	1	0.09
							Index 5	1/3	1/5	3	1	1	0.11

As shown in table 2, the indexes 1 to 5 may correspond to 5 different indexes (e.g., 5 indexes in table 1), respectively, and the matrix formed by the values in the 2 nd to 6th columns in table 2 may be the judgment matrix a corresponding to the indexes 1 to 5, for example, the value of (index 1, index 2) is 1/2, which may indicate the importance degree of the index 1 relative to the index 2, wherein the larger the value of (index 1, index 2) is, the more important the index 1 relative to the index 2 is, for example, the value of (index 2, index 3) in the index 1 to 5 exemplified in table 2 is 7, which may indicate that the index 2 is, the more important the index 3; for example, since the index 2 is "large in the acceleration change rate" and the index 3 is "sharp turn", the index 2 is relatively important to the index 3 because the correlation therebetween is relatively strong. The value of (index 4, index 2) is 1/5, which may indicate that index 4 is less important than index 2, for example, index 4 is "curve", index 2 is "large acceleration change rate", and the correlation between both is not strong, and thus index 4 is less important than index 2. The other values in columns 2-6 of table 2 represent the same meanings and so on. The last column in table 2 shows the weights corresponding to the indices calculated by the formulas (1) to (3), and from the determination matrix shown in table 2, the weight corresponding to the index 1 is 0.26, the weight corresponding to the index 2 is 0.48, the weight corresponding to the index 3 is 0.06, the weight corresponding to the index 4 is 0.09, and the weight corresponding to the index 5 is 0.11.

It should be noted that, when calculating the weight corresponding to each index, the weight corresponding to each index may be calculated by using an Analytic Hierarchy Process (AHP) according to the determination matrix a, or the weight corresponding to each index may be calculated by directly using an Analytic Network Process (ANP) without using the determination matrix a.

In step S404, the vehicle determines a first frame rate according to the weight and the frame rate interval corresponding to each index.

After the vehicle determines the weight corresponding to each index, it may be determined whether the index value of each index satisfies a condition that the target vehicle is in the target vehicle state indicated by the index, and the weights corresponding to the indexes determined to satisfy the condition in all the indexes are added and scored by a percentage, see formula (4):

wherein Score may represent a Score, p, indicative of a current target vehicle state _i It may be represented whether or not the index value of the i-th index satisfies a condition, p, that causes the target vehicle to be in the target vehicle state indicated by the index at that time _i When the value of (b) is 0, it can indicate that the i-th index does not satisfy the condition, p _i When the value of (b) is 1, it can indicate that the i-th index satisfies the condition. Satisfying the condition may include the index value exceeding a corresponding threshold, for example, in the case where the index is "curve", in the case where the current index value is greater than a set curve wheel axle angle threshold, it may be determined that the above condition is satisfied, when p _i The value of (b) is 1.

After obtaining the Score, a first frame rate may be obtained according to an averaging method according to the set frame rate interval, see formula (5):

FR＝Score*(f _max -f _min )/100+f _min formula (5)

Where FR may represent a first frame rate, f _max Can express frame rateUpper limit of interval, f _min The lower limit value of the frame rate interval may be indicated.

Under the condition that the determined index weight is not changed, since the index value can be determined according to the data collected by the sensor in real time, the index meeting the above conditions can be different at different time, and therefore, the Score can be changed along with the index value at different time, so that the first frame rate can be dynamically updated.

Wherein if all index values do not exceed the corresponding threshold, i.e. all p _i Is 0, the Score is 0, and the first frame rate FR is the lower limit f of the frame rate interval _min . If all index values exceed the corresponding threshold values, i.e. all p _i Is 1, the Score is 100, and the first frame rate FR is the upper limit value f of the frame rate interval _max . Score is a value between 0 and 100 if the partial index value exceeds the corresponding threshold value, the first frame rate FR is greater than a lower limit value and less than an upper limit value, wherein the size of the value of Score depends on p having a value of 1 _i The value of the first frame rate FR depends on the value of Score, p is 1 _i The larger the number of the index values, the more the index values which are over the threshold value are, that is, the more the target vehicle is indicated in the state indicated by the index, the larger the obtained first frame rate is, so that when the vehicle is in the state indicated by more indexes (for example, in the state indicated by a plurality of indexes such as rapid acceleration and more pedestrians), the more data is transmitted according to the higher first frame rate, so that the data amount meets the requirement of a complex scene, and when the vehicle is measured in the state indicated by less indexes, the less data is transmitted according to the smaller first frame rate, so that the transmitted data meets the requirement, and the redundancy is reduced.

In a possible implementation manner, after the vehicle determines the weight corresponding to each index, the Score may not be calculated, and the first frame rate may be obtained directly according to an averaging method, see formula (6):

in one possible implementation, the vehicle may also not need to haveAnd determining the weight corresponding to each index, and determining the first frame rate directly according to the index value and the frame rate interval. For example, p corresponding to each index may be determined from the index value _i According to each index, the corresponding p _i Is the ratio of the number of 1 to the total number of metrics, determine the first frame rate, see equation (7):

referring to the above analysis, similarly, the first frame rate determined based on equation (6) or (7) is also satisfied to be within the frame rate section, and increases as the number of indexes whose index values exceed the threshold value increases.

The above-mentioned manner of determining the first frame rate is only referred to as an example, and the present application is not limited to the manner of determining the first frame rate, as long as the first frame rate can be adjusted according to a change in the state of the target vehicle.

In step S405, the vehicle determines second data according to the first data and the first frame rate.

The vehicle may use data corresponding to the first frame rate in the first data as the second data, and the second data may be all or part of the first data.

For example, when the frame rate of the first data collected by the sensor is 30Hz (i.e., 30 frames/second) and the first frame rate is 15Hz (i.e., 15 frames/second), for the first data in every 1 second, 15 frames of data may be selected as the corresponding second data, wherein 1 frame of data may be selected as the corresponding second data every 1 frame of data, and the first 15 frames of data in the 30 frames of first data may also be selected as the corresponding second data, as long as the second data in every 1 second may be 15 frames less than the first data, which is not limited in the present application.

In step S406, the vehicle uploads the second data to the server.

In one possible implementation, the vehicle may upload the second data to a memory in the server in preparation for the server to perform the relevant operation.

Therefore, the data volume of the second data uploaded by the vehicle can be dynamically changed according to the change of the state of the target vehicle, so that redundant data can be eliminated, the second data obtained by the server are data with higher value, the current requirement can be met, the data redundancy is avoided, meanwhile, the cost of storing the data by the server can be reduced by dynamically adjusting the data volume of the second data, and the data uploading speed is accelerated.

Fig. 5 shows a schematic diagram of an acquisition phase according to an embodiment of the application. The target vehicle may determine the index as shown in table 1.

As shown in fig. 5, for the target vehicle "hush AXXXXX", the index values that the current vehicle detects over the set threshold are acceleration: 0.4g, corresponding to the index "sharp acceleration/sharp deceleration", the curve wheel axle angle: when the index "curve" is determined to have a weight of 0.26 for the index "rapid acceleration/rapid deceleration" and a weight of 0.48 for the index "curve" and the set frame rate interval is [0.1,5] at 15 °, the first frame rate at this time can be calculated to be 3.7Hz from the expressions (4) to (5). At this time, the vehicle may upload the second data corresponding to the first frame rate of 3.7Hz in the first data to the server, for example, may round off the 3.7Hz, and upload the second data corresponding to the first data to the server at a frame rate of 4 Hz.

In a possible implementation manner, the server or the vehicle may further perform bandwidth adjustment if a preset condition is met. The preset condition may be a condition set for the server to perform the first operation according to the second data with respect to a time delay of the server receiving the second data.

For example, the second data received by the server may be stored in a memory of the server, and since the second data may be obtained from the memory of the server when the server performs the first operation according to the second data, there may be a fluctuation in bandwidth of data transmission from the memory to the module for performing the first operation, and the fluctuation may be reflected in a time delay when the server performs the first operation according to the received second data, in order to make the receiving of the second data and the performing of the first operation according to the second data as synchronous as possible and reduce the time delay, the server may process the second data according to the received second data, and in a possible implementation manner, the server may further notify the vehicle, the vehicle may process the second data, and the second data may be processed by further reducing the data amount of the data, for example, to adapt to the current transmission bandwidth.

After the second data is processed, the execution speed of the server for executing the operation according to the processed data can be increased, so that the time delay of executing the first operation according to the data and receiving the data is shortened, the server can execute the operation according to the data in synchronization with the server for receiving the data as much as possible, and the timeliness of the data can be improved.

Examples of the above processing of the second data can be seen in the following description in conjunction with fig. 6 to 9.

Fig. 6 shows a flow chart of a data processing method according to an embodiment of the present application. Fig. 6 illustrates the above-described scenario in which the vehicle is notified by the server and the second data is processed by the vehicle.

As shown in fig. 6, the following steps S601-S602 may be included, wherein the steps S601-S602 may be performed by the vehicle:

step S601, acquiring a first message;

step S602, according to the first message, processing the second data to obtain third data, where the size of the third data is smaller than or equal to the size of the second data, and information of the third data is used to instruct the server to perform the first operation.

In a possible implementation manner, the first message indicates that a time difference between a second time and a first time exceeds a predetermined threshold, where the first time is a time when the server receives the second data or the third data, and the second time is a time when the server starts to execute the first operation according to the received second data or the third data.

The first message may be generated by the server, that is, the server may further perform the following steps: generating a first message and sending the first message to the target vehicle, wherein the first message is used for instructing the target vehicle to process the first data.

For example, when the second data is processed by the vehicle to obtain the third data, the server notifies the vehicle, for example, by generating a first message indicating that a time difference between the second time and the first time exceeds a threshold value, and transmitting the first message to the vehicle. After the vehicle receives the first message, the second data can be processed according to the instruction of the first message to obtain third data, and the third data is sent to the server. The data transmission between the vehicle and the server can be carried out in real time, i.e. before the third data is obtained, the vehicle can still send the unprocessed second data to the server until the third data is obtained by the vehicle.

Fig. 7 shows a flow chart of a data processing method according to an embodiment of the application. Fig. 7 illustrates the above-described scenario in which the server performs processing based on the received second data. As shown in fig. 7, when the server executes the step S302 and performs the first operation according to the instruction of the information of the second data, the server may include the following steps:

step S701, obtaining third data according to the indication of the information of the second data, wherein the size of the third data is smaller than or equal to that of the second data;

step S702, according to the indication of the information of the third data, a first operation is executed.

According to the embodiment of the application, the server can further process the data, so that the data volume of the second data is further reduced, and the fluency and the timeliness of the server in the first operation are guaranteed.

For example, when the server processes the second data to obtain third data, the server may process the second data received by the interface for receiving external data in the server to obtain third data, and send the third data to the module for executing the first operation; when the server stores the received second data in the memory of the server, the server may also process the second data in the memory to obtain third data, and send the third data to the module for executing the first operation. Before the module for performing the first operation in the server obtains the third data, the module for performing the first operation obtains the second data (which may be obtained from an interface for receiving external data in the server or from a memory) that is still unprocessed until the module for performing the first operation obtains the third data.

The information of the third data may refer to the data content of the third data or the information carried by the third data.

Examples of the above-described processes of fig. 6 and 7 may refer to the relevant steps in fig. 8 below.

Fig. 8 shows a flow chart of an execution operation phase of a data processing method according to an embodiment of the present application. As shown in fig. 8, the flow of executing the operation phase may include:

in step S801, the server performs a first operation according to the received second data.

The first operation may be one or more of data playback, data annotation, scene mining, and data simulation, and the related operation may also be any other operation that processes the second data, which is not limited in this application.

In one possible implementation, the second data received from the vehicle may be stored in a memory of the server, and the second data may be retrieved from the memory when the server performs the first operation based on the second data.

An example of this step can be seen in step S302 in fig. 3.

In step S802, the server determines a first time and a second time.

The first time may be a time when the server receives the second data, and the second time may be a time when the server starts to perform the first operation according to the second data.

In a possible implementation manner, the first time may be a time when the interface for receiving the external data in the server receives the second data. In the case where the server stores the second data received from the vehicle in the memory of the server, the first time may also be a time when the memory of the server receives the second data, and the second time may be a time when the module in the server, which is responsible for performing the first operation, acquires the second data from the memory and starts performing the first operation according to the second data.

For example, in a case where the first operation is data playback, the second time may be a time at which the server starts playback of the second data.

When the first operation is data playback and the server stores the second data received from the vehicle in the memory of the server, the first time may be a time when the memory of the server receives the second data, the second time may be a time when the module in the server responsible for data playback acquires the second data from the memory and starts a series of steps of data playback according to the second data, and the second time may be a time when the server in the server responsible for data playback starts displaying playback content on the interface according to the second data.

The determination manner of the first time and the second time can be selected as required, as long as the delay between the second time and the first time can reflect the delay from the receiving of the second data to the execution of the first operation according to the second data by the server.

It should be noted that the server may determine the first time and the second time corresponding to each frame of data in the second data, and the server may also determine the first time and the second time corresponding to the frame of data at fixed frame intervals, and the server may select how many frames are spaced according to needs to determine the time interval between the first time and the second time at each time, which is not limited in this application.

In step S803, the server determines whether the time difference between the second time and the first time exceeds a threshold.

The threshold may be set as needed, a smaller threshold may be set when the requirement on the synchronization is high, or a larger threshold may be set, and may be set in the server in advance, or may be determined by direct input of the user.

And step S804, processing the second data to obtain third data under the condition that the time difference between the second time and the first time exceeds a threshold value, and otherwise, returning to execute the step S801 and the subsequent steps.

When the time difference between the second time and the first time exceeds the threshold, the server may consider that the time delay for performing the first operation on the second data is too long, and in order to ensure that receiving the second data and performing the first operation on the second data are performed as synchronously as possible, the second data may be processed to further reduce the data amount of the second data and reduce the time delay when performing the first operation.

Wherein, the server may process the second data to obtain third data, for example, through the process shown in fig. 7; the server may also notify the vehicle, and the vehicle may process the second data to obtain the third data, for example, through the process shown in fig. 6.

The second data may include multiple types of data, and for different types of second data, a processing manner of the second data may be flexibly selected as needed to obtain third data. In one possible implementation, the second data includes image data, and processing the second data includes processing any one or more of: the resolution of the image data, the frame rate of the image data, and the second data corresponding to the first priority sensor. In yet another possible implementation, the second data includes point cloud data, and processing the second data includes processing any one or more of: the density of the point cloud data, the frame rate of the point cloud data, and second data corresponding to the first priority sensor.

In a possible implementation manner, the definition of the second data may be reduced first, for example, the definition of the related data in the second data may be reduced sequentially according to a predetermined sequence, for example, the resolution of the image data in the second data may be reduced first, and the density of the point cloud data in the second data may be reduced to a corresponding density threshold when the resolution is reduced to a corresponding resolution threshold and the time difference still exceeds the threshold. The minimum value (e.g., the resolution threshold and the density threshold) to which the second data is allowed to be reduced is set, so that the key information in the second data is not lost, for example, for the image data, the resolution is reduced to be higher than the resolution threshold, and the pedestrian existing in the image is not influenced. In a possible implementation manner, the definition of the related data in the second data may also be reduced while the data is processed, which is not limited in this application.

Secondly, the frame rate corresponding to the second data can be lowered to reduce the data amount. For example, the first frame rate corresponding to the second data may be decreased to a corresponding threshold (may be referred to as a second frame rate), the second frame rate may be determined as needed, for example, the second frame rate may be a lower limit value of the frame rate interval, and data corresponding to the second frame rate in the second data may be regarded as third data.

Again, the second data may also be filtered, for example, data other than the first priority in the second data may be selected as the third data so that the data amount of the third data is less than or equal to the second data.

The data corresponding to the first priority may be data corresponding to one or more current lowest priorities in a preset priority sequence, an order in the priority sequence may be determined according to a need of executing the first operation, and for different types of first operations, the order in the priority sequence may be different.

For example, for data playback operation, the data corresponding to the priority sequence may be chassis data, positioning data, control data, planning data, prediction data, perception data, point cloud data, and image data, the priority is sequentially increased, the data corresponding to the current first priority may be the chassis data, the chassis data in the second data may be first screened out, and under the condition that the chassis data is screened out, and the time difference between the second time and the first time still exceeds the threshold, the data corresponding to the current first priority may be continuously screened out, where the data may be positioning data.

In the above-described process of reducing the data amount of the second data, the data amount of the second data may be reduced to the threshold value at a time (for example, the resolution of the image data in the second data may be reduced to the resolution threshold value at a time) when the time difference between the second time and the first time exceeds the threshold value, or a part of the data amount may be reduced (may be set as necessary) until the resolution is reduced to the threshold value each time the first message is received, or the data amount of the second data that needs to be reduced when the time difference between the second time and the first time does not exceed the threshold value may be first calculated, and then the second data is processed according to the calculated result, which is not limited in the present application.

For example, the server may calculate a time difference between the second time and the first time according to a preset frame number interval, and send the first message to the vehicle when the time difference exceeds a threshold, when the vehicle receives the first message for the first time, may first reduce the definition of all the second data to a predetermined threshold at one time, when the vehicle receives the first message for the second time, and when the time difference still exceeds the threshold, may reduce the frame rate corresponding to the second data to a predetermined threshold (i.e., a second frame rate) at one time, when the vehicle receives the first message for the third time, and when the time difference still exceeds the threshold, may first screen out the data corresponding to the first priority in the second data, and when the first message is subsequently received, screen out the data corresponding to the current first priority in the second data in sequence until the time difference does not exceed the threshold (i.e., the first message is no longer received).

For another example, when the vehicle receives the first message for the first time, it may also first reduce a part of the resolution of the image data in the second data, for example, reduce 1/3 of the difference between the initial resolution and the corresponding resolution threshold, when the vehicle receives the first message for the second time, i.e., when the time difference still exceeds the threshold, it may continue to reduce 1/3 of the difference between the initial resolution and the corresponding resolution threshold, and when the vehicle receives the first message for the third time, i.e., when the time difference still exceeds the threshold, it reduces the resolution to the corresponding resolution threshold; secondly, when the first message is received for the fourth time and the time difference still exceeds the threshold, a part of the density of the point cloud data in the second data can be reduced, and the specific reduction mode can refer to the mode of reducing the resolution of the image data, and is not described again; when the definition of all the second data is reduced to a predetermined threshold and the time difference still exceeds the threshold, a part of the frame rate corresponding to the second data may also be reduced, for example, 1/2 of the difference between the current frame rate and the corresponding frame rate threshold is reduced, and when the time difference still exceeds the threshold, the frame rate may continue to be reduced to the corresponding frame rate threshold; when the frame rate is reduced and the time difference still exceeds the threshold, the data corresponding to the first priority in the second data may be first screened out, and when the first message is subsequently received (i.e., the time difference still exceeds the threshold), the data corresponding to the current first priority in the second data may be sequentially screened out until it is confirmed that the time difference does not exceed the threshold (i.e., the first message is not received).

For example, when the server processes the second data to obtain the third data, the server itself may process the second data according to the time difference to obtain the third data, with reference to the processes of the above two examples.

Referring back to fig. 8, in step S805, the server performs the first operation according to the third data.

The third data can be sent to the server under the condition that the vehicle processes the second data to obtain the third data, the server can obtain the third data from a memory of the server during the first operation, the server can also obtain the third data received by an interface for receiving external data, and the first operation is carried out according to the third data; when the server processes the second data to obtain the third data, the server may directly perform the first operation on the processed third data without acquiring the third data from a memory of the server or from an interface.

After step S805 is executed, step S802 and the following steps may be executed continuously, and at this time, the third data may be used as new second data to determine whether the time difference exceeds the threshold value, so as to perform the following steps.

Through the steps shown in fig. 8, the process of performing the first operation on the data by the server may be synchronized with the process of uploading the data as much as possible, so that the real-time performance of performing the first operation on the data is achieved.

The flow of executing the operation phase shown in fig. 8 is exemplarily described below with reference to fig. 9 as an example.

Fig. 9 is a schematic diagram illustrating a data playback operation according to an embodiment of the present application. As shown in fig. 9, the latest data time may be a first time, the playback data time may be a second time, and the collection speed V ₀ The speed of data uploading from the vehicle to the server (i.e. the first frame rate above) can be represented, and the playback speed V _t The speed of data playback of the uploaded data by the server can be represented, the playback speed can be determined according to the moment of starting data playback of the second data of multiple frames, for example, the number of frames played back in unit time can be used as the playback speed, and when the collection speed is greater than the playback speed, the collection speed can cause the second moment and the first moment to be in betweenThe time difference becomes large, and when the collection speed is close to the playback speed, the time difference becomes small.

As shown in FIG. 9 (a), at the start of data playback, t _delay May represent the time difference between the second time and the first time, at this time t _delay Less than threshold

Playback speed V ₁ And a collection velocity V ₀ The data uploading server and the data playback server are basically synchronous, and the data uploading server and the data playback server are basically synchronous.

As shown in fig. 9 (b), when data playback is performed, if there is fluctuation in network bandwidth of transmission data from the memory in the server to the module for performing data playback, there may be a large delay of data playback, t' _delay At this time, a time difference between the second time and the first time, at t' _delay Greater than a threshold value

Playback speed V' ₁ Less than the collection velocity V ₀ It can be considered that there is a time delay when the server performs the data playback operation on the data, and the data needs to be processed to reduce the data amount of the data.

As shown in fig. 9 (c), the data such as the image data, the point cloud data, the perception data, the prediction data, and the planning data in the data may be processed by the vehicle or the server to reduce the data amount of the data, so as to increase the speed of data playback operation on the data, and the method for reducing the data amount of the data may be shown in step S804 in fig. 8, and is not repeated here.

As shown in FIG. 9 (d), after the data amount of the data is reduced, the data playback speed, t ″, can be increased " _delay May represent the time difference between the second time and the first time, at this time t' _delay Is less than the threshold value

Playback speed V' ₁ And collection speedV ₀ The data are played back by the server basically synchronously relative to the data uploading server, so that the event remodeling cost can be reduced, and the fluency and timeliness of the playback process are stronger.

Fig. 10 shows a block diagram of a data processing apparatus according to an embodiment of the present application. The apparatus may be deployed in the vehicle described above, as shown in fig. 10, the apparatus 1000 includes:

a first obtaining module 1001, configured to obtain first data collected by a sensor;

a first determining module 1002, configured to determine a first frame rate according to a frame rate interval and an index value, where the index value is used to indicate a state of a target vehicle, and the first frame rate is within the frame rate interval;

a second determining module 1003, configured to determine second data according to the first frame rate and the first data, where information of the second data is used to instruct a server to perform a first operation.

In one possible implementation, the first operation includes one or more of: data playback, data annotation, scene mining and data simulation.

In a possible implementation manner, determining the first frame rate according to the frame rate interval and the index value includes: when all index values in the index values exceed a threshold value corresponding to each index value, the first frame rate is an upper limit value of the frame rate section; or when part of the index values exceed the threshold value corresponding to each index value, the first frame rate is larger than the lower limit value of the frame rate interval and smaller than the upper limit value of the frame rate interval; or when all index values do not exceed the threshold value corresponding to each index value, the first frame rate is the lower limit value of the frame rate section.

In one possible implementation, the apparatus further includes: the third acquisition module is used for acquiring the first message; and the processing module is used for processing the second data according to the first message to obtain third data, wherein the size of the third data is smaller than or equal to that of the second data, and the information of the third data is used for indicating the server to perform the first operation.

Fig. 11 shows a block diagram of a data processing apparatus according to an embodiment of the present application. The apparatus may be deployed in the server, and as shown in fig. 11, the apparatus 1100 includes:

a second obtaining module 1101, configured to obtain second data sent by a target vehicle, where the second data is determined according to a first frame rate and first data acquired by a sensor, where the first frame rate is determined according to a frame rate interval and an index value, the index value is used to indicate a state of the target vehicle, and the first frame rate is within the frame rate interval;

an operation module 1102, configured to perform a first operation according to the indication of the information of the second data.

In a possible implementation manner, the determining the first frame rate according to the frame rate interval and the index value includes: when all index values in the index values exceed a threshold value corresponding to each index value, the first frame rate is an upper limit value of the frame rate section; or when part of the index values exceed the threshold value corresponding to each index value, the first frame rate is larger than the lower limit value of the frame rate interval and smaller than the upper limit value of the frame rate interval; or when all index values do not exceed the threshold value corresponding to each index value, the first frame rate is the lower limit value of the frame rate section.

According to the embodiment of the application, the index value is used for indicating the state of the target vehicle, the more the number of the index values exceeding the threshold value is, the more complex the driving scene is, and the larger data amount is needed to complete the first operation, that is, the larger first frame rate is needed, whereas the less the number of the index values exceeding the threshold value is, the simpler or more regular the driving scene is, the smaller data amount is needed to complete the first operation, that is, the smaller first frame rate is needed. The first frame rate is respectively an upper limit value and a lower limit value of a frame rate interval when the first frame rate is related to the number of index values exceeding a threshold value, and the index values all exceed the threshold value and do not exceed the threshold value, and the first frame rate is located between the upper limit value and the lower limit value when part of the index values exceed the threshold value, so that the first frame rate is ensured to be within the frame rate interval, and the first frame rate can be changed according to the number of the index values exceeding the threshold value, so that the first frame rate is suitable for the change of the state of a target vehicle, and the operation requirement is met while the data redundancy is reduced.

In one possible implementation, the performing the first operation according to the indication of the information of the second data includes: obtaining third data according to the indication of the information of the second data, wherein the size of the third data is smaller than or equal to that of the second data; and executing a first operation according to the indication of the information of the third data.

In one possible implementation, the apparatus further includes: and the generating module is used for generating a first message, wherein the first message indicates that the time difference between a second time and a first time exceeds a preset threshold, the first time is the time when the second data is received, and the second time is the time when the first operation is started to be executed according to the second data.

In one possible implementation, the apparatus further includes: and the sending module is used for sending the first message to the target vehicle, and the first message is used for indicating the target vehicle to process the first data.

Fig. 12 shows a block diagram of a data processing apparatus according to an embodiment of the present application. The data processing device may be applied to the data processing system shown in fig. 1, and performs the functions of the vehicle or the server in the data processing method shown in any one of fig. 2 to 9. For example, the data processing device may be the server or the vehicle, or may be a chip (system) or other components or assemblies that may be disposed inside the server or inside the vehicle. For example, the data processing apparatus may be the data processing apparatus 1000 or the data processing apparatus 1100. The embodiments of the present application do not limit this.

As shown in fig. 12, the data processing apparatus 700 may include a processor 701 and a transceiver 702. Optionally, the data processing apparatus 700 may comprise a memory 703. The processor 701 is coupled to a transceiver 702 and a memory 703, such as may be connected by a communication bus.

The respective constituent components of the data processing device 700 will be specifically described below with reference to fig. 12.

The processor 701 is a control center of the data processing apparatus 700, and may be a single processor or a combination of multiple processing elements. For example, the processor 701 is one or more Central Processing Units (CPUs), or may be an Application Specific Integrated Circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present application, for example: one or more microprocessors (digital signal processors, DSPs), or one or more Field Programmable Gate Arrays (FPGAs).

Alternatively, the processor 701 may perform various functions of the data processing apparatus 700 by running or executing software programs stored in the memory 703 and calling data stored in the memory 703.

In a particular implementation, processor 701 may include one or more CPUs, such as CPU0 and CPU1 shown in fig. 12, as one embodiment.

In one possible implementation, the data processing apparatus 700 may also include multiple processors, such as the processor 701 and the processor 704 shown in fig. 12. Each of these processors may be a single-Core Processor (CPU) or a multi-Core Processor (CPU). A processor herein may refer to one or more communication devices, circuitry, and/or processing cores for processing data (e.g., computer program instructions).

The transceiver 702 is used for communication with other data processing devices. For example, referring to fig. 1, where the data processing device 700 is a vehicle, the transceiver 702 may be used to communicate with a server, or with another vehicle. As another example, where data processing device 700 is a server, transceiver 702 may be used to communicate with a vehicle, or with another server.

Optionally, the transceiver 702 may include a receiver and a transmitter (not separately shown in fig. 12). Wherein the receiver is configured to perform a receiving function and the transmitter is configured to perform a transmitting function.

Alternatively, the transceiver 702 may be integrated with the processor 701, or may exist independently, and is coupled to the processor 701 through an input/output port (not shown in fig. 12) of the data processing apparatus 700, which is not limited in this embodiment.

The memory 703 may be used to store a software program for executing the scheme of the present application, and the processor 701 controls the execution of the software program.

The memory 703 may be, but is not limited to, a read-only memory (ROM) or other type of static storage communication device that can store static information and instructions, a Random Access Memory (RAM) or other type of dynamic storage communication device that can store information and instructions, an electrically erasable programmable read-only memory (EEPROM), a compact disk read-only memory (CD-ROM) or other optical disk storage, optical disk storage (including compact disk, laser disk, optical disk, digital versatile disk, blu-ray disk, etc.), a magnetic disk storage medium or other magnetic storage communication device, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and that can be accessed by a computer. It should be noted that the memory 703 may be integrated with the processor 701, or may exist independently, and is coupled to the processor 701 through an input/output port (not shown in fig. 12) of the data processing apparatus 700, which is not limited in this embodiment.

It should be noted that the structure of the data processing apparatus 700 shown in fig. 12 does not constitute a limitation to the implementation of the data processing apparatus, and an actual data processing apparatus may include more or less components than those shown, or combine some components, or arrange different components.

An embodiment of the present application provides a data processing apparatus, including: a data processing apparatus, the apparatus comprising: a processor and a memory; the memory is used for storing programs; the processor is used for executing the program stored in the memory so as to enable the device to realize the method.

Embodiments of the present application provide a terminal device, which may perform the method shown in any one of the embodiments of fig. 2 to 9.

Embodiments of the present application provide a computer-readable storage medium, comprising: computer instructions which, when executed, implement the method of any of the embodiments of figures 2 to 9 described above.

Embodiments of the present application provide a computer-readable medium comprising instructions that, when executed on a computer, cause the computer to perform the method as described in any of the embodiments of fig. 2-9 above.

Embodiments of the present application provide an in-vehicle computing device comprising a processor configured to perform the method illustrated in any of the embodiments of fig. 2-9 described above.

Embodiments of the present application provide a server including a processor configured to perform the method shown in any one of the embodiments of fig. 2 to 9.

Embodiments of the present application provide a chip system, which includes at least one processor, and is configured to support implementation of the method shown in any one of fig. 2 to 9.

In a possible implementation manner, the chip system further includes a memory, which is used for storing program instructions and data, the memory is located inside the processor or outside the processor, and the chip system may be formed by a chip and may also include a chip and other discrete devices.

Embodiments of the present application provide a data processing apparatus comprising at least one processor and a communication interface for transmitting and/or receiving data, the at least one processor being configured to invoke a computer program stored in at least one memory to cause the data processing apparatus to implement the above-described data processing method.

The computer readable storage medium may be a tangible device that can hold and store the instructions for use by the instruction execution device. The computer readable storage medium may be, for example, but not limited to, an electronic memory device, a magnetic memory device, an optical memory device, an electromagnetic memory device, a semiconductor memory device, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of the computer readable storage medium would include the following: a portable computer diskette, a hard disk, a Random Access Memory (RAM), a Read-Only Memory (ROM), an erasable Programmable Read-Only Memory (EPROM or flash Memory), a Static Random Access Memory (SRAM), a portable Compact Disc Read-Only Memory (CD-ROM), a Digital Versatile Disc (DVD), a Memory stick, a floppy disk, a mechanical coding device, a punch card or an in-groove protrusion structure, for example, having instructions stored thereon, and any suitable combination of the foregoing.

The computer readable program instructions or code described herein may be downloaded from a computer readable storage medium to a respective computing/processing device, or to an external computer or external storage device over a network, such as the internet, a local area network, a wide area network, and/or a wireless network. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers and/or edge servers. The network adapter card or network interface in each computing/processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in a computer-readable storage medium in the respective computing/processing device.

The computer program instructions for carrying out operations of the present application may be assembler instructions, instruction Set Architecture (ISA) instructions, machine-related instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including an object oriented programming language such as Smalltalk, C + + or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The computer-readable program instructions may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of Network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet service provider). In some embodiments, the electronic circuitry can execute computer-readable program instructions to implement aspects of the present application by utilizing state information of the computer-readable program instructions to personalize custom electronic circuitry, such as Programmable Logic circuits, field-Programmable Gate arrays (FPGAs), or Programmable Logic Arrays (PLAs).

Various aspects of the present application are described herein with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the application. It will be understood that each block of the flowchart illustrations and/or block diagrams, and combinations of blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer readable program instructions.

These computer-readable program instructions may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions/acts specified in the flowchart and/or block diagram block or blocks. These computer-readable program instructions may also be stored in a computer-readable storage medium that can direct a computer, programmable data processing apparatus, and/or other devices to function in a particular manner, such that the computer-readable medium storing the instructions comprises an article of manufacture including instructions which implement the function/act specified in the flowchart and/or block diagram block or blocks.

The computer readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other devices to cause a series of operational steps to be performed on the computer, other programmable apparatus or other devices to produce a computer implemented process such that the instructions which execute on the computer, other programmable apparatus or other devices implement the functions/acts specified in the flowchart and/or block diagram block or blocks.

The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of apparatus, systems, methods and computer program products according to various embodiments of the present application. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of instructions, which comprises one or more executable instructions for implementing the specified logical function(s). In some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved.

It is also noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by hardware (e.g., a Circuit or an ASIC) for performing the corresponding function or action, or by combinations of hardware and software, such as firmware.

While the invention has been described in connection with various embodiments, other variations to the disclosed embodiments can be understood and effected by those skilled in the art in practicing the claimed invention, from a review of the drawings, the disclosure, and the appended claims. In the claims, the word "comprising" does not exclude other elements or steps, and the word "a" or "an" does not exclude a plurality. A single processor or other unit may fulfill the functions of several items recited in the claims. The mere fact that certain measures are recited in mutually different dependent claims does not indicate that a combination of these measures cannot be used to advantage.

Having described embodiments of the present application, the foregoing description is intended to be exemplary, not exhaustive, and not limited to the disclosed embodiments. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope of the described embodiments. The terminology used herein is chosen in order to best explain the principles of the embodiments, the practical application, or improvements made to the technology in the marketplace, or to enable others of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. A method of data processing, the method comprising:

acquiring first data acquired by a sensor;

determining a first frame rate according to a frame rate interval and an index value, wherein the index value is used for indicating the state of a target vehicle, and the first frame rate is within the frame rate interval;

and determining second data according to the first frame rate and the first data, wherein the information of the second data is used for instructing a server to execute a first operation.

2. The method of claim 1, wherein determining the first frame rate based on the frame rate interval and the indicator value comprises:

when all index values in the index values exceed a threshold value corresponding to each index value, the first frame rate is an upper limit value of the frame rate interval; or

When part of the index values exceed the threshold value corresponding to each index value, the first frame rate is larger than the lower limit value of the frame rate interval and smaller than the upper limit value of the frame rate interval; or

And when all the index values do not exceed the threshold value corresponding to each index value, the first frame rate is the lower limit value of the frame rate section.

3. The method according to claim 1 or 2, characterized in that the method further comprises:

acquiring a first message;

and processing the second data according to the first message to obtain third data, wherein the size of the third data is smaller than or equal to that of the second data, and the information of the third data is used for indicating the server to perform the first operation.

4. The method according to claim 3, wherein the first message indicates that a time difference between a first time and a second time exceeds a predetermined threshold, wherein the first time is a time when the server receives the second data or the third data, and the second time is a time when the server starts to perform the first operation according to the received second data or the third data.

5. The method of any of claims 1-4, wherein the first operation comprises one or more of: data playback, data annotation, scene mining and data simulation.

6. A method of data processing, the method comprising:

acquiring second data sent by a target vehicle, wherein the second data is determined by the target vehicle according to a first frame rate and first data acquired by a sensor, the first frame rate is determined according to a frame rate interval and an index value, the index value is used for indicating the state of the target vehicle, and the first frame rate is within the frame rate interval;

and performing a first operation according to the indication of the information of the second data.

7. The method of claim 6, wherein the first frame rate is determined according to a frame rate interval and an index value, comprising:

8. The method according to claim 6 or 7, wherein the performing a first operation according to the indication of the information of the second data comprises:

obtaining third data according to the indication of the information of the second data, wherein the size of the third data is smaller than or equal to that of the second data;

and executing a first operation according to the indication of the information of the third data.

9. The method according to any one of claims 6-8, further comprising:

and generating a first message, wherein the first message indicates that the time difference between a second time and a first time exceeds a preset threshold, the first time is the time when the second data is received, and the second time is the time when the first operation is started according to the second data.

10. The method of claim 9, further comprising:

and sending the first message to the target vehicle, wherein the first message is used for instructing the target vehicle to process the first data.

11. The method according to any of claims 6-10, wherein the first operation comprises one or more of: data playback, data annotation, scene mining and data simulation.

12. A data processing apparatus, characterized in that the apparatus comprises:

the first acquisition module is used for acquiring first data acquired by the sensor;

the system comprises a first determination module, a second determination module and a third determination module, wherein the first determination module is used for determining a first frame rate according to a frame rate interval and an index value, the index value is used for indicating the state of a target vehicle, and the first frame rate is within the frame rate interval;

and the second determining module is used for determining second data according to the first frame rate and the first data, and the information of the second data is used for indicating the server to execute the first operation.

13. The apparatus of claim 12, wherein determining the first frame rate based on the frame rate interval and the index value comprises:

when all index values in the index values exceed a threshold value corresponding to each index value, the first frame rate is an upper limit value of the frame rate section; or

14. The apparatus of claim 12 or 13, further comprising:

the third acquisition module is used for acquiring the first message;

and the processing module is used for processing the second data according to the first message to obtain third data, wherein the size of the third data is smaller than or equal to that of the second data, and the information of the third data is used for indicating the server to perform the first operation.

15. The apparatus according to claim 14, wherein the first message indicates that a time difference between a first time and a second time exceeds a predetermined threshold, the first time is a time when the second data or the third data is received by the server, and the second time is a time when the server starts to perform the first operation according to the received second data or the third data.

16. The apparatus of any of claims 12-15, wherein the first operation comprises one or more of: data playback, data annotation, scene mining and data simulation.

17. A data processing apparatus, characterized in that the apparatus comprises:

the second acquisition module is used for acquiring second data sent by a target vehicle, wherein the second data is determined by the target vehicle according to a first frame rate and first data acquired by a sensor, the first frame rate is determined according to a frame rate interval and an index value, the index value is used for indicating the state of the target vehicle, and the first frame rate is within the frame rate interval;

and the operation module is used for performing first operation according to the indication of the information of the second data.

18. The apparatus of claim 17, wherein the first frame rate is determined according to a frame rate interval and an index value, comprising:

19. The apparatus according to claim 17 or 18, wherein the performing a first operation according to the indication of the information of the second data comprises:

20. The apparatus of any one of claims 17-19, further comprising:

and the generating module is used for generating a first message, wherein the first message indicates that the time difference between a second time and a first time exceeds a preset threshold, the first time is the time when the second data is received, and the second time is the time when the first operation is started to be executed according to the second data.

21. The apparatus of claim 20, further comprising:

and the sending module is used for sending the first message to the target vehicle, and the first message is used for indicating the target vehicle to process the first data.

22. The apparatus of any of claims 17-21, wherein the first operation comprises one or more of: data playback, data annotation, scene mining and data simulation.

23. A data processing apparatus, comprising: a processor and a memory;

the memory is used for storing programs;

the processor is configured to execute the program stored in the memory to cause the apparatus to implement the method of any one of claims 1-5 or to implement the method of any one of claims 6-11.

24. A computer-readable storage medium, comprising: computer instructions which, when executed, implement the method of any one of claims 1 to 5 or the method of any one of claims 6 to 11.

25. A computer program product comprising instructions which, when run on a computer, cause the computer to perform the method of any one of claims 1-5 or perform the method of any one of claims 6-11.

26. An in-vehicle computing device, characterized in that the in-vehicle computing device comprises a processor for performing the data processing method according to any of claims 1-5.

27. A server, characterized in that the server comprises a processor for performing the data processing method of any of claims 6-11.