CN111930306A

CN111930306A - Data processing method and device

Info

Publication number: CN111930306A
Application number: CN202010744185.2A
Authority: CN
Inventors: 李春晓; 潘余昌; 王建伟
Original assignee: Beijing Jingdong Qianshi Technology Co Ltd
Current assignee: Beijing Jingdong Qianshi Technology Co Ltd
Priority date: 2020-07-29
Filing date: 2020-07-29
Publication date: 2020-11-13

Abstract

The invention discloses a data processing method and device, and relates to the technical field of computers. One embodiment of the method comprises: monitoring automatic driving data of a travelable device, and determining a first data level and a first time interval of the monitored automatic driving data; storing data information of the automatic driving data with the generation time in a first time interval in a data information table in correspondence with a first data level; and when the data deletion is carried out, deleting the automatic driving data one by one according to the data levels in the data information table and the generation moments in the corresponding data information according to a preset deletion sequence until the data storage space meets the preset requirement. The implementation mode can regularly maintain the states of the data and the disks in a simplified and convenient mode, reduce the network requirement and the disk pressure generated by redundant data, conditionally reserve or delete the data, is compatible with the disks with different capacities, meets the processing requirement of an automatic driving scene, and improves the automation degree of data processing in the scene.

Description

Data processing method and device

Technical Field

The present invention relates to the field of computer technologies, and in particular, to a data processing method and apparatus.

Background

Each travelable device (e.g., autonomous vehicle) generates a large amount of data, such as source data, intermediate data, etc., during routine testing or operation, which places a large strain on the capacity of the travelable device (end-of-vehicle) disk. If the data at the vehicle end is not processed in time, situations such as data loss and error reporting of the disk will be caused.

Data generated by the autopilot side is stored on a magnetic disk, and although a large-capacity hard disk can be mounted, the use and maintenance of the data in the large-capacity hard disk are inconvenient. For example, if the hard disk is replaced when the user is full, the hard disk can reach the user's hand only by obtaining the transfer of the physical space; and by adopting the mode of cleaning the disk space, if the disk space capacity is large, the accumulated historical data is more, the cleaning time is longer, and if the disk space capacity is smaller, the corresponding scene data may be omitted, lost or unavailable, which is very inconvenient. In addition, the requirement for network and flow is high when data are uploaded in large batches, the uploading time is long when the data volume is large, the drivable device is always in a mobile state, the network state is not completely stable even if a flow card is loaded, the requirement for flow is high, and the method is not suitable for transmission of a large amount of data.

In the process of implementing the invention, the inventor finds that at least the following problems exist in the prior art:

data maintenance is inconvenient, the requirement on a network is high, the disk pressure of the driving equipment cannot be effectively reduced, and the data processing requirement of an automatic driving scene cannot be met.

Disclosure of Invention

In view of this, embodiments of the present invention provide a data processing method and apparatus, which can regularly maintain data and disk states in a simplified and convenient manner, reduce network requirements and disk pressures generated by redundant data, conditionally retain or delete data, be compatible with disks of different capacities, meet data processing requirements in an autopilot scenario, and improve the automation degree of data processing in the autopilot scenario.

To achieve the above object, according to an aspect of an embodiment of the present invention, there is provided a data processing method.

A method of data processing, comprising: monitoring automatic driving data generated by a travelable device at each moment, and determining a first data level and a first time interval of the monitored automatic driving data after the automatic driving data is monitored, wherein the first time interval comprises a first moment which is the generation moment of the monitored automatic driving data; storing data information of the automatic driving data with the generation time in the first time interval in a data information table in correspondence with the first data level, wherein the data information comprises the generation time and the storage path of the corresponding automatic driving data; and when data deletion is carried out, deleting the automatic driving data under the storage path one by one according to the data level in the data information table and the generation time of the automatic driving data in the corresponding data information according to a preset deletion sequence, and stopping the data deletion operation until the data storage space of the travelable equipment meets a preset requirement.

Optionally, the step of determining a first data level of the monitored autonomous driving data comprises: determining a source module of the monitored autopilot data, the source module being a functional module in an autopilot program on the travelable device; and presetting the monitored automatic driving data according to the processing logic of the source module, and determining a first data level of the monitored automatic driving data according to an obtained processing result.

Optionally, the step of determining a first time interval of the monitored autonomous driving data comprises: taking a time interval formed by a first selected time point before the first time and a second selected time point after the first time as a pending time interval; searching whether a target time interval with intersection with the undetermined time interval exists in a level interval table, wherein the level interval table is used for storing each data level and the corresponding time interval; under the condition that the target time interval exists, if the data level corresponding to the target time interval is higher than the first data level, taking the part outside the intersection of the undetermined time interval and the target time interval as the first time interval; if the data level corresponding to the target time interval is lower than the first data level, taking the undetermined time interval as the first time interval; if the data level corresponding to the target time interval is the same as the first data level, taking the union of the target time interval and the undetermined time interval as the first time interval; and taking the undetermined time interval as the first time interval under the condition that the target time interval does not exist.

Optionally, after determining the first data level and the first time interval of the monitored autonomous driving data, further comprising: correspondingly storing the first data level and the first time interval in the level interval table; the method further comprises the following steps: updating the data information table at regular time according to the corresponding relation between the data level and the time interval in the level interval table, wherein the updating comprises the following steps: polling each piece of automatic driving data under the storage path, determining a second data level of the polled automatic driving data according to a time interval of the generation time of the polled automatic driving data, and inserting the data information of the polled automatic driving data into a list of data information corresponding to the second data level in the data information table.

Optionally, the method further comprises: when data uploading is carried out, acquiring the designated automatic driving data to be uploaded from the storage path and uploading the designated automatic driving data to the server under the condition that an uploading instruction designates the automatic driving data to be uploaded, wherein the uploading instruction is generated when a triggering condition of the data uploading is reached; when the uploading instruction does not specify the automatic driving data to be uploaded, uploading the automatic driving data under the storage path one by one according to a data level in the data information table and a generation time of the automatic driving data in the corresponding data information according to a preset uploading sequence, wherein when the uploaded automatic driving data is of a specific data level, the data information of the uploaded automatic driving data is deleted from a first list of the data information table and added to a second list of the data information table, the first list is a list of data information corresponding to the specific data level in the data information table, the second list is a list of data information corresponding to a priority deletion level in the data information table, and the specific data level and the priority deletion level are two different data levels, and the particular data level is of higher importance than the prioritized deletion level.

Optionally, the preset deleting sequence is: deleting the automatic driving data of different data levels according to the sequence of the importance of the corresponding data levels from low to high, and deleting the automatic driving data of the same data level according to the sequence of the generation time from front to back; the preset uploading sequence is as follows: and uploading the automatic driving data of different data levels according to the importance of the corresponding data levels from high to low, and uploading the automatic driving data of the same data level according to the sequence of the generation time from front to back.

Optionally, when data deletion is performed, deleting the automatic driving data in the storage path one by one according to a preset deletion order according to the data level in the data information table and the generation time of the automatic driving data in the corresponding data information, where the deleting includes: and when the data deletion is executed, deleting the automatic driving data in the storage path in steps, wherein the generation time of the automatic driving data is before one or more preset time thresholds, each step of deletion operation corresponds to one time threshold, and after the step of deletion is executed, deleting the automatic driving data in the storage path one by one according to a preset deletion sequence according to the data level in the data information table and the generation time of the corresponding data information under the condition that the data storage space still does not meet the preset requirement.

According to another aspect of the embodiments of the present invention, there is provided a data processing apparatus.

A data processing apparatus comprising: the system comprises a mark monitoring unit, a data processing unit and a data processing unit, wherein the mark monitoring unit is used for monitoring automatic driving data generated by a travelable device at each moment, and determining a first data level and a first time interval of the monitored automatic driving data after the automatic driving data is monitored, the first time interval comprises a first moment, and the first moment is the generation moment of the monitored automatic driving data; the updating unit is used for storing data information of the automatic driving data with the generation time in the first time interval in a data information table in a corresponding mode with the first data level, and the data information comprises the generation time and the storage path of the corresponding automatic driving data; and the automatic deleting unit is used for deleting the automatic driving data under the storage path one by one according to a preset deleting sequence according to the data level in the data information table and the generation time of the automatic driving data in the corresponding data information when data deletion is carried out, and stopping the data deleting operation until the data storage space of the travelable equipment meets a preset requirement.

Optionally, the tag monitoring unit is further configured to: determining a source module of the monitored autopilot data, the source module being a functional module in an autopilot program on the travelable device; and presetting the monitored automatic driving data according to the processing logic of the source module, and determining a first data level of the monitored automatic driving data according to an obtained processing result.

Optionally, wherein the marker monitoring unit is further configured to: taking a time interval formed by a first selected time point before the first time and a second selected time point after the first time as a pending time interval; searching whether a target time interval with intersection with the undetermined time interval exists in a level interval table, wherein the level interval table is used for storing each data level and the corresponding time interval; under the condition that the target time interval exists, if the data level corresponding to the target time interval is higher than the first data level, taking the part outside the intersection of the undetermined time interval and the target time interval as the first time interval; if the data level corresponding to the target time interval is lower than the first data level, taking the undetermined time interval as the first time interval; if the data level corresponding to the target time interval is the same as the first data level, taking the union of the target time interval and the undetermined time interval as the first time interval; and taking the undetermined time interval as the first time interval under the condition that the target time interval does not exist.

Optionally, the tag monitoring unit is further configured to: correspondingly storing the first data level and the first time interval in the level interval table; the update unit is further configured to: updating the data information table at regular time according to the corresponding relation between the data level and the time interval in the level interval table, wherein the updating comprises the following steps: polling each piece of automatic driving data under the storage path, determining a second data level of the polled automatic driving data according to a time interval of the generation time of the polled automatic driving data, and inserting the data information of the polled automatic driving data into a list of data information corresponding to the second data level in the data information table.

Optionally, the apparatus further comprises an uploading unit, configured to: when data uploading is carried out, acquiring the designated automatic driving data to be uploaded from the storage path and uploading the designated automatic driving data to the server under the condition that an uploading instruction designates the automatic driving data to be uploaded, wherein the uploading instruction is generated when a triggering condition of the data uploading is reached; when the uploading instruction does not specify the automatic driving data to be uploaded, uploading the automatic driving data under the storage path one by one according to a data level in the data information table and a generation time of the automatic driving data in the corresponding data information according to a preset uploading sequence, wherein when the uploaded automatic driving data is of a specific data level, the data information of the uploaded automatic driving data is deleted from a first list of the data information table and added to a second list of the data information table, the first list is a list of data information corresponding to the specific data level in the data information table, the second list is a list of data information corresponding to a priority deletion level in the data information table, and the specific data level and the priority deletion level are two different data levels, and the particular data level is of higher importance than the prioritized deletion level.

Optionally, the automatic deleting unit is further configured to: and when the data deletion is executed, deleting the automatic driving data in the storage path in steps, wherein the generation time of the automatic driving data is before one or more preset time thresholds, each step of deletion operation corresponds to one time threshold, and after the step of deletion is executed, deleting the automatic driving data in the storage path one by one according to a preset deletion sequence according to the data level in the data information table and the generation time of the corresponding data information under the condition that the data storage space still does not meet the preset requirement.

According to yet another aspect of an embodiment of the present invention, an electronic device is provided.

An electronic device, comprising: one or more processors; a memory for storing one or more programs, which when executed by the one or more processors, cause the one or more processors to implement the data processing method provided by the embodiments of the present invention.

According to yet another aspect of an embodiment of the present invention, a computer-readable medium is provided.

A computer-readable medium, on which a computer program is stored, which, when executed by a processor, implements a data processing method provided by an embodiment of the present invention.

One embodiment of the above invention has the following advantages or benefits: monitoring the automatic driving data generated by the travelable device at each moment, and determining a first data level and a first time interval of the monitored automatic driving data, wherein the first time interval comprises the generation moment of the monitored automatic driving data; storing data information of the automatic driving data with the generation time in a first time interval in a data information table in correspondence with a first data level; and when the data deletion is carried out, deleting the automatic driving data under the storage path one by one according to the data level in the data information table and the generation time of the automatic driving data in the corresponding data information according to a preset deletion sequence, and stopping the data deletion until the data storage space of the travelable equipment meets the preset requirement. The method can maintain the states of the data and the disks at regular time in a simple and convenient mode, reduce network requirements and disk pressure generated by redundant data, conditionally reserve or delete the data, be compatible with the disks with different capacities, meet the data processing requirements of an automatic driving scene, and improve the automation degree of data processing in the automatic driving scene.

Further effects of the above-mentioned non-conventional alternatives will be described below in connection with the embodiments.

Drawings

The drawings are included to provide a better understanding of the invention and are not to be construed as unduly limiting the invention. Wherein:

FIG. 1 is a schematic diagram of the main steps of a data processing method according to one embodiment of the present invention;

FIG. 2 is a schematic diagram of a level interval table according to one embodiment of the invention;

FIG. 3 is a schematic diagram of a data information table according to one embodiment of the invention;

FIG. 4 is a schematic diagram of a process for updating a data information table according to an embodiment of the present invention;

FIG. 5 is a flow diagram of data deletion according to one embodiment of the invention;

FIG. 6 is a main configuration diagram of a data processing apparatus according to an embodiment of the present invention;

FIG. 7 is a schematic diagram of the operation of a data processing apparatus according to one embodiment of the present invention;

FIG. 8 is an exemplary system architecture diagram in which embodiments of the present invention may be employed;

fig. 9 is a schematic block diagram of a computer system suitable for use in implementing a terminal device of an embodiment of the present invention.

Detailed Description

Exemplary embodiments of the present invention are described below with reference to the accompanying drawings, in which various details of embodiments of the invention are included to assist understanding, and which are to be considered as merely exemplary. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the invention. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.

Fig. 1 is a schematic diagram of main steps of a data processing method according to an embodiment of the present invention.

As shown in fig. 1, the data processing method according to an embodiment of the present invention mainly includes steps S101 to S103 as follows.

Step S101: the method includes monitoring autopilot data generated by a travelable device at each time, and determining a first data level and a first time interval of the monitored autopilot data each time autopilot data is monitored, the first time interval including a first time, the first time being a time of generation of the monitored autopilot data.

Step S102: and storing data information of the automatic driving data with the generation time in the first time interval in a data information table in correspondence with the first data level, wherein the data information comprises the generation time and the storage path of the corresponding automatic driving data.

Step S103: and when the data deletion is executed, deleting the automatic driving data under the storage path one by one according to the data level in the data information table and the generation time of the automatic driving data in the corresponding data information according to a preset deletion sequence, and stopping the data deletion operation until the data storage space of the travelable equipment meets the preset requirement.

The drivable device can be an autonomous vehicle, such as an AGV (automatic guided vehicle), among others.

The drivable device includes an autopilot module, which is an autopilot program of the vehicle itself, and includes all modules required for supporting the autopilot of the vehicle, such as a positioning module, a planning control module, a sensing module, sensor hardware, a driver module, and other modules. The travelable device according to the embodiment of the present invention generates the data (sensor data, data calculated and processed by each module, and the like) generated by the autopilot module, which is the autopilot data generated at each time.

In the embodiment of the present invention, the automatic driving data generated by the travelable device at each time may include the automatic driving data newly created or the automatic driving data newly modified.

In the embodiment of the present invention, "time" refers to a time point at which the travelable apparatus generates the automated driving data, for example, if automated driving data are generated at t1, t2, … …, tn, respectively, "time" in the automated driving data generated at each time by the monitoring travelable apparatus refers to t1, t2, … …, tn.

The data level of the embodiment of the invention can be customized, is a priority sequence set for conveniently and orderly managing the data, and aims to more intelligently and accurately maintain the disk space, delete invalid data and keep meaningful data. The data levels may be roughly divided into 5 types (including, but not limited to, may be increased as needed, or otherwise define the manner in which the data levels are divided): HIGH _ RETAINING (HIGH priority reservation); MEDIUM _ RETAINING (MEDIUM priority reservation); LOW _ RETAINING (LOW priority reservation); NORMAL grade; DELETE (priority DELETE level).

The storage path of the autopilot data is a path in which the autopilot data is stored in a magnetic disk of the traveling device, and the storage path may specifically include multiple paths, for example, autopilot data from different source modules are stored in different paths.

The data storage space of the travelable device is the disk space of the travelable device.

In one embodiment, the step of determining a first data level of the monitored autonomous driving data comprises: determining a source module of the monitored automatic driving data, wherein the source module is a functional module in an automatic driving program on the drivable device; and presetting the monitored automatic driving data according to the processing logic of the source module, and determining a first data level of the monitored automatic driving data according to the obtained processing result.

The functional modules in the autopilot program include, for example, the positioning module, the planning control module, the sensing module, the sensor hardware, the driver module, and other modules described above. Each functional module has respective processing logic, and the functional modules have cooperative processing logic.

The source module of the monitored autopilot data may be one or more and the processing logic of the source module may include the processing logic of the source module itself and processing logic in cooperation with other functional modules.

The method comprises the steps of presetting the monitored automatic driving data according to the processing logic of a source module, namely processing the monitored automatic driving data according to the processing logic of the source module or the processing logic cooperated with other functional modules, wherein the processing result is a certain value or a certain group of values generally. The first data level of the monitored automatic driving data is determined according to the obtained processing result, for example, it can be determined which preset interval range the value of the processing result belongs to (the interval range is preset, a group of interval thresholds with different sizes can be preset, and are arranged according to the ascending order or the descending order of the size sequence of the interval thresholds, and an interval range is obtained by limiting two adjacent interval thresholds), and each interval range corresponds to one data level, so that the first data level is determined. If the processing result is a certain group of values, the range of each group of intervals for judging each value can be set respectively, and the specific data level to which the value belongs can be determined according to the corresponding relation between the predefined interval combination and the data level through the combination of the intervals where the values are located. The specific rules are not limited to those listed above.

Taking the monitored automatic driving data as an example of positioning data, the positioning data such as longitude and latitude coordinates of a vehicle, the longitude and latitude coordinates of the current position of the vehicle can be calculated by using the vehicle speed and the previously acquired longitude and latitude coordinates of the vehicle (i.e. the positioning data of the last moment of the current moment), according to the processing logic of the positioning module, by calculating the deviation between the longitude and latitude coordinates in the monitored positioning data and the longitude and latitude coordinates of the current position of the vehicle, whether the deviation exists in the positioning data of the monitored automatic driving data and the specific deviation amount can be judged, so as to obtain the deviation value of a positioning position (for example, the straight line distance between the monitored positioning position and the position deviates by 1 meter and the like through the longitude and latitude coordinates), a plurality of interval ranges are preset to respectively correspond to each data level, for example, the interval range where the deviation is 1 meter corresponds to the data level of LOW-priority-RETAINING (LOW-priority reservation), while the range of the interval within which the deviation is 10 meters corresponds to the data level HIGH RETAINING (HIGH priority reservation), and so on.

In one embodiment, the step of determining a first time interval of monitored autonomous driving data comprises: taking a time interval formed by a first selected time point before the first time and a second selected time point after the first time as a pending time interval; searching whether a target time interval with intersection with the undetermined time interval exists in a level interval table, wherein the level interval table is used for storing each data level and the corresponding time interval; under the condition that the target time interval exists, if the data level corresponding to the target time interval is higher than the first data level, taking the part outside the intersection of the undetermined time interval and the target time interval as the first time interval; if the data level corresponding to the target time interval is lower than the first data level, taking the undetermined time interval as a first time interval; if the data level corresponding to the target time interval is the same as the first data level, taking the union of the target time interval and the undetermined time interval as the first time interval; and taking the pending time interval as a first time interval when the target time interval does not exist.

The first selected point in time and the second selected point in time may be set as desired, for example 30s (seconds) before the first time and 30s after the first time, respectively.

The intersection of the two time intervals refers to a part where the two time intervals are overlapped, and the union of the two time intervals refers to the maximum range covered by the two time intervals.

For example, if the first time is 35s, the pending time interval is [5s,65s ], assuming that there is a target time interval [0s,30s ] intersecting the pending time interval (denoted as interval a), if the interval a corresponds to a data level higher than the first data level, the first time interval is [31s,65s ], if the interval a corresponds to a data level lower than the first data level, the first time interval is [5s,65s ], if the interval a corresponds to a data level equal to the first data level, the first time interval is [0s,65s ]. Accordingly, the original range of the interval a needs to be updated, for example, if the first time interval is [5s,65s ], the interval a needs to be changed accordingly, and the new range of the interval a is [0s,4s ], so that the data level corresponding to the same data is unique. When the first time interval is [0s,65s ], the original interval a needs to be deleted, which is equivalent to combining the first time interval and the interval a into one time interval.

Fig. 2 is a schematic diagram of a level interval table according to an embodiment of the present invention, in which data levels are reserved with HIGH _ RETAINING (HIGH priority); MEDIUM _ RETAINING (MEDIUM priority reservation); LOW _ RETAINING (LOW priority reservation); NORMAL grade; for example, DELETE (priority DELETE level) is an example, each data level has a corresponding time interval, and the time intervals corresponding to the same data level may be one or more. The priority interval is a chain structure formed by a plurality of groups of time intervals. The relationship between the data level and the priority interval is as follows: each data level corresponds to a set of priority intervals, i.e. the data level corresponds to one or more time intervals, each time interval consisting of a start time and an end time. To improve the operation efficiency, the priority intervals are usually stored as an ordered chain structure, i.e., in ascending order of start _ time. The time of the data (i.e., the automatic driving data) is an attribute of the data and is a consistent time taken in the embodiment of the present invention, such as the creation time of the data or the last modification time of the data, which is the generation time of the automatic driving data. If the time of the data belongs to a certain time interval, the data belongs to the data level corresponding to the time interval. The level section table is a table structure held in the form of key-value pairs, in which a data level is a key and a plurality of priority sections are values.

After determining the first data level and the first time interval of the monitored autonomous driving data, further comprising: and correspondingly storing the first data level and the first time interval in the level interval table.

In one embodiment, the updating of the data information table according to the corresponding relationship between the data level and the time interval in the level interval table includes: polling each automatic driving data in the storage path, determining a second data level of the polled automatic driving data according to a time interval of the generation time of the polled automatic driving data, and inserting the data information of the polled automatic driving data into a list of data information corresponding to the second data level in the data information table.

A schematic diagram of a data information table according to an embodiment of the present invention is shown in fig. 3. As in fig. 3, the data level is HIGH _ RETAINING (HIGH priority reservation); … …, respectively; DELETE (priority DELETE level) is an example, each data level corresponds to one or more data information (denoted data _ info, e.g., data _ info _0, … …, data _ info _ n, etc.). The data information table is a table structure stored in the form of key-value key value pairs. Here, the data level is used as a key, a plurality of data _ info (data information) is used as a value, and the data information is also inserted in order. The conditions under which data information is attributed to a certain data level are: the time of the data belongs to a certain time interval of the data level.

Fig. 4 is a schematic diagram of an update flow of a data information table according to an embodiment of the present invention. According to fig. 4, data under a specified path, that is, data under a storage path of the automated driving data, is polled, and by taking the data levels as a retaining level, a delete level, and a normal level as examples, whether the polled data belongs to a time interval of the retaining level or the delete level is determined (that is, a step of determining whether the data belongs to the retaining or delete interval in the figure), if so, the data is inserted into a corresponding level list of the data information table in order, for example, if the data belongs to the retaining level, the data is inserted into a data information list corresponding to the retaining level in the data information table. And if the polled data do not belong to the time interval of the retanning level or the delete level, orderly inserting the data into the list of the data information of the normal level in the data information table (corresponding to the step of orderly inserting the data into the normal list of the data information table in the figure).

In one embodiment, when data uploading is performed, in a case where an upload instruction specifies autonomous driving data to be uploaded, the specified autonomous driving data to be uploaded is acquired from a storage path and uploaded to a server, the upload instruction being generated when a trigger condition for data uploading is met, for example, when a travelable device has a connectable stable wifi or is manually triggered to upload, the upload instruction is generated; under the condition that the uploading instruction does not specify the automatic driving data to be uploaded, the automatic driving data under the storage path are uploaded one by one according to the data level in the data information table and the generation time of the automatic driving data in the corresponding data information according to a preset uploading sequence, wherein when an uploaded automatic driving data is of a specific data level, the data information of the uploaded automatic driving data is deleted from the first list of the data information table, and adding the data information into a second list of the data information table, wherein the first list is a list of data information corresponding to a specific data level in the data information table, the second list is a list of data information corresponding to a priority deletion level in the data information table, the specific data level and the priority deletion level are two different data levels, and the importance of the specific data level is higher than that of the priority deletion level. With the data level HIGH _ RETAINING (HIGH priority reservation); MEDIUM _ RETAINING (MEDIUM priority reservation); LOW _ RETAINING (LOW priority reservation); NORMAL grade; DELETE (priority DELETE level) for example, the specific data level may be any one of a high priority reservation, a medium priority reservation, a low priority reservation, and a normal level. The above levels are listed in order of importance from high to low.

In one embodiment, the preset upload sequence is: and uploading the automatic driving data of different data levels according to the corresponding data level importance from high to low, and uploading the automatic driving data of the same data level according to the sequence of the generation time from front to back, namely, preferentially uploading the data with high data level importance, and preferentially uploading the earlier generated data for the same data level.

The trigger condition for deleting data may be that the disk occupancy rate exceeds a preset ratio, for example, exceeds 90%.

In one embodiment, when data deletion is performed, deleting the automatic driving data under the storage path one by one according to a preset deletion order based on the data level in the data information table and the generation time of the automatic driving data in the corresponding data information, including: and when data deletion is carried out, deleting the automatic driving data under the storage path one by one in a preset deleting sequence under the condition that the data storage space does not meet the preset requirement after the deletion operation at each step corresponds to one time threshold, and deleting the automatic driving data under the storage path one by one according to the data level in the data information table and the generation time in the corresponding data information after the deletion operation at each step is carried out.

The preset requirement that the data storage space needs to meet can be set as required, for example, when the disk occupancy rate is reduced to 50%, the preset requirement is met.

In one embodiment, the preset deletion order is: and deleting the automatic driving data of different data levels according to the sequence of the importance of the corresponding data levels from low to high, and deleting the automatic driving data of the same data level from front to back according to the generation time.

A flow diagram of data deletion according to an embodiment of the present invention is shown in fig. 5. According to fig. 5, all data paths to be deleted and time thresholds are obtained, and automatic deletion is performed in two stages: and deleting according to the time and then according to the data information table. Wherein:

according to the time deletion, all data before the minimum time threshold are deleted layer by layer without distinguishing data levels. Taking an autonomous vehicle as an example, data maintenance of the autonomous vehicle generally has a maintenance period and a time limit, for example, data of a vehicle end (i.e., the autonomous vehicle end) is processed all once every several hours, every day, every week or every interval time, and the mandatory processing time is different according to different requirements of manufacturers or companies. Since a screening or processing has already been performed, these time-out data are not important, and if they are not cleaned up, they are completely discarded, taking up disk space. So that deleting by time is simply deleting expired data by time. For example: the company requires that the data on the day is processed and analyzed every 24 hours, the data before 24 hours are processed by filtering and uploading, the time threshold value when deleting can be set to 72 hours (hours), 36 hours and 24 hours in sequence, each time threshold value is arranged in a descending order, one time threshold value is taken out each time, all the data under the path needing to be deleted are polled, all the data before the time threshold value are deleted, the current disk occupancy rate is checked after deleting is finished, and if the disk occupancy rate meets the exit standard (for example, the disk occupancy rate is reduced to 50%), one-time automatic deleting is finished. If the disk occupancy rate does not meet the quit standard, continuing to take the next time threshold value for deletion and quit judgment (namely judging whether the disk occupancy rate meets the quit standard or not). That is, when the automatic deletion function is triggered, data before 72 hours is deleted first, after the deletion of data before 72 hours is completed, if the disk occupancy rate is reduced to meet the exit criterion, the data is not deleted, if the disk occupancy rate is not met the exit criterion, the data before 36 hours is deleted, and so on, when the data before the minimum time (i.e. the time threshold value of 24 hours) is deleted, that is, the data before 24 hours still does not meet the exit criterion, the data within 24 hours are deleted according to the priority policy of the data information table.

When the disk occupancy rate still does not meet the exit standard after the data is deleted according to the time threshold, the deletion method of deleting according to the data information table is triggered to carry out careful and fine deletion, and because the data in the time limit is not processed, a large amount of lost data is deleted without trade, the data can be selectively protected by deletion according to the level, and the disk state is maintained. Specifically, the deletion may be performed in data levels (for example, in the order of DELETE-NORMAL-LOW _ RETAINING-MEDIUM _ RETAINING-HIGH _ RETAINING), and the deletion may be performed in an ordered data _ info sequence at the same data level, so as to perform the deletion in the order from the front to the back of the generation timing of the automatic driving data, that is, from the old data to the new data. The current disk occupancy rate can be checked at regular time in the deleting process, and if the current disk occupancy rate meets the exit standard, one-time automatic deleting is finished. And if the disk occupancy rate meets the exit standard or the data is completely deleted, exiting the automatic deletion process.

The data processing method provided by the embodiment of the invention is mainly used for data generated by automatic driving, is suitable for an automatic driving scene, and improves the automation degree of data processing of the scene.

Fig. 6 is a main configuration diagram of a data processing apparatus according to an embodiment of the present invention.

As shown in fig. 6, a data processing apparatus 600 according to an embodiment of the present invention mainly includes: a mark monitoring unit 601, an updating unit 602, and an automatic deleting unit 603.

The mark monitoring unit 601 is configured to monitor the automatic driving data generated by the travelable device at each time, and determine a first data level and a first time interval of the monitored automatic driving data each time the automatic driving data is monitored, where the first time interval includes a first time, and the first time is a time when the monitored automatic driving data is generated.

An updating unit 602, configured to store data information of the automatic driving data with a generation time within the first time interval in a data information table in correspondence with the first data level, where the data information includes the generation time and the storage route of the corresponding automatic driving data.

An automatic deleting unit 603 configured to delete the automatic driving data in the storage path one by one according to a preset deleting order based on the data level in the data information table and a generation time of the automatic driving data in the corresponding data information when data deletion is performed, and stop an operation of data deletion until a data storage space of the travelable device meets a preset requirement.

In one embodiment, the tag monitoring unit 601 may be configured to: determining a source module of the monitored automatic driving data, wherein the source module is a functional module in an automatic driving program on the drivable device; and presetting the monitored automatic driving data according to the processing logic of the source module, and determining a first data level of the monitored automatic driving data according to the obtained processing result.

In one embodiment, the tag monitoring unit 601 may be configured to: taking a time interval formed by a first selected time point before the first time and a second selected time point after the first time as a pending time interval; searching whether a target time interval with intersection with the undetermined time interval exists in a level interval table, wherein the level interval table is used for storing each data level and the corresponding time interval; under the condition that the target time interval exists, if the data level corresponding to the target time interval is higher than the first data level, taking the part outside the intersection of the undetermined time interval and the target time interval as the first time interval; if the data level corresponding to the target time interval is lower than the first data level, taking the undetermined time interval as a first time interval; if the data level corresponding to the target time interval is the same as the first data level, taking the union of the target time interval and the undetermined time interval as the first time interval; and taking the pending time interval as a first time interval when the target time interval does not exist.

In one embodiment, the tag monitoring unit 601 may be configured to: and correspondingly storing the first data level and the first time interval in the level interval table.

The updating unit 602 may be configured to: updating the data information table regularly according to the corresponding relation between the data level and the time interval in the level interval table, wherein the updating comprises the following steps: polling each automatic driving data in the storage path, determining a second data level of the polled automatic driving data according to a time interval of the generation time of the polled automatic driving data, and inserting the data information of the polled automatic driving data into a list of data information corresponding to the second data level in the data information table.

In one embodiment, the data processing apparatus 600 further comprises an upload unit for: when data uploading is carried out, acquiring the designated automatic driving data to be uploaded from a storage path and uploading the designated automatic driving data to be uploaded to a server under the condition that an uploading instruction designates the automatic driving data to be uploaded, wherein the uploading instruction is generated when a triggering condition of data uploading is reached; under the condition that the uploading instruction does not specify the automatic driving data to be uploaded, the automatic driving data under the storage path are uploaded one by one according to the data level in the data information table and the generation time of the automatic driving data in the corresponding data information according to a preset uploading sequence, wherein when an uploaded automatic driving data is of a specific data level, the data information of the uploaded automatic driving data is deleted from the first list of the data information table, and adding the data information into a second list of the data information table, wherein the first list is a list of data information corresponding to a specific data level in the data information table, the second list is a list of data information corresponding to a priority deletion level in the data information table, the specific data level and the priority deletion level are two different data levels, and the importance of the specific data level is higher than that of the priority deletion level.

In one embodiment, the preset upload sequence is: and for the automatic driving data of different data levels, uploading the automatic driving data of the same data level from the front to the back according to the generation time.

In one embodiment, the automatic deletion unit 603 is configured to: and when data deletion is carried out, deleting the automatic driving data under the storage path one by one in a preset deleting sequence under the condition that the data storage space does not meet the preset requirement after the deletion operation at each step corresponds to one time threshold, and deleting the automatic driving data under the storage path one by one according to the data level in the data information table and the generation time in the corresponding data information after the deletion operation at each step is carried out.

Fig. 7 is a schematic diagram of the working principle of a data processing device according to an embodiment of the present invention. As shown in fig. 7, the data processing apparatus according to an embodiment of the present invention is located at a travelable device, such as an autonomous vehicle (end), and the data processing apparatus may include a data monitoring module and an execution module, and the data processing apparatus receives autonomous driving data of the autonomous driving module and processes the autonomous driving data through the data processing method according to the embodiment of the present invention. The data monitoring module comprises a mark monitoring unit, an uploading instruction monitoring unit and an automatic deleting monitoring unit, and the execution module comprises an updating unit, an uploading unit and an automatic deleting unit. The functions of the mark monitoring unit, the updating unit, the automatic deleting unit, and the uploading unit in this embodiment are respectively the same as the functions of the mark monitoring unit 601, the updating unit 602, the automatic deleting unit 603, and the uploading unit in the embodiment shown in fig. 6.

The mark monitoring unit is responsible for monitoring the automatic driving data (data for short) of the automatic driving module, judging whether the received data is redundant data, useless data, typical data with research significance and the like according to various processing logics, namely whether the data needs to be reserved or deleted, and distributing a data level (namely a first data level) to the data, wherein the detailed description of how to determine the data level of the monitored automatic driving data is described in detail above, and is not repeated here.

A time interval before and after the data occurs is saved, for example, the time of the data occurrence is pushed forward by 30s as a start time, the time of the data occurrence is pushed backward by 30s as an end time, thereby forming a time interval (the time interval is an undetermined time interval), the finally determined time interval also needs to consider whether a target time interval having an intersection with the undetermined time interval exists in the level interval table, whether a data level corresponding to the target time interval is the same as a data level allocated to the monitored automatic driving data, which level is higher, and the like, and then the finally determined time interval (i.e., the first time interval) is determined in a comprehensive manner, and the method for specifically determining the first time interval is described in detail above and is not repeated here.

The mark monitoring unit is responsible for maintaining the grade interval table, and the obtained time intervals are stored in the grade interval table in order under the data grade corresponding to the data. The mark monitoring unit can also call the updating unit of the execution module at regular time to update the data information table.

Because the automatic driving technology is composed of complex logics of multiple modules (the modules are functional modules of the automatic driving program), each datum can be generated or contained in a trend, each datum abnormity can cause abnormity of other data, meanwhile, other data at a time point can be subjected to passive abnormity, and a certain time interval can provide sufficient operation and processing time for the logic and the data, so that problems are fully exposed. In the daily testing and operation process, typical problem scenes (for example, abnormal positioning data) or smooth perfect scenes (for example, in complex road conditions, a vehicle runs perfectly according to a planned path without abnormality) are likely to occur, if the scenes are to be kept and reproduced, data at a certain moment is not enough to support, and the problem can be solved by keeping the time interval.

The uploading instruction monitoring unit is responsible for monitoring whether an uploading instruction exists or not. When the vehicle end has connectable stable wifi or is triggered to upload manually, an uploading instruction is generated. The upload instruction may specify data to be uploaded, and if not, upload the data in a default order (i.e., a preset upload order). Correspondingly, if the unit monitors an upload command, an upload unit in the execution module is called to upload data.

The automatic deletion monitoring unit is used for monitoring whether the automatic deletion function is triggered, if so, the deletion unit of the execution module is called to delete the data. The automatic deletion monitoring unit monitors the disk occupancy rate in real time and is provided with a deletion function triggering threshold, for example, if the disk occupancy rate exceeds 90%, the deletion function is triggered.

The updating unit is triggered by the mark monitoring unit and is responsible for updating the data information table.

The uploading unit is triggered by the uploading instruction monitoring unit and is responsible for uploading data. The processing method of the uploaded data comprises the following steps: if the upload instruction does not specify the uploaded data, the data are sequentially uploaded according to the sequence in the data information table, and since the data levels of the data information table are ordered (for example, a high-priority retention level-a medium-priority retention level-a low-priority retention level-a normal level-a priority deletion level), the data information data _ info in one data level is also arranged according to the time sequence, so that the data are uploaded from high to low according to the data levels, and the data in the same data level are uploaded from front to back, namely, the data which need to be retained most and have the oldest time are uploaded first. If the data to be uploaded is specified, the data information of the data is searched in the data information table and the data is uploaded, if the data is uploaded successfully, the data is deleted from the data level corresponding to the data in the data information table, and the data _ info (data information) is added to the list of the data information corresponding to the priority deletion level in order (for example, inserted into the forefront). And if the data does not exist or the uploading failure is that the uploading processing result is failure, not performing other processing.

The automatic deleting unit is triggered by the automatic deleting monitoring unit and is responsible for automatically deleting the data according to the sequence in the data information table, wherein the deleting sequence of the data level is from back to front, namely the deleting sequence is as follows: priority deletion level data-normal level data-low priority retention level data-medium priority retention level data-high priority retention level data. Since data _ info (data information) of the same data level is arranged in ascending order of time, the deletion order is from front to back, i.e., the oldest data is preferentially deleted.

The updating unit is triggered by the mark monitoring unit. The data processing flow of the updating unit is as follows: polling data under a specified path (i.e., a storage path of the autonomous driving data); according to the level interval table, if the polled data belongs to a certain time interval of any reservation level or priority deletion level, data _ info (data information) formed by information of the data, such as path, time and the like, is orderly inserted into a list of data information corresponding to a corresponding data level of the data information table; if the polled data does not belong to the time interval, the data _ info of the data is orderly inserted into a list of data information corresponding to a normal level of a data information table. And finishing one updating process until all data are polled. The data processing flow of the updating unit may specifically refer to the above description of the updating flow of the data information table.

The automatic deleting unit is triggered by the automatic deleting monitoring unit, and the data processing flow of the automatic deleting unit may refer to the above description of the data deleting flow, which is not described herein again.

In addition, the detailed implementation of the data processing apparatus in the embodiment of the present invention has been described in detail in the above data processing method, and therefore, the repeated content will not be described again.

Fig. 8 shows an exemplary system architecture 800 of a data processing method or data processing apparatus to which embodiments of the present invention may be applied.

As shown in fig. 8, the system architecture 800 may include

terminal devices

801, 802, 803, a network 804, and a server 805. The network 804 serves to provide a medium for communication links between the

terminal devices

801, 802, 803 and the server 805. Network 804 may include various types of connections, such as wire, wireless communication links, or fiber optic cables, to name a few.

A user may use the

terminal devices

801, 802, 803 to interact with a server 805 over a network 804 to receive or send messages or the like. The

terminal devices

801, 802, 803 may have installed thereon various communication client applications, such as a web browser application, a search-type application, an instant messaging tool, etc. (by way of example only).

The

terminal devices

801, 802, 803 may be various electronic devices having a display screen and supporting web browsing, including but not limited to smart phones, tablet computers, laptop portable computers, desktop computers, and the like.

The server 805 may be a server providing various services, such as a background management server (for example only) providing support for pages browsed by users using the

terminal devices

801, 802, 803. The background management server may store, analyze, and otherwise process the received data such as the automatic driving data, and feed back a processing result (for example, a feedback message of successful storage — just an example) to the terminal device.

It should be noted that the data processing method provided by the embodiment of the present invention is generally executed by the

terminal devices

801, 802, and 803, and accordingly, the data processing apparatus is generally disposed in the

terminal devices

801, 802, and 803.

It should be understood that the number of terminal devices, networks, and servers in fig. 8 is merely illustrative. There may be any number of terminal devices, networks, and servers, as desired for implementation.

Referring now to FIG. 9, shown is a block diagram of a computer system 900 suitable for use in implementing a terminal device of an embodiment of the present application. The terminal device shown in fig. 9 is only an example, and should not bring any limitation to the functions and the scope of use of the embodiments of the present application.

As shown in fig. 9, the computer system 900 includes a Central Processing Unit (CPU)901 that can perform various appropriate actions and processes in accordance with a program stored in a Read Only Memory (ROM)902 or a program loaded from a storage section 908 into a Random Access Memory (RAM) 903. In the RAM 903, various programs and data necessary for the operation of the system 900 are also stored. The CPU 901, ROM 902, and RAM 903 are connected to each other via a bus 904. An input/output (I/O) interface 905 is also connected to bus 904.

The following components are connected to the I/O interface 905: an input portion 906 including a keyboard, a mouse, and the like; an output section 907 including components such as a Cathode Ray Tube (CRT), a Liquid Crystal Display (LCD), and the like, and a speaker; a storage portion 908 including a hard disk and the like; and a communication section 909 including a network interface card such as a LAN card, a modem, or the like. The communication section 909 performs communication processing via a network such as the internet. The drive 910 is also connected to the I/O interface 905 as necessary. A removable medium 911 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is mounted on the drive 910 as necessary, so that a computer program read out therefrom is mounted into the storage section 908 as necessary.

In particular, according to the embodiments of the present disclosure, the processes described above with reference to the flowcharts may be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product comprising a computer program embodied on a computer readable medium, the computer program comprising program code for performing the method illustrated in the flow chart. In such an embodiment, the computer program may be downloaded and installed from a network through the communication section 909, and/or installed from the removable medium 911. The above-described functions defined in the system of the present application are executed when the computer program is executed by a Central Processing Unit (CPU) 901.

It should be noted that the computer readable medium shown in the present invention can be a computer readable signal medium or a computer readable storage medium or any combination of the two. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples of the computer readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the present application, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In this application, however, a computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, fiber optic cable, RF, etc., or any suitable combination of the foregoing.

The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present application. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams or flowchart illustration, and combinations of blocks in the block diagrams or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.

The modules/units described in the embodiments of the present invention may be implemented by software or hardware. The described modules/units may also be provided in a processor, which may be described as: a processor includes a tag monitoring unit, an updating unit, and an automatic deleting unit. Here, the names of these units do not constitute a limitation to the unit itself in some cases, and for example, the updating unit may also be described as "a unit for storing data information for automatic driving data whose generation time is within the first time interval in correspondence with the first data level in the data information table".

As another aspect, the present invention also provides a computer-readable medium that may be contained in the apparatus described in the above embodiments; or may be separate and not incorporated into the device. The computer readable medium carries one or more programs which, when executed by a device, cause the device to comprise: monitoring automatic driving data generated by a travelable device at each moment, and determining a first data level and a first time interval of the monitored automatic driving data after the automatic driving data is monitored, wherein the first time interval comprises a first moment which is the generation moment of the monitored automatic driving data; storing data information of the automatic driving data with the generation time in the first time interval in a data information table in correspondence with the first data level, wherein the data information comprises the generation time and the storage path of the corresponding automatic driving data; and when data deletion is carried out, deleting the automatic driving data under the storage path one by one according to the data level in the data information table and the generation time of the automatic driving data in the corresponding data information according to a preset deletion sequence, and stopping the data deletion operation until the data storage space of the travelable equipment meets a preset requirement.

According to the technical scheme of the embodiment of the invention, the automatic driving data generated by the travelable equipment at each moment is monitored, and the first data level and the first time interval of the monitored automatic driving data are determined, wherein the first time interval comprises the generation moment of the monitored automatic driving data; storing data information of the automatic driving data with the generation time in a first time interval in a data information table in correspondence with a first data level; and when the data deletion is carried out, deleting the automatic driving data under the storage path one by one according to the data level in the data information table and the generation time of the automatic driving data in the corresponding data information according to a preset deletion sequence, and stopping the data deletion until the data storage space of the travelable equipment meets the preset requirement. The method can maintain the states of the data and the disks at regular time in a simple and convenient mode, reduce network requirements and disk pressure generated by redundant data, conditionally reserve or delete the data, be compatible with the disks with different capacities, meet the data processing requirements of an automatic driving scene, and improve the automation degree of data processing in the automatic driving scene.

The above-described embodiments should not be construed as limiting the scope of the invention. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions can occur, depending on design requirements and other factors. Any modification, equivalent replacement, and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A data processing method, comprising:

monitoring automatic driving data generated by a travelable device at each moment, and determining a first data level and a first time interval of the monitored automatic driving data after the automatic driving data is monitored, wherein the first time interval comprises a first moment which is the generation moment of the monitored automatic driving data;

storing data information of the automatic driving data with the generation time in the first time interval in a data information table in correspondence with the first data level, wherein the data information comprises the generation time and the storage path of the corresponding automatic driving data;

and when data deletion is carried out, deleting the automatic driving data under the storage path one by one according to the data level in the data information table and the generation time of the automatic driving data in the corresponding data information according to a preset deletion sequence, and stopping the data deletion operation until the data storage space of the travelable equipment meets a preset requirement.

2. The method of claim 1, wherein determining the first data level of the monitored autonomous driving data comprises:

determining a source module of the monitored autopilot data, the source module being a functional module in an autopilot program on the travelable device;

and presetting the monitored automatic driving data according to the processing logic of the source module, and determining a first data level of the monitored automatic driving data according to an obtained processing result.

3. The method of claim 1 or 2, wherein the step of determining a first time interval of the monitored autonomous driving data comprises:

taking a time interval formed by a first selected time point before the first time and a second selected time point after the first time as a pending time interval;

searching whether a target time interval with intersection with the undetermined time interval exists in a level interval table, wherein the level interval table is used for storing each data level and the corresponding time interval;

under the condition that the target time interval exists, if the data level corresponding to the target time interval is higher than the first data level, taking the part outside the intersection of the undetermined time interval and the target time interval as the first time interval; if the data level corresponding to the target time interval is lower than the first data level, taking the undetermined time interval as the first time interval; if the data level corresponding to the target time interval is the same as the first data level, taking the union of the target time interval and the undetermined time interval as the first time interval;

and taking the undetermined time interval as the first time interval under the condition that the target time interval does not exist.

4. The method of claim 3, wherein the determining after the first data level and the first time interval of the monitored autonomous driving data further comprises: correspondingly storing the first data level and the first time interval in the level interval table;

the method further comprises the following steps:

updating the data information table at regular time according to the corresponding relation between the data level and the time interval in the level interval table, wherein the updating comprises the following steps: polling each piece of automatic driving data under the storage path, determining a second data level of the polled automatic driving data according to a time interval of the generation time of the polled automatic driving data, and inserting the data information of the polled automatic driving data into a list of data information corresponding to the second data level in the data information table.

5. The method of claim 1, further comprising:

when data uploading is carried out, acquiring the designated automatic driving data to be uploaded from the storage path and uploading the designated automatic driving data to the server under the condition that an uploading instruction designates the automatic driving data to be uploaded, wherein the uploading instruction is generated when a triggering condition of the data uploading is reached;

when the uploading instruction does not specify the automatic driving data to be uploaded, uploading the automatic driving data under the storage path one by one according to a data level in the data information table and a generation time of the automatic driving data in the corresponding data information according to a preset uploading sequence, wherein when the uploaded automatic driving data is of a specific data level, the data information of the uploaded automatic driving data is deleted from a first list of the data information table and added to a second list of the data information table, the first list is a list of data information corresponding to the specific data level in the data information table, the second list is a list of data information corresponding to a priority deletion level in the data information table, and the specific data level and the priority deletion level are two different data levels, and the particular data level is of higher importance than the prioritized deletion level.

6. The method of claim 5, wherein the predetermined deletion order is: deleting the automatic driving data of different data levels according to the sequence of the importance of the corresponding data levels from low to high, and deleting the automatic driving data of the same data level according to the sequence of the generation time from front to back;

the preset uploading sequence is as follows: and uploading the automatic driving data of different data levels according to the importance of the corresponding data levels from high to low, and uploading the automatic driving data of the same data level according to the sequence of the generation time from front to back.

7. The method according to claim 1, wherein the deleting the automatic driving data under the storage path one by one according to a preset deletion order based on a data level in the data information table and a generation time of the automatic driving data in the corresponding data information when the data deletion is performed comprises:

and when the data deletion is executed, deleting the automatic driving data in the storage path in steps, wherein the generation time of the automatic driving data is before one or more preset time thresholds, each step of deletion operation corresponds to one time threshold, and after the step of deletion is executed, deleting the automatic driving data in the storage path one by one according to a preset deletion sequence according to the data level in the data information table and the generation time of the corresponding data information under the condition that the data storage space still does not meet the preset requirement.

8. A data processing apparatus, comprising:

the system comprises a mark monitoring unit, a data processing unit and a data processing unit, wherein the mark monitoring unit is used for monitoring automatic driving data generated by a travelable device at each moment, and determining a first data level and a first time interval of the monitored automatic driving data after the automatic driving data is monitored, the first time interval comprises a first moment, and the first moment is the generation moment of the monitored automatic driving data;

the updating unit is used for storing data information of the automatic driving data with the generation time in the first time interval in a data information table in a corresponding mode with the first data level, and the data information comprises the generation time and the storage path of the corresponding automatic driving data;

and the automatic deleting unit is used for deleting the automatic driving data under the storage path one by one according to a preset deleting sequence according to the data level in the data information table and the generation time of the automatic driving data in the corresponding data information when data deletion is carried out, and stopping the data deleting operation until the data storage space of the travelable equipment meets a preset requirement.

9. The apparatus of claim 8, wherein the indicia monitoring unit is further configured to:

10. The apparatus of claim 8 or 9, wherein the indicia monitoring unit is further configured to:

11. The apparatus of claim 10, wherein the indicia monitoring unit is further configured to: correspondingly storing the first data level and the first time interval in the level interval table;

the update unit is further configured to:

12. The apparatus of claim 8, further comprising an upload unit to:

13. The apparatus of claim 8, wherein the automatic deletion unit is further configured to:

14. An electronic device, comprising:

one or more processors;

a memory for storing one or more programs,

the one or more programs, when executed by the one or more processors, cause the one or more processors to implement the method recited in any of claims 1-7.

15. A computer-readable medium, on which a computer program is stored, which, when being executed by a processor, carries out the method according to any one of claims 1-7.