CN115994183A

CN115994183A - Data processing method and device, electronic equipment and storage medium

Info

Publication number: CN115994183A
Application number: CN202310171531.6A
Authority: CN
Inventors: 钱承军; 王宇
Original assignee: FAW Group Corp
Current assignee: FAW Group Corp
Priority date: 2023-02-27
Filing date: 2023-02-27
Publication date: 2023-04-21

Abstract

The embodiment of the invention discloses a data processing method, a data processing device, electronic equipment and a storage medium. The method comprises the following steps: acquiring a configuration file and original data; the configuration file comprises a data tag to be converted; determining data to be converted in the original data based on the data tag to be converted; and converting the data to be converted into target format data. The technical scheme of the embodiment of the invention can realize unified processing of different types of data.

Description

Data processing method and device, electronic equipment and storage medium

Technical Field

The embodiment of the invention relates to the field of data processing, in particular to a data processing method, a data processing device, electronic equipment and a storage medium.

Background

Currently, the processing of data for autopilot is an important part of integrated testing.

However, the number and types of the operating systems and sensors carried by the autopilot are different, so that the types of the data of the autopilot generated by the operating systems and the sensors are different, and the data of the autopilot of various types are difficult to uniformly process at present, so that the data of the autopilot is to be improved.

Disclosure of Invention

The embodiment of the invention provides a data processing method, a device, electronic equipment and a storage medium, so as to realize unified processing of different types of data.

According to an aspect of the present invention, there is provided a data processing method, which may include:

acquiring a configuration file and original data; the configuration file comprises a data tag to be converted;

determining data to be converted in the original data based on the data to be converted label;

and converting the data to be converted into target format data.

According to another aspect of the present invention, there is provided a data processing apparatus, which may include:

the original data acquisition module is used for acquiring the configuration file and the original data; the configuration file comprises a data tag to be converted;

the data to be converted determining module is used for determining data to be converted in the original data based on the data to be converted label;

and the target format data conversion module is used for converting the data to be converted into target format data.

According to another aspect of the present invention, there is provided an electronic device, which may include:

at least one processor; and

a memory communicatively coupled to the at least one processor; wherein,,

The memory stores a computer program executable by the at least one processor, the computer program being executable by the at least one processor to cause the at least one processor to perform the method of processing data provided by any of the embodiments of the present invention.

According to another aspect of the present invention, there is provided a computer readable storage medium having stored thereon computer instructions for causing a processor to perform the method of processing data provided by any of the embodiments of the present invention.

According to the technical scheme, the configuration file and the original data are acquired; the configuration file comprises a data tag to be converted; determining data to be converted in the original data based on the data to be converted label; and converting the data to be converted into target format data. According to the technical scheme, the data to be converted with different data types are determined through the data to be converted label, and the data to be converted is converted into the target format data with the uniform type, so that the uniform processing of the data with different types is realized.

It should be understood that the description in this section is not intended to identify key or critical features of the embodiments of the invention, nor is it intended to be used to limit the scope of the invention. Other features of the present invention will become apparent from the description that follows.

Drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for the description of the embodiments will be briefly described below, and it is apparent that the drawings in the following description are only some embodiments of the present invention, and other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.

FIG. 1 is a flow chart of a method for processing data according to a first embodiment of the present invention;

FIG. 2 is a flow chart of a method for processing data according to a second embodiment of the present invention;

FIG. 3 is a flow chart of a method for processing data according to a third embodiment of the present invention;

FIG. 4 is a flowchart of an alternative example of a data processing method provided in the third embodiment of the present invention;

FIG. 5 is a workflow diagram of a non-merging point cloud module in an alternative example of a data processing method provided in the third embodiment of the present invention;

FIG. 6 is a workflow diagram of a merge point cloud module in an alternative example of a method for processing data provided in embodiment three of the present invention;

FIG. 7 is a workflow diagram of determining a time synchronization relationship in an alternative example of a data processing method provided in the third embodiment of the present invention;

FIG. 8 is a block diagram showing a data processing apparatus according to a fourth embodiment of the present invention;

fig. 9 is a schematic structural diagram of an electronic device implementing a data processing method according to an embodiment of the present invention.

Detailed Description

In order that those skilled in the art will better understand the present invention, a technical solution in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in which it is apparent that the described embodiments are only some embodiments of the present invention, not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the present invention without making any inventive effort, shall fall within the scope of the present invention.

It should be noted that the terms "first," "second," and the like in the description and the claims of the present invention and the above figures are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used may be interchanged where appropriate such that the embodiments of the invention described herein may be implemented in sequences other than those illustrated or otherwise described herein. The cases of "target", "original", etc. are similar and will not be described in detail herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.

Example 1

Fig. 1 is a flowchart of a data processing method according to a first embodiment of the present invention. The present embodiment is applicable to a case of processing data. The method may be performed by a data processing apparatus provided by an embodiment of the present invention, where the apparatus may be implemented by software and/or hardware, and the apparatus may be integrated on an electronic device, where the electronic device may be a variety of user terminals or servers.

Referring to fig. 1, the method of the embodiment of the present invention specifically includes the following steps:

s110, acquiring a configuration file and original data; the configuration file comprises a data tag to be converted.

Wherein a profile may be understood as a file capable of determining the relevant configuration of the processing of the data; the configuration file may be automatically generated according to the processing requirement of the data, may be automatically generated according to a default configuration, or may be manually generated by a staff member, and the generation manner of the configuration file is not particularly limited herein. Raw data may be understood as data comprising data to be converted; the raw data may be, for example, data of a period of time currently acquired, data of a period of time historically acquired, or externally imported data. In the field of autopilot, the raw data may be point cloud data collected during autopilot, for example, may be data collected by the same or different sensors, and the sensors may be, for example, radars or cameras of the same or different models, etc.; the data format of the original data may be, for example, a file format of a bag (. Bag) recorded by a plurality of lidars based on Robot Operating System (ROS), and the data format of the original data is not particularly limited in the embodiment of the present invention. The data tag to be converted may be understood as a tag corresponding to data to be converted, which requires processing of the data.

It will be appreciated that a raw data may include a plurality of data queues or topics (topics) from different sources, for example, the raw data may be acquired by a plurality of lidars, the data acquired by a lidar may generate a topic in the raw data, and the data to be converted may be understood as a topic in the raw data that requires the data.

In an embodiment of the present invention, there may be a case where a plurality of original files need to be processed at the same time, and in this case, the configuration file may further include an original data tag, so as to determine at least one original file according to the original data tag.

S120, determining the data to be converted in the original data based on the data to be converted label.

It will be appreciated that there may be instances where the raw data may be data acquired by different types or models of sensors over a period of time, for example, resulting in the possibility that there may be acquired data acquired by a plurality of different sensors within one raw data, however, in embodiments of the present invention, it may not be necessary to process all of the acquired data in the raw data. In order to determine the data to be converted which needs to be processed, corresponding tags can be set for all collected data stored in the original data in advance, and the data to be converted can be configured in a configuration file under the condition of data processing according to requirements, so that the data to be converted in the original data can be determined based on the data to be converted.

In the embodiment of the present invention, if the number of the original data is more than one, the configuration file may further include a data tag to be converted corresponding to each original data, so as to determine at least one data to be converted from at least one original data according to each data tag to be converted.

S130, converting the data to be converted into target format data.

The target format data may be understood as data corresponding to the target format into which the format of the data to be converted is required to be converted, and the target format data may be, for example, data in ". Pcd" format. The name of the target format data can be determined according to at least one of the storage address of the original data, the name of the storage catalog of the original data, the naming parameter in the configuration file, the name of the original data, the acquisition time of the data to be converted and the name of the data to be converted, and can also be user-defined setting.

It will be appreciated that there may be data to be converted derived from data collected by different types or models of sensors, resulting in different data to be converted having variability; there may be cases where different companies use different data recording formats, for example, a company records data using the ". Record" format, a company records data using the ". Bag" format recorded by ROS, and a company records data using the ". Pcd" format of the point cloud library (point cloud library, PCL). Under the condition that data are required to be analyzed, particularly under the condition that software which can be suitable for various different types of data is required to be adopted for data analysis, the data to be converted can be subjected to unified processing and are uniformly converted into target format data, so that unified processing of the different types of data is realized, the data are conveniently analyzed, and the software which can be subjected to data testing can be conveniently tested and optimized.

According to the technical scheme, the data processing log information can be output after the data is processed, so that a user can monitor the processing condition of the data and find and process the data in time after the processing error of the data.

It should be noted that in the embodiment of the present invention, the number of fields and the data type of the target format data may be determined according to the number of fields and the data type of the data to be converted, for example, the number of fields of the data to be converted is 8, the data type is a floating point type, and then the number of fields of the data to be converted is 8, and the data type is a floating point type; or, according to the field number parameter or the type parameter in the configuration file, determining the field number and the data type of the target format data, for example, 8 fields of the data to be converted, the data type is a floating point type, 6 fields in the configuration file and 6 types of the data type parameters are integer, and then determining the field number of the target format data to be 6 and the data type to be integer; or, according to the format type of the target format data, determining the number of fields and the data type of the target format data, for example, the number of fields of the data to be converted is 8, the data type is a floating point type, and according to the format type, determining the number of fields of the target format data to be required to be 6, and the type parameter is an integer, then determining the number of fields of the target format data to be 6, the data type is an integer, and each field type can be independently set.

An optional technical scheme, the configuration file further comprises field parameters; the data processing method further comprises the following steps: and determining the attribute field of the target format data according to the attribute information and the field parameters of the original data.

The attribute information may be information related to an attribute of the original data, for example, may be a data acquisition source of the original data or a time period corresponding to the original data acquisition data; if the original data includes at least one point cloud data, the attribute information may also be point cloud coordinate position information, point intensity information, timestamp information corresponding to the point cloud acquisition time, and/or a serial number of a sensor of the point cloud source, etc. of the point cloud data. The field parameter may be understood as a parameter indicating the content of an attribute field of the target format data. The attribute field may be understood as an attribute related field of the target format data, for example, may be a data acquisition source field of the original data corresponding to the target format data, or a field of a time period corresponding to the original data acquisition data, etc.; if the target format data includes at least one target point cloud data, the attribute field may also be a point cloud coordinate position information field, a point intensity information field, a timestamp information field corresponding to a point cloud acquisition time, and/or a serial number field of a sensor of the point cloud source of the target point cloud data.

It can be understood that, because the sensors of the acquisition sources of the original data or the sensors of the acquisition sources of the acquisition data in the original data are different, in order to ensure the completeness, normalization and traceability of the target format data, the condition that the attribute information of the target format data is not normalized is prevented, and the attribute field of the target format data can be determined according to the attribute information and the field parameters of the original data.

In the embodiment of the invention, the condition that partial attribute fields cannot be determined due to imperfect attribute information of the original data may exist, and the attribute fields may be perfected or complemented, for example, the attribute fields may be perfected or complemented by information such as a data source of the original data, a storage address of the original data, a name of a storage directory of the original data, and/or a name of the original data.

For example, if the raw data is data including at least one point cloud data collected by a plurality of radars, the target format data includes at least one target point cloud data, the field parameters include point cloud coordinate position information (x, y, z) of the target point cloud data, point intensity information (intensity), timestamp information (timestamp) corresponding to a point cloud collection time, a serial number (lidar_id) corresponding to a radar of the point cloud source, a row (row) and a column (column) where the target point cloud data is located in the target format data, and the attribute field of each of the at least one target point cloud data may be x y z intensity timestamp lidar _id row column. The field parameter may also be the number of fields, and the field parameter and its corresponding attribute field may be as shown in table 1 below:

Table 1 attribute field of target point cloud data

According to the technical scheme provided by the embodiment of the invention, the attribute field of the target format data is determined according to the attribute information and the field parameters of the original data, so that the completeness, normalization and traceability of the target format data can be ensured, and the standardization of the attribute information of the target format data can be ensured.

In another alternative technical solution, the data processing method further includes: determining a target save address according to at least one of: the method comprises the steps of storing an address of original data, a name of a storage catalog of the original data, a storage address parameter in a configuration file, a name of the original data and a name of data to be converted; after converting the data to be converted into the target format data, the data processing method further comprises the following steps: and saving the target format data into a target saving address.

Where a memory address is understood to be the address of the original data memory. The name of the storage directory may be understood as the name of the directory in which the original data is stored. The save address parameter may be understood as a parameter in the configuration file indicating the target save address. The name of the raw data may be, for example, a data acquisition period of the raw data or a sensor of a data source, or the like. The name of the data to be converted may be, for example, the name of the sensor from which the data to be converted originates or the time period in which the data to be converted is acquired, etc.

It should be noted that, there may be a case where there are a plurality of target format data due to the presence of a plurality of data to be converted, so that the plurality of target format data may be stored in the same target storage address, and the target storage addresses corresponding to the data to be converted respectively may be determined according to at least one of the following: the storage address of the original data, the name of the storage directory of the original data, the save address parameter in the configuration file, the name of the original data and the name of the data to be converted.

In the embodiment of the invention, the target format data is required to be saved in consideration of the fact that the data is processed, so that the target format data is convenient to use in the follow-up process, and therefore, the target saving address can be determined according to at least one of the storage address of the original data, the name of the storage catalog of the original data, the saving address parameter in the configuration file, the name of the original data and the name of the data to be converted, so that the target format data is saved in the target saving address, and the target saving address is the address required to be saved by the target format data. The method and the device can save the target format data, and facilitate the subsequent quick search of the target format data during use.

Example two

Fig. 2 is a flowchart of another data processing method provided in the second embodiment of the present invention. The present embodiment is optimized based on the above technical solutions. In this embodiment, optionally, the configuration file further includes a combination parameter, and the number of data to be converted is at least two; after converting the data to be converted into the target format data, the data processing method further comprises the following steps: under the condition that the merging parameters are merging, determining the time synchronization relation of each data to be converted; and merging the target format data corresponding to each data to be converted based on the time synchronization relationship. Wherein, the explanation of the same or corresponding terms as the above embodiments is not repeated herein.

Referring to fig. 2, the method of this embodiment may specifically include the following steps:

s210, acquiring a configuration file and original data; the configuration file comprises a data tag to be converted and merging parameters.

The merging parameter may be understood as a parameter indicating whether to merge the target format data; the merge parameter may be, for example, merge or not merge.

S220, determining data to be converted in the original data based on the data to be converted labels, wherein the number of the data to be converted is at least two.

It can be understood that, in the case that the number of data to be converted is one, only one target format data is obtained, without considering whether to merge or not; in the case where the number of data to be converted is at least two, it is required to consider whether or not to combine the acquired at least two target format data.

S230, converting the data to be converted into target format data.

And S240, determining the time synchronization relation of each data to be converted under the condition that the merging parameters are merging.

It can be understood that, since each data to be converted may be data collected in a period of time, the collection time of the data to be converted is not necessarily in a completely corresponding relationship, for example, the collection time of the data to be converted a is 2022, 1, 10, to 2022, 1, 11, and the collection time of the data to be converted B is 2022, 1, 10, 30 minutes to 2022, 1, 11, 30 minutes, and this is the case; for example, the acquisition time interval of the data to be converted C and the data to be converted D is required to be 10 seconds, but the acquisition time of the sensor corresponding to the data to be converted C is 0.05 seconds later than the acquisition time of the sensor corresponding to the data to be converted D due to the difference of the sensors of the data to be converted C and the data to be converted D. In order to ensure that the merged data can correspond in time so as to ensure the merging effect, the time synchronization relation of each piece of data to be converted can be determined, wherein the time synchronization relation is the synchronization corresponding relation of each piece of data to be converted in time.

S250, merging the target format data corresponding to each data to be converted based on the time synchronization relationship.

In the embodiment of the invention, the target format data corresponding to each data to be converted can be combined correspondingly in time based on the time synchronization relation.

It can be understood that the manner of merging the target format data may be determined according to the processing requirement of the data, for example, in the case of determining the global point cloud of the vehicle at the current time point, the target format data may be merged; for another example, in the case where the overall characteristics of the target format data need to be determined, the target format data may be fused and combined.

According to the technical scheme, the configuration file further comprises merging parameters, and the number of data to be converted is at least two; after converting the data to be converted into the target format data, determining the time synchronization relation of each data to be converted under the condition that the merging parameters are merging; and merging the target format data corresponding to each data to be converted based on the time synchronization relationship. According to the technical scheme, under the condition that a plurality of data to be converted exist, data can be processed in batches rapidly and efficiently, under the condition that a plurality of target format data are selected to be combined and processed, multi-data cooperation can be realized, the data can be conveniently used from the whole hierarchy in the follow-up process, particularly under the condition that the sensor is tested according to the target format data, the cooperation test of the plurality of sensors can be effectively ensured, the test adaptability is strong, and the test efficiency is high.

Example III

Fig. 3 is a flowchart of another data processing method provided in the third embodiment of the present invention. The present embodiment is optimized based on the above technical solutions. In this embodiment, optionally, each piece of data to be converted includes point cloud data to be converted acquired according to a time sequence, and determining a time synchronization relationship of each piece of data to be converted includes: determining a time synchronization relationship among the point cloud data to be converted according to the acquisition time of the point cloud data to be converted; based on the time synchronization relationship, merging the target format data corresponding to each data to be converted, including: and merging the target format data corresponding to the point cloud data to be converted with the time synchronization relationship. Wherein, the explanation of the same or corresponding terms as the above embodiments is not repeated herein.

Referring to fig. 3, the method of this embodiment may specifically include the following steps:

s310, acquiring a configuration file and original data; the configuration file comprises a data tag to be converted and merging parameters.

S320, determining to-be-converted data in the original data based on to-be-converted data labels, wherein the number of to-be-converted data is at least two, and each to-be-converted data comprises to-be-converted point cloud data acquired according to time sequence.

The point cloud data to be converted may be understood as point cloud data which is required to be processed and included in the data to be converted.

It can be understood that, in the embodiment of the present invention, the data to be converted may include at least one point cloud data to be converted, and since one data to be converted may correspond to data collected by one sensor, the sensor typically collects data according to a time sequence, and thus each data to be converted includes the point cloud data to be converted collected according to the time sequence.

S330, converting the data to be converted into target format data.

And S340, under the condition that the merging parameters are merging, determining the time synchronization relationship among the point cloud data to be converted according to the acquisition time of the point cloud data to be converted.

The time of collecting the point cloud data to be converted can be understood as a time point when the sensor collects the point cloud data to be converted.

It can be understood that according to the collection time of each point cloud data to be converted, the time synchronization relationship between each point cloud data to be converted can be determined, for example, the collection time of the point cloud data a to be converted is 2022, 1 month, 1 day, 10 hours, 1 minute, 10 seconds, the collection time of the point cloud data b to be converted is 2022, 1 month, 1 day, 10 hours, 1 minute, 10 seconds, the collection time of the point cloud data c to be converted is 2022, 1 month, 1 hour, 1 minute, 10 seconds, and the point cloud data a to be converted, the point cloud data b to be converted, and the point cloud data c to be converted are determined to be collected at the same time, so that the point cloud data a to be converted, the point cloud data b to be converted, and the point cloud data c to be converted can be determined to have the time synchronization relationship; for example, the collection time of the point cloud data d to be converted is 2022, 1 month, 1 day, 10 hours, 20 minutes, 10 seconds, the collection time of the point cloud data e to be converted is 2022, 1 month, 1 day, 10 hours, 20 minutes, 20 seconds, the collection time of the point cloud data f to be converted is 2022, 1 month, 1 day, 10 hours, 20 minutes, 20 seconds, 20 minutes, 19 seconds, the collection time of the point cloud data g to be converted is 2022, 1 month, 1 day, 10 hours, the collection time of the point cloud data d to be converted and the collection time of the point cloud data e to be converted are very close, and therefore the point cloud data f to be converted and the point cloud data e to be converted can be determined to have a time synchronization relationship.

For example, determining the time synchronization relationship between the point cloud data to be converted may be to record all the time stamps corresponding to the acquisition time of the point cloud data to be converted and perform time classification, for example, classifying the time stamps into at least one time period range, and determining the time synchronization relationship between the point cloud data to be converted according to the classification result.

And S350, merging the target format data corresponding to the point cloud data to be converted with the time synchronization relationship.

It can be understood that the target format data corresponding to the point cloud data to be converted with the time synchronization relationship can be synchronously corresponding in time, and the target format data corresponding to the point cloud data to be converted with the time synchronization relationship can be combined, so that the problem of disorder of the combined data on the time level caused by combining the target format data which does not belong to the corresponding time sequence when the target format data is combined is avoided.

In the embodiment of the invention, the target format point cloud data in the target format data corresponding to the point cloud data to be converted with the time synchronization relationship can be combined into one frame of data.

According to the technical scheme, the time synchronization relationship among the point cloud data to be converted is determined according to the acquisition time of the point cloud data to be converted; and merging the target format data corresponding to the point cloud data to be converted with the time synchronization relationship. According to the technical scheme, the combined data can have time relevance, the data cooperation in time sequence is realized, the data is arranged according to the time sequence, so that the combined data can be scheduled according to the time stamp, and the data can be used later conveniently.

An optional technical scheme, according to the acquisition time of each point cloud data to be converted, determines a time synchronization relationship between each point cloud data to be converted, including: initializing starting point cloud data; acquiring the point cloud data to be converted according to the acquisition time of the point cloud data to be converted and the time sequence; the method comprises the steps that point cloud data to be converted, of which the absolute value of a difference value between acquisition time and acquisition time of starting point cloud data is smaller than or equal to the acquisition time difference, are used as point cloud data to be converted, wherein the point cloud data to be converted has a time synchronization relationship with the starting point cloud data; and taking the first point cloud data to be converted which does not have a time synchronization relationship with the starting point cloud data as new starting point cloud data until the time synchronization relationship among the point cloud data to be converted is determined.

The start point cloud data may be understood as first point cloud data to be converted in time sequence among point cloud data to be converted having a time synchronization relationship.

In the embodiment of the invention, when the first starting point cloud data is required to be determined, the first starting point cloud data needs to be initialized according to the difference value between the acquisition time of the point cloud data to be converted and the acquisition time of the starting point cloud data. The initialization of the starting point cloud data may be to set the acquisition time of the starting point cloud data to a preset time value, or may be determined according to the acquisition time period of each data to be converted.

In the embodiment of the invention, the data of each point cloud to be converted can be acquired according to the acquisition time of the data of each point cloud to be converted according to the time sequence, for example, a unified time axis can be established through a time synchronizer, the time stamp corresponding to the acquisition time of the data of each point cloud to be converted corresponds to the time axis, and the data of each point cloud to be converted is sequentially acquired according to the time sequence of the time stamp of the data of each point cloud to be converted on the time axis.

It should be noted that, because the sensors for collecting the point cloud data to be converted may be different types or models of sensors, the point cloud data to be converted having a time synchronization relationship may not be completely corresponding in time due to the variability of the sensors, and there may be a certain error, for example, each sensor collects data once every 10 seconds, the conversion point cloud data h and the conversion point cloud data i should theoretically be data collected at the same time point, but the collection time of the point cloud data h to be converted is 0.01 seconds slower than the collection time of the point cloud data i to be converted due to the individual differences of the sensors, in this case, the conversion point cloud data h and the conversion point cloud data i still have a time synchronization relationship. In order to ensure that the time synchronization relationship between the point cloud data to be converted can be determined under the above conditions, an acquisition time difference can be preset, wherein the acquisition time difference is the maximum error value allowable by the acquisition time of the point cloud data to be converted and the starting point cloud data with the time synchronization relationship; comparing the acquisition time of the currently acquired point cloud data to be converted with the acquisition time of the starting point cloud data, and under the condition that the absolute value of the difference value between the acquisition time of the currently acquired point cloud data to be converted and the acquisition time of the starting point cloud data is smaller than or equal to the acquisition time difference, namely, the fact that the currently acquired point cloud data to be converted is smaller in acquisition time difference value with the starting point cloud data is indicated, and the currently acquired point cloud data to be converted can be used as the point cloud data to be converted which has time synchronization relation with the starting point cloud data.

It can be understood that, when the first point cloud data to be converted does not have a time synchronization relationship with the starting point cloud data, that is, the point cloud data to be converted which does not have a time synchronization relationship with the starting point cloud data exists in the point cloud data to be converted, the first point cloud data to be converted which does not have a time synchronization relationship with the starting point cloud data can be used as new starting point cloud data, so that the point cloud data to be converted which is acquired subsequently can be compared with the new starting point cloud data, and the point cloud data to be converted which has a time synchronization relationship with the new starting point cloud data can be determined until the time synchronization relationship between the point cloud data to be converted is determined.

According to the technical scheme, the starting point cloud data are initialized; acquiring the point cloud data to be converted according to the acquisition time of the point cloud data to be converted and the time sequence; the method comprises the steps that point cloud data to be converted, of which the absolute value of a difference value between acquisition time and acquisition time of starting point cloud data is smaller than or equal to the acquisition time difference, are used as point cloud data to be converted, wherein the point cloud data to be converted has a time synchronization relationship with the starting point cloud data; and taking the first point cloud data to be converted which does not have a time synchronization relationship with the starting point cloud data as new starting point cloud data until the time synchronization relationship among the point cloud data to be converted is determined. The method can determine the time synchronization relationship among the point cloud data to be converted under the condition that the acquisition time of acquiring the point cloud data to be converted exists due to the difference of sensors for acquiring the data, and can be better applied to practice.

On the basis of the above scheme, another optional technical scheme, the determining step of the point cloud data to be converted, which has a time synchronization relationship with the start point cloud data, further includes: and if the absolute value of the difference value between the acquisition time of the currently acquired point cloud data to be converted and the acquisition time of the starting point cloud data is smaller than or equal to the acquisition time difference, and the number of the point cloud data to be converted which has a time synchronization relationship with the starting point cloud data does not reach the preset number, taking the currently acquired point cloud data to be converted as the point cloud data to be converted which has a time synchronization relationship with the starting point cloud data.

The preset number may be understood as a preset maximum number of point cloud data to be converted having a time synchronization relationship.

It is understood that the preset number may be a preset fixed number; the preset number may also be determined by the number of data to be converted, for example, the number of data to be converted is 2, and then the two data to be converted may originate from two different sensors, so the number of point cloud data to be converted, which are acquired by the two different sensors and have a time synchronization relationship, can only be two at maximum, and thus the preset number may be 2.

In the embodiment of the invention, if the absolute value of the difference between the acquisition time of the currently acquired point cloud data to be converted and the acquisition time of the starting point cloud data is smaller than or equal to the acquisition time difference, and the number of the point cloud data to be converted, which has a time synchronization relationship with the starting point cloud data, does not reach the preset number, the currently acquired point cloud data to be converted is used as the point cloud data to be converted, which has a time synchronization relationship with the starting point cloud data; even if the absolute value of the difference between the acquisition time of the currently acquired point cloud data to be converted and the acquisition time of the starting point cloud data is smaller than or equal to the acquisition time difference, the number of the point cloud data to be converted which has a time synchronization relationship with the starting point cloud data already reaches a preset number, that is, the fact that the currently acquired point cloud data to be converted does not have a time synchronization relationship with the starting point cloud data is indicated, and the currently acquired point cloud data to be converted can be used as new starting point cloud data.

According to the technical scheme, if the absolute value of the difference value between the acquisition time of the currently acquired point cloud data to be converted and the acquisition time of the starting point cloud data is smaller than or equal to the acquisition time difference, and the number of the point cloud data to be converted, which has a time synchronization relationship with the starting point cloud data, does not reach the preset number, the currently acquired point cloud data to be converted is used as the point cloud data to be converted, which has a time synchronization relationship with the starting point cloud data. The condition that the point cloud data to be converted which does not have a time synchronization relationship with the starting point cloud data due to the reasons of errors of acquisition time or the difference of sensors and the like can be avoided, for example, the condition that the point cloud data to be converted which has a time synchronization relationship with the starting point cloud data is used as the point cloud data to be converted can be avoided, and in the condition that the preset number is 2, after two different sensors acquire the point cloud data j to be converted and the point cloud data k to be converted, one sensor fails to acquire the point cloud data l to be converted in advance, and the point cloud data l to be converted, the point cloud data j to be converted and the point cloud data k to be converted do not have a time synchronization relationship, but due to the fact that the point cloud data l to be converted is acquired in advance, the point cloud data l to be converted may be mistakenly used as the point cloud data to be converted which has a time synchronization relationship with the point cloud data j to be converted and the point cloud data k to be converted. Therefore, the accuracy of determining the time synchronization relationship between the point cloud data to be converted is improved.

Based on the above scheme, according to another alternative technical scheme, the acquisition time difference is obtained by the following steps: acquiring acquisition frequency corresponding to the original data, and determining an acquisition time interval according to the acquisition frequency; and obtaining the acquisition time difference based on the acquisition time interval and a preset time interval algorithm.

The acquisition frequency is understood to be the frequency at which the raw data is acquired. The acquisition time interval may be understood as a data interval in which raw data is acquired. The preset time interval algorithm may be understood as a preset algorithm for determining the acquisition time difference according to the acquisition time interval.

In the embodiment of the invention, the acquisition frequency corresponding to the original data can be acquired, the acquisition time interval is determined according to the acquisition frequency, and the acquisition time difference is obtained based on the acquisition time interval and a preset time interval algorithm, so that the acquisition time difference can be automatically adjusted according to the acquisition frequency, and the acquisition time difference can be compatible with the original data in different conditions.

For better understanding of the technical solution of the embodiment of the present invention described above, an alternative example is provided herein. Fig. 4 is a flowchart of an alternative example of a data processing method provided in the third embodiment of the present invention, and exemplarily, referring to fig. 4, an ROS bag file in a ". Bag" file format recorded by ROS based on multiple radars is required to be converted into target format data in a ". Pcd" format after data processing, where the ROS bag file includes at least one topic, and each topic is acquired by a corresponding radar. In particular, the method comprises the steps of,

Step one: judging whether the configuration file can be acquired or not, and if the configuration file cannot be read, directly exiting; if the configuration file can be acquired, the configuration file is acquired, and relevant parameters of the processing data and the ROS are initialized according to the configuration file.

Step two: the configuration file comprises a storage address of original data, a folder corresponding to the storage address can be obtained, the folder is traversed, and all ROS (reactive oxygen species) bag files requiring data processing are obtained; judging whether ROS bag files requiring data processing exist in the folder, and if not, directly exiting.

Step three: and determining a topic for processing the data according to the data label to be converted in the configuration file.

Step four: judging whether to need to merge the multi-radar data at the same moment according to the merging parameters in the configuration file, if so, calling a merging point cloud module to read the topic of each need to be merged, converting the topic of each need to be merged into target format data in a 'pcd' format, merging the target format data corresponding to the point cloud data to be converted with a time synchronization relationship, and storing the merged target format data; and if the merging is not required, calling a non-merging point cloud module to acquire each topic, converting each topic into target format data in a 'pcd' format, and storing the target format data.

Step five: judging whether the data is successfully stored or not, and if not, recording an error log; if so, judging whether an unprocessed ROS bag file exists in the storage address, if so, returning to the step four, and if not, executing the step six.

Step six: and outputting the summary log and exiting.

Fig. 5 is a workflow diagram of a non-merging point cloud module in an alternative example of a data processing method provided in the third embodiment of the present invention. The workflow of the non-merging point cloud module in the above step four may be referred to as fig. 5, specifically,

step 4.11: judging whether the conversion parameters used for indicating the conversion of the data format in the configuration file are complete or not, if not, directly exiting, and if so, executing the step 4.12.

Step 4.12: and determining the target storage address of each topic for storing the corresponding target format data according to the name of the storage catalog of the ROS bag file, the storage address parameter in the configuration file or the name of each topic requiring data processing.

Step 4.13: each topic was acquired.

Step 4.14: judging whether the data tag of each topic is the same as the data tag to be converted in the configuration file, if so, acquiring the point cloud coordinate position information, intensity, topic name, time stamp information, lidar_id and other attribute information of the topic corresponding to each topic respectively; and if the data labels are different, determining the topic which requires data processing according to the data labels to be converted in the configuration file, and returning to the step 4.13.

Step 4.15: sequentially reading the topic, judging whether the point cloud data to be converted in the currently read topic is empty or not, and re-executing the step 4.15 if the point cloud data to be converted in the currently read topic is empty; if not, the step 4.16 is executed.

Step 4.16: and converting the point cloud data to be converted in the topic into target format point cloud data in the ". Pcd" format.

Step 4.17: determining whether attribute fields of the point cloud data in each target format contain intensity, timestamp information corresponding to the point cloud acquisition time and lidar_id according to field parameters in the configuration file, and supplementing the attribute fields of the point cloud data in the target format, which do not contain the attribute fields, according to the attribute information.

Step 4.18: and (4) storing target format data formed by the target format point cloud data into a target storage address, and returning to the step 4.15 until all topics are read.

Fig. 6 is a flowchart of a merging point cloud module in an alternative example of a data processing method provided in the third embodiment of the present invention. The workflow of the merging point cloud module in the above step four may be referred to fig. 6, specifically,

step 4.21: judging whether the conversion parameters used for indicating the conversion of the data format in the configuration file are complete or not, if not, directly exiting, and if so, executing the step 4.22.

Step 4.22: determining a time stamp corresponding to the acquisition time of the point cloud data to be converted in each topic; and determining the target storage address of the combined data according to the name of the storage catalog of the ROS bag file, the storage address parameter in the configuration file or the name of the topic for processing the data according to each requirement.

Step 4.23: each topic was acquired.

Step 4.24: judging whether the data tag of each topic is the same as the data tag to be converted in the configuration file, if so, acquiring the point cloud coordinate position information, intensity, topic name, time stamp information, lidar_id and other attribute information of the topic corresponding to each topic respectively; and if the data labels are different, determining the topic which requires data processing according to the data labels to be converted in the configuration file, and returning to the step 4.23.

Step 4.25: sequentially reading each topic, judging whether the point cloud data to be converted in the current topic is empty or not, and re-executing the step 4.25 if the point cloud data to be converted in the current topic is empty; if not, step 4.26 is performed.

Step 4.26: and converting the point cloud data to be converted in the topic into target format point cloud data in the ". Pcd" format.

Step 4.27: determining whether attribute fields of the point cloud data in each target format contain intensity, timestamp information corresponding to the point cloud acquisition time and lidar_id according to field parameters in the configuration file, and supplementing the attribute fields of the point cloud data in the target format, which do not contain the attribute fields, according to the attribute information.

Step 4.27: returning to step 4.25 until all topics are read.

Step 4.28: and determining a time synchronization relation between the point cloud data to be converted according to the time stamp corresponding to the acquisition time of the point cloud data to be converted in each topic, and merging the target format point cloud data corresponding to the point cloud data to be converted with the time synchronization relation.

Step 4.28: and storing the integrated data formed by the combined data into a target storage address.

Fig. 7 is a flowchart of a process for determining a time synchronization relationship in an alternative example of a data processing method provided in the third embodiment of the present invention. The specific workflow of determining the time synchronization relationship between the point cloud data to be converted according to the time stamp corresponding to the acquisition time of the point cloud data to be converted in each topic in the above step 4.28 may be referred to fig. 7, in particular,

step 4.2801: the number num_topic, which determines the number of topics, may be used to reflect the maximum number of target format point cloud data that each frame needs to merge.

Step 4.2802: and determining the position address of each frame of combined point cloud in the integrated data formed after combination according to the point cloud coordinate position information, intensity, topic name, timestamp information and/or lidar_id and other attribute information corresponding to the point cloud data to be converted respectively.

Step 4.2803: and determining an acquisition time interval according to the acquisition frequency f of the ROS bag file.

Step 4.2804: corresponding time stamps corresponding to the acquisition time of the point cloud data to be converted on a time axis, and establishing a current frame point cloud array array_pc_current and a history point cloud array array_pc_history; and sequentially acquiring the point cloud data to be converted according to the time sequence of the time stamp of the point cloud data to be converted on a time axis.

Step 4.2804: judging whether the timestamp corresponding to the currently acquired point cloud data to be converted is the first moment in the current frame, if so, setting a frame start flag bit (first_time_of_frame) as true, and entering a step 4.2805; if not, go to step 4.2806; judging whether the timestamp corresponding to the currently acquired point cloud data to be converted is the first moment in the current frame, for example, judging whether the absolute value of the time difference between the timestamp corresponding to the currently acquired point cloud data to be converted and the timestamp of the current frame is smaller than or equal to interval/2.0, if so, the currently acquired point cloud data to be converted is not the first moment in the current frame, and if not, the currently acquired point cloud data to be converted is the first moment in the current frame.

Step 4.2805: and taking the timestamp corresponding to the currently acquired point cloud data to be converted as the current frame timestamp, and setting a frame start flag bit (first_time_of_frame) as false.

Step 4.2806: judging whether the absolute value of the time difference between the time stamp corresponding to the currently acquired point cloud data to be converted and the time stamp of the current frame is smaller than or equal to interval/2.0, if so, entering a step 4.2807; if not, step 4.2812 is entered.

Step 4.2807: and storing the currently acquired point cloud data to be converted into array_pc_current.

Step 4.2808: judging whether the number of the point cloud data to be converted in the array_pc_current is equal to num_topic, if so, entering a step 4.2809; if not, the method proceeds to step 4.2804 to acquire the point cloud data to be converted in sequence.

Step 4.2809: judging whether point cloud data to be converted exist in the array_pc_history, if not, merging target format point cloud data corresponding to the point cloud data to be converted in the array_pc_current into one frame of data, and entering a step 4.2811; if so, step 4.2810 is entered.

Step 4.2810: and merging target format point cloud data corresponding to point cloud data to be converted in the array_pc_history into one frame of data, and recording that the frame has frame loss under the condition that the number of the point cloud data to be converted in the array_pc_history is not equal to num_topic. If step 4.2810 is from step 4.2809, step 4.2811 is entered and if the current step is from step 4.2815, step 4.2813 is entered.

Step 4.2811: first_time_of_frame is set to true, and step 4.2816 is entered.

Step 4.2812: judging whether the array of array_pc_history is empty, if so, entering a step 4.2813; if not, step 4.2814 is entered.

Step 4.2813: and moving the point cloud data to be converted in the array_pc_current into the array_pc_history, taking the current frame time stamp as a historical frame time stamp, storing the currently acquired point cloud data to be converted into the array_pc_current, taking the time stamp corresponding to the currently acquired point cloud data to be converted as the current frame time stamp, and entering a step 4.2816.

Step 4.2814: and sequentially judging whether the time difference between the time stamp corresponding to the point cloud data to be converted in the array_pc_current and the historical frame time stamp in the array_pc_history is smaller than the interval. If so, go to step 4.2815; if not, step 4.2810 is entered.

Step 4.2815: and moving the point cloud data to be converted in the array_pc_current with the time difference smaller than the interval into the array_pc_history, and changing the determination condition of the time synchronization relationship to avoid the existence of the point cloud data to be converted in the array_pc_current, which has the time synchronization relationship with the point cloud data to be converted in the array_pc_history. Step 4.2810 is entered.

Step 4.2816: judging whether the point cloud data to be converted is not acquired, if yes, entering a step 4.2804 to acquire the point cloud data to be converted in sequence; if not, directly exiting.

Example IV

Fig. 8 is a block diagram of a data processing apparatus according to a fourth embodiment of the present invention, which is configured to execute the data processing method according to any of the above embodiments. The device and the data processing method of each embodiment belong to the same invention conception, and the details of the device for processing data are not described in detail in the embodiments of the device for processing data, and reference may be made to the embodiments of the method for processing data. Referring to fig. 8, the apparatus may specifically include: a raw data acquisition module 410, a data to be converted determination module 420, and a target format data conversion module 430.

The raw data obtaining module 410 is configured to obtain a configuration file and raw data; the configuration file comprises a data tag to be converted;

the to-be-converted data determining module 420 is configured to determine to-be-converted data in the original data based on the to-be-converted data tag;

the target format data conversion module 430 is configured to convert the data to be converted into target format data.

Optionally, the configuration file further includes a combination parameter, and the number of data to be converted is at least two;

the data processing device may further include:

the time synchronization relation determining module is used for determining the time synchronization relation of each data to be converted under the condition that the merging parameters are merging after the data to be converted are converted into the target format data;

and the target format data merging module is used for merging the target format data corresponding to each data to be converted based on the time synchronization relation.

On the basis of the above scheme, optionally, each data to be converted includes data of point cloud to be converted acquired according to time sequence, and the time synchronization relationship determining module may include:

the time synchronization relation determining unit is used for determining the time synchronization relation among the point cloud data to be converted according to the acquisition time of the point cloud data to be converted;

the target format data merging module may include:

and the target format data merging unit is used for merging the target format data corresponding to the point cloud data to be converted with the time synchronization relationship.

On the basis of the above-described aspect, optionally, the time synchronization relationship determining unit may include:

A starting point cloud data initializing subunit, configured to initialize starting point cloud data;

the to-be-converted point cloud data acquisition subunit is used for acquiring each to-be-converted point cloud data according to the acquisition time of each to-be-converted point cloud data and the time sequence;

the point cloud data to be converted is taken as a subunit, and the point cloud data to be converted, of which the absolute value of the difference value between the acquisition time and the acquisition time of the initial point cloud data is less than or equal to the acquisition time difference, is taken as the point cloud data to be converted, which has a time synchronization relationship with the initial point cloud data;

the starting point cloud data is used as a subunit, and is used for taking the first point cloud data to be converted which does not have a time synchronization relationship with the starting point cloud data as new starting point cloud data until the time synchronization relationship among the point cloud data to be converted is determined.

On the basis of the above scheme, optionally, the data processing device may further include the following module to determine point cloud data to be converted having a time synchronization relationship with the point cloud data:

the point cloud data to be converted is used as a module, and if the absolute value of the difference value between the acquisition time of the currently acquired point cloud data to be converted and the acquisition time of the starting point cloud data is smaller than or equal to the acquisition time difference, and the number of the point cloud data to be converted, which has a time synchronization relationship with the starting point cloud data, does not reach the preset number, the currently acquired point cloud data to be converted is used as the point cloud data to be converted, which has a time synchronization relationship with the starting point cloud data.

On the basis of the above scheme, optionally, the data processing device may further include the following module to obtain the acquisition time difference:

the acquisition time interval determining module is used for acquiring acquisition frequency corresponding to the original data and determining an acquisition time interval according to the acquisition frequency;

the acquisition time difference obtaining module is used for obtaining the acquisition time difference based on the acquisition time interval and a preset time interval algorithm.

Optionally, the configuration file further includes field parameters; the data processing device may further include:

and the attribute field determining module is used for determining the attribute field of the target format data according to the attribute information and the field parameters of the original data.

Optionally, the data processing apparatus may further include:

the target save address determining module is used for determining the target save address according to at least one of the following: the method comprises the steps of storing an address of original data, a name of a storage catalog of the original data, a storage address parameter in a configuration file, a name of the original data and a name of data to be converted;

the data processing device may further include:

and the target general data storage module is used for storing the target format data into the target storage address after converting the data to be converted into the target format data.

According to the data processing device provided by the fourth embodiment of the invention, the configuration file and the original data are acquired through the original data acquisition module; the configuration file comprises a data tag to be converted; determining the data to be converted in the original data based on the data tag to be converted by a data to be converted determining module; and converting the data to be converted into target format data through a target format data conversion module. According to the device, the data to be converted with possibly different data types is determined through the data tag to be converted, and the data to be converted is converted into the target format data with the uniform type, so that the uniform processing of the data with different types is realized.

The data processing device provided by the embodiment of the invention can execute the data processing method provided by any embodiment of the invention, and has the corresponding functional modules and beneficial effects of the execution method.

It should be noted that, in the embodiment of the data processing apparatus, each unit and module included are only divided according to the functional logic, but not limited to the above-mentioned division, so long as the corresponding functions can be implemented; in addition, the specific names of the functional units are also only for distinguishing from each other, and are not used to limit the protection scope of the present invention.

Example five

Fig. 9 shows a schematic diagram of an electronic device 20 that may be used to implement an embodiment of the invention. Electronic devices are intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. Electronic equipment may also represent various forms of mobile devices, such as personal digital processing, cellular telephones, smartphones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be exemplary only, and are not meant to limit implementations of the inventions described and/or claimed herein.

As shown in fig. 9, the electronic device 20 includes at least one processor 22, and a memory, such as a Read Only Memory (ROM) 22, a Random Access Memory (RAM) 23, etc., communicatively coupled to the at least one processor 22, wherein the memory stores computer programs executable by the at least one processor, and the processor 22 may perform various suitable actions and processes according to the computer programs stored in the Read Only Memory (ROM) 22 or the computer programs loaded from the storage unit 28 into the Random Access Memory (RAM) 23. In the RAM 23, various programs and data required for the operation of the electronic device 20 may also be stored. The processor 22, ROM 22 and RAM 23 are connected to each other by a bus 24. An input/output (I/O) interface 25 is also connected to bus 24.

Various components in the electronic device 20 are connected to the I/O interface 25, including: an input unit 26 such as a keyboard, a mouse, etc.; an output unit 27 such as various types of displays, speakers, and the like; a storage unit 28 such as a magnetic disk, an optical disk, or the like; and a communication unit 29 such as a network card, modem, wireless communication transceiver, etc. The communication unit 29 allows the electronic device 20 to exchange information/data with other devices via a computer network, such as the internet, and/or various telecommunication networks.

The processor 22 may be a variety of general and/or special purpose processing components having processing and computing capabilities. Some examples of processor 22 include, but are not limited to, a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), various specialized Artificial Intelligence (AI) computing chips, various processors running machine learning model algorithms, digital Signal Processors (DSPs), and any suitable processor, controller, microcontroller, etc. The processor 22 performs the various methods and processes described above, such as the processing of data.

In some embodiments, the method of processing data may be implemented as a computer program tangibly embodied on a computer-readable storage medium, such as storage unit 28. In some embodiments, part or all of the computer program may be loaded and/or installed onto the electronic device 20 via the ROM 22 and/or the communication unit 29. When the computer program is loaded into the RAM 23 and executed by the processor 22, one or more steps of the data processing method described above may be performed. Alternatively, in other embodiments, the processor 22 may be configured to perform the processing of the data in any other suitable manner (e.g., by means of firmware).

Various implementations of the systems and techniques described here above may be implemented in digital electronic circuitry, integrated circuit systems, field Programmable Gate Arrays (FPGAs), application Specific Integrated Circuits (ASICs), application Specific Standard Products (ASSPs), systems On Chip (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs, the one or more computer programs may be executed and/or interpreted on a programmable system including at least one programmable processor, which may be a special purpose or general-purpose programmable processor, that may receive data and instructions from, and transmit data and instructions to, a storage system, at least one input device, and at least one output device.

A computer program for carrying out methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus, such that the computer programs, when executed by the processor, cause the functions/acts specified in the flowchart and/or block diagram block or blocks to be implemented. The computer program may execute entirely on the machine, partly on the machine, as a stand-alone software package, partly on the machine and partly on a remote machine or entirely on the remote machine or server.

In the context of the present invention, a computer-readable storage medium may be a tangible medium that can contain, or store a computer program for use by or in connection with an instruction execution system, apparatus, or device. The computer readable storage medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. Alternatively, the computer readable storage medium may be a machine readable signal medium. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

To provide for interaction with a user, the systems and techniques described here can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and a pointing device (e.g., a mouse or a trackball) through which a user can provide input to the electronic device. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including acoustic input, speech input, or tactile input.

The systems and techniques described here can be implemented in a computing system that includes a background component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such background, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local Area Networks (LANs), wide Area Networks (WANs), blockchain networks, and the internet.

The computing system may include clients and servers. The client and server are typically remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server can be a cloud server, also called a cloud computing server or a cloud host, and is a host product in a cloud computing service system, so that the defects of high management difficulty and weak service expansibility in the traditional physical hosts and VPS service are overcome.

It should be appreciated that various forms of the flows shown above may be used to reorder, add, or delete steps. For example, the steps described in the present invention may be performed in parallel, sequentially, or in a different order, so long as the desired results of the technical solution of the present invention are achieved, and the present invention is not limited herein.

The above embodiments do not limit the scope of the present invention. It will be apparent to those skilled in the art that various modifications, combinations, sub-combinations and alternatives are possible, depending on design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the scope of the present invention.

Claims

1. A method of processing data, comprising:

determining data to be converted in the original data based on the data tag to be converted;

and converting the data to be converted into target format data.

2. The method of claim 1, wherein the configuration file further comprises a merge parameter, the amount of data to be converted being at least two items;

After converting the data to be converted into the target format data, the method further comprises the following steps:

under the condition that the merging parameters are merging, determining the time synchronization relation of each piece of data to be converted;

and merging the target format data corresponding to each data to be converted based on the time synchronization relation.

3. The method of claim 2, wherein each of the data to be converted includes time-sequentially collected point cloud data to be converted, and the determining the time synchronization relationship of each of the data to be converted includes:

determining a time synchronization relationship among the point cloud data to be converted according to the acquisition time of the point cloud data to be converted;

the merging, based on the time synchronization relationship, the target format data corresponding to each data to be converted includes:

and merging the target format data corresponding to the point cloud data to be converted with the time synchronization relationship.

4. A method according to claim 3, wherein determining the time synchronization relationship between the point cloud data to be converted according to the collection time of the point cloud data to be converted comprises:

initializing starting point cloud data;

acquiring the point cloud data to be converted according to time sequence according to the acquisition time of the point cloud data to be converted;

The point cloud data to be converted, which has a time synchronization relationship with the starting point cloud data, is used as the point cloud data to be converted, wherein the absolute value of the difference value between the acquisition time and the acquisition time of the starting point cloud data is smaller than or equal to the acquisition time difference;

and taking the first point cloud data to be converted which does not have a time synchronization relationship with the starting point cloud data as new starting point cloud data until the time synchronization relationship among the point cloud data to be converted is determined.

5. The method of claim 4, wherein the step of determining the point cloud data to be converted having a time synchronization relationship with the start point cloud data further comprises:

and if the absolute value of the difference value between the acquisition time of the currently acquired point cloud data to be converted and the acquisition time of the starting point cloud data is smaller than or equal to the acquisition time difference, and the number of the point cloud data to be converted which has a time synchronization relationship with the starting point cloud data does not reach the preset number, taking the currently acquired point cloud data to be converted as the point cloud data to be converted which has a time synchronization relationship with the starting point cloud data.

6. The method according to claim 4 or 5, wherein the acquisition time difference is obtained by:

Acquiring acquisition frequency corresponding to the original data, and determining an acquisition time interval according to the acquisition frequency;

and obtaining the acquisition time difference based on the acquisition time interval and a preset time interval algorithm.

7. The method of claim 1, wherein the configuration file further comprises field parameters; the method further comprises the steps of:

and determining the attribute field of the target format data according to the attribute information of the original data and the field parameters.

8. The method as recited in claim 1, further comprising:

determining a target save address according to at least one of: the storage address of the original data, the name of the storage catalog of the original data, the storage address parameter in the configuration file, the name of the original data and the name of the data to be converted;

and storing the target format data into the target storage address.

9. A data processing apparatus, comprising:

10. An electronic device, comprising:

at least one processor; and

a memory communicatively coupled to the at least one processor; wherein,,

the memory stores a computer program executable by the at least one processor to cause the at least one processor to perform the method of processing data as claimed in any one of claims 1 to 8.

11. A computer readable storage medium storing computer instructions for causing a processor to perform the method of processing data according to any one of claims 1-8.