CN112269827B - Data processing method and device, computer equipment and computer readable storage medium - Google Patents
Data processing method and device, computer equipment and computer readable storage medium Download PDFInfo
- Publication number
- CN112269827B CN112269827B CN202011285120.2A CN202011285120A CN112269827B CN 112269827 B CN112269827 B CN 112269827B CN 202011285120 A CN202011285120 A CN 202011285120A CN 112269827 B CN112269827 B CN 112269827B
- Authority
- CN
- China
- Prior art keywords
- data
- driving
- scene
- processor
- travel
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/26—Visual data mining; Browsing structured data
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/21—Design, administration or maintenance of databases
- G06F16/215—Improving data quality; Data cleansing, e.g. de-duplication, removing invalid entries or correcting typographical errors
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2458—Special types of queries, e.g. statistical queries, fuzzy queries or distributed queries
- G06F16/2465—Query processing support for facilitating data mining operations in structured databases
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Quality & Reliability (AREA)
- Fuzzy Systems (AREA)
- Mathematical Physics (AREA)
- Probability & Statistics with Applications (AREA)
- Software Systems (AREA)
- Computational Linguistics (AREA)
- Traffic Control Systems (AREA)
Abstract
The application provides a data processing method, a data processing device, computer equipment and a computer readable storage medium, and belongs to the technical field of data processing. According to the method and the device, after the driving data are obtained, at least one group of driving data with characteristics having relevance is identified from the obtained driving data through a data mining algorithm, so that the labels corresponding to the groups of driving data are obtained, the labels can indicate the characteristics of the driving data, and further, after the scene condition parameters limiting a specific driving scene are obtained, the driving data marked by the labels corresponding to the scene condition parameters are correlated with the specific driving scene limited by the scene condition parameters, so that the driving data only corresponding to a specific driving scene can be obtained on the basis of the scene condition parameters, and the data processing efficiency is improved.
Description
Technical Field
The present application relates to the field of data processing technologies, and in particular, to a data processing method and apparatus, a computer device, and a computer-readable storage medium.
Background
The automatic driving vehicle automatically and safely runs on the road without human operation by means of the cooperative cooperation of artificial intelligence, visual calculation, radar, a monitoring device and a global positioning system. Data are collected by each automatic driving vehicle in the driving process, and then the automatic driving vehicles are trained based on the collected data, so that the identification accuracy and decision-making accuracy of the automatic driving vehicles are improved, and the safety of automatic driving is improved.
At present, when data collected in the running process of an automatic driving vehicle is processed, the obtained data is mainly subjected to data cleaning, and then the data obtained after cleaning is stored and used as subsequent running process analysis of the automatic driving vehicle, or training data used in training of the automatic driving vehicle.
In the implementation process, although some illegal data can be cleaned by data cleaning to reduce a certain data amount, the data amount obtained by data cleaning is still very large, and it is very difficult to acquire data in a certain driving scene from the cleaned data, so that the data processing efficiency is low.
Disclosure of Invention
The embodiment of the application provides a data processing method, a data processing device, computer equipment and a computer readable storage medium, and the data processing efficiency can be improved. The technical scheme is as follows:
in one aspect, a data processing method is provided, and the method includes:
obtaining a plurality of driving data of the autonomous vehicle, the driving data including at least one characteristic related to a driving scene;
identifying at least one group of driving data with characteristics having relevance from the driving data through a data mining algorithm;
determining a label of each set of driving data in the at least one set of driving data, wherein the label is used for indicating the characteristic in each set of driving data;
the method includes the steps of obtaining scene condition parameters for defining a specific driving scene, identifying the label corresponding to the scene condition parameters, and associating the at least one set of driving data marked by the label with the specific driving scene.
In one possible implementation, the identifying, by the data mining algorithm, at least one set of travel data from the travel data having a correlation in characteristics includes:
the at least one set of travel data having the same characteristics is identified from the travel data by the data mining algorithm.
In one possible implementation, the identifying, by the data mining algorithm, at least one set of travel data from which features have relevance includes:
inputting the travel data into at least one processor for executing a data mining script, wherein the data mining script corresponds to the feature;
and any processor in the at least one processor processes the driving data through the corresponding data mining script to obtain a group of driving data with associated characteristics, and the group of driving data obtained by processing of any processor has the characteristics corresponding to the data mining script corresponding to any processor.
In one possible implementation, before the identifying, by the data mining algorithm, at least one set of travel data having a characteristic with relevance from the travel data, the method further includes:
and carrying out pre-classification on the driving data according to the data type of the driving data, and inputting the driving data of the same data type into the same processor to execute the data mining algorithm.
In one possible implementation, the obtaining the plurality of travel data of the autonomous vehicle includes at least one of:
acquiring the driving data on line in the driving process of the automatic driving vehicle;
and after the automatic driving vehicle finishes driving, downloading the driving data acquired in the driving process off line.
In one possible implementation, before the step of identifying at least one set of driving data with associated features from the driving data by a data mining algorithm, the method further includes:
and performing data cleaning processing on the running data, wherein the data cleaning processing is used for deleting wrong or illegal running data in the running data.
In one possible implementation, after associating the at least one set of driving data labeled by the label with the specific driving scenario, the method further includes:
the driving data corresponding to the same specific driving scene in the driving data is stored as one data set.
In one possible implementation, the obtaining of the scene condition parameters for defining the specific driving scene comprises:
the scene condition parameters defining a particular driving scene input by a user are received.
In one aspect, a data processing apparatus is provided, the apparatus comprising:
an acquisition module for acquiring a plurality of driving data of an autonomous vehicle, the driving data including at least one characteristic related to a driving scenario;
the identification module is used for identifying at least one group of driving data with characteristics having relevance from the driving data through a data mining algorithm;
a determination module for determining a tag of each set of travel data of the at least one set of travel data, the tag indicating the characteristic of each set of travel data;
the association module is used for acquiring scene condition parameters for limiting a specific driving scene, identifying the label corresponding to the scene condition parameters, and associating the at least one set of driving data marked by the label with the specific driving scene.
In one possible implementation, the identification module is configured to identify the at least one set of driving data having the same characteristics from the driving data through the data mining algorithm.
In one possible implementation, the identification module is configured to input the driving data into at least one processor configured to execute a data mining script, wherein the data mining script corresponds to the feature; and any processor in the at least one processor processes the driving data through the corresponding data mining script to obtain a group of driving data with associated characteristics, and the group of driving data obtained by processing of any processor has the characteristics corresponding to the data mining script corresponding to any processor.
In one possible implementation, the apparatus further includes:
and the classification module is used for carrying out pre-classification on the driving data according to the data type of the driving data, and inputting the driving data of the same data type into the same processor to execute the data mining algorithm.
In one possible implementation, the obtaining module is configured to:
acquiring the driving data on line in the driving process of the automatic driving vehicle;
and after the automatic driving vehicle finishes driving, downloading the driving data acquired in the driving process in an off-line manner.
In one possible implementation, the apparatus further includes:
and the data cleaning module is used for carrying out data cleaning processing on the driving data, and the data cleaning processing is used for deleting wrong or illegal driving data in the driving data.
In one possible implementation, the apparatus further includes:
and the storage module is used for storing the running data corresponding to the same specific running scene in the running data into a data set.
In one possible implementation, the association module is configured to receive the scene condition parameters defining a specific driving scene input by a user.
In one aspect, a computer device is provided that includes one or more processors and one or more memories having at least one program code stored therein, the program code being loaded and executed by the one or more processors to perform the operations performed by the data processing method.
In one aspect, a computer-readable storage medium having at least one program code stored therein is provided, the program code being loaded and executed by a processor to implement the operations performed by the data processing method.
In one aspect, a computer program product is provided that includes computer instructions stored in a computer readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions to implement the operations performed by the data processing method.
According to the scheme provided by the application, after the driving data is acquired, at least one set of driving data with characteristics having relevance is identified from the acquired driving data through a data mining algorithm, so that the labels corresponding to the sets of driving data are obtained, the labels can indicate the characteristics of the driving data, and further, after the scene condition parameters for limiting a specific driving scene are acquired, the driving data marked by the labels corresponding to the scene condition parameters are correlated with the specific driving scene limited by the scene condition parameters, so that the driving data only corresponding to a specific driving scene can be acquired on the basis of the scene condition parameters, and the data processing efficiency is improved.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the description of the embodiments are briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.
Fig. 1 is a schematic diagram of an implementation environment of a data processing method according to an embodiment of the present application;
fig. 2 is a flowchart of a data processing method provided in an embodiment of the present application;
fig. 3 is a flowchart of a data processing method provided in an embodiment of the present application;
FIG. 4 is a diagram illustrating a data processing process according to an embodiment of the present disclosure;
fig. 5 is a schematic structural diagram of a data processing apparatus according to an embodiment of the present application;
fig. 6 is a schematic structural diagram of a computer device according to an embodiment of the present application.
Detailed Description
To make the objects, technical solutions and advantages of the present application more clear, embodiments of the present application will be described in further detail below with reference to the accompanying drawings.
Fig. 1 is a schematic diagram of an implementation environment of a data processing method according to an embodiment of the present application, and referring to fig. 1, the implementation environment includes an external storage device 101 and a computer device 102, or the implementation environment includes a computer device 102 and a vehicle-mounted terminal 103, and the following two implementation environments are respectively described:
for the first implementation environment, the external storage device 101 is at least one of a removable hard disk, a floppy disk, an optical disk, and the like. The external storage device communicates with the computer device 102 by wired communication, and uploads a plurality of pieces of travel data stored in the external storage device 101 to the computer device 102. The computer device 102 is at least one of a desktop computer, a computer cluster, a server, a plurality of servers, a cloud server, a cloud computing platform, and a virtualization center. The computer apparatus 102 communicates with the external storage apparatus 101 by wired communication. The computer device 102 receives the plurality of pieces of travel data transmitted from the external storage device 101, processes the plurality of pieces of travel data, and stores the processed pieces of travel data. Optionally, the number of the computer devices is greater or smaller, which is not limited in the embodiments of the present application. Of course, the computer device 102 can also include other functional devices to provide more comprehensive and diverse services.
In the second embodiment, the in-vehicle terminal 103 communicates with the computer device 102 by wireless communication, and uploads a plurality of pieces of travel data stored in the in-vehicle terminal 103 to the computer device 102. The computer device 102 is at least one of a desktop computer, a computer cluster, a server, a plurality of servers, a cloud server, a cloud computing platform, and a virtualization center. The computer device 102 communicates with the in-vehicle terminal 103 by wireless communication. The computer device 102 receives the plurality of pieces of travel data transmitted from the in-vehicle terminal 103, processes the plurality of pieces of travel data, and stores the processed pieces of travel data. Optionally, the number of the computer devices is greater or smaller, which is not limited in the embodiments of the present application. Of course, the computer device 102 can also include other functional devices in order to provide more comprehensive and diverse services.
Fig. 2 is a flowchart of a data processing method provided in an embodiment of the present application, and referring to fig. 2, the method includes:
201. a computer device obtains a plurality of driving data of an autonomous vehicle, the driving data including at least one characteristic associated with a driving scenario.
202. The computer device identifies at least one set of driving data from the driving data, the characteristics of which have relevance, by means of a data mining algorithm.
203. The computer device determines a label for each of the at least one set of travel data, the label indicating the characteristic in the each set of travel data.
204. The computer device obtains a scene condition parameter defining a particular driving scenario, identifies the tag corresponding to the scene condition parameter, and associates the at least one set of driving data tagged by the tag with the particular driving scenario.
According to the scheme provided by the embodiment of the application, after the driving data is acquired, at least one group of driving data with characteristics having relevance is identified from the acquired driving data through a data mining algorithm, so that the label corresponding to each group of driving data is obtained, the label can indicate the characteristics of the driving data, and further, after the scene condition parameters for limiting a specific driving scene are acquired, the driving data marked by the label corresponding to the scene condition parameters are correlated with the specific driving scene limited by the scene condition parameters, so that the driving data only corresponding to a specific driving scene can be acquired on the basis of the scene condition parameters, and the data processing efficiency is improved.
In one possible implementation, the identifying, by the data mining algorithm, at least one set of travel data from the travel data having a correlation in characteristics includes:
the at least one set of travel data having the same characteristics is identified from the travel data by the data mining algorithm.
In one possible implementation, the identifying, by the data mining algorithm, at least one set of travel data from which features have relevance includes:
inputting the travel data into at least one processor for executing a data mining script, wherein the data mining script corresponds to the feature;
and any processor in the at least one processor processes the driving data through the corresponding data mining script to obtain a group of driving data with associated characteristics, and the group of driving data obtained by processing of any processor has the characteristics corresponding to the data mining script corresponding to any processor.
In one possible implementation, before the identifying, by the data mining algorithm, at least one set of travel data having a characteristic with relevance from the travel data, the method further includes:
and carrying out pre-classification on the driving data according to the data type of the driving data, and inputting the driving data of the same data type into the same processor to execute the data mining algorithm.
In one possible implementation, the obtaining the plurality of travel data of the autonomous vehicle includes at least one of:
acquiring the driving data on line in the driving process of the automatic driving vehicle;
and after the automatic driving vehicle finishes driving, downloading the driving data acquired in the driving process off line.
In one possible implementation, before the step of identifying at least one set of driving data with associated features from the driving data by a data mining algorithm, the method further includes:
and performing data cleaning processing on the running data, wherein the data cleaning processing is used for deleting wrong or illegal running data in the running data.
In one possible implementation, after associating the at least one set of driving data labeled by the label with the specific driving scenario, the method further includes:
the travel data corresponding to the same specific travel scene in the travel data is stored as one data set.
In one possible implementation, the obtaining of the scene condition parameters for defining the specific driving scene comprises:
the scene condition parameters defining a particular driving scene input by a user are received.
Fig. 3 is a flowchart of a data processing method provided in an embodiment of the present application, and referring to fig. 3, the method includes:
301. a computer device obtains a plurality of driving data of an autonomous vehicle, the driving data including at least one characteristic associated with a driving scenario.
In one possible implementation, the computer device obtains the driving data online during driving of the autonomous vehicle.
The online acquisition process is also a process of acquiring data acquired by the autonomous vehicle in real time in the autonomous driving process. For example, in the driving process of the autonomous vehicle, the vehicle-mounted terminal of the autonomous vehicle acquires data of the external environment and the vehicle in real time in the driving process, and after acquiring one driving data, the acquired driving data is sent to the computer device, so that the computer device can acquire the currently acquired driving data, and so on, so that the computer device can acquire a plurality of driving data in the driving process of the autonomous vehicle.
In another possible implementation, the computer device downloads the travel data acquired during travel offline after the autonomous vehicle completes travel.
The off-line downloading process is to acquire the running data in real time in the running process of the automatic driving vehicle, store the acquired running data, and transmit the stored running data to the computer equipment once after the automatic driving vehicle finishes running, so that the computer equipment can acquire a plurality of running data acquired in the running process of the automatic driving vehicle once. For example, after the autonomous vehicle finishes traveling, the related art person inserts a mobile hard disk for storing a plurality of traveling data in the autonomous vehicle into a computer device, and the computer device automatically acquires the traveling data stored in the mobile hard disk after connecting to the mobile hard disk.
The feature is related to a travel scene, and can be used to indicate a travel scene corresponding to travel data. For example, the feature includes a traffic light, a highway, a branch intersection, a tunnel, a ramp, rain, vehicle merging, following, and the like, and optionally, the feature includes others, which is not limited in the embodiment of the present application.
In a more possible implementation manner, after acquiring a plurality of driving data, the computer device performs data cleaning processing on the driving data, wherein the data cleaning processing is used for deleting wrong or illegal driving data in the driving data. The data cleansing process includes checking data consistency, processing invalid values and missing values, and the like, and optionally, the data cleansing process includes other types of processing operations, which is not limited in this embodiment.
By carrying out data cleaning processing on the acquired driving data, error data and invalid data in the driving data can be effectively removed, the data quality is improved, and the data after the data cleaning processing is more suitable for data mining. In addition, since the error data, the invalid data and the like are removed from the driving data, the data do not need to be mined when the data are mined, the processing pressure of computer equipment is reduced, and the speed and the efficiency of data mining are improved.
After performing data cleansing processing on a plurality of pieces of travel data, the computer device stores the travel data subjected to the data cleansing processing in the big data system, and further executes the following steps 302 to 305 by the big data system to process the travel data.
302. The computer device inputs the travel data into at least one processor for executing a data mining script, wherein the data mining script corresponds to the feature.
It should be noted that, in the embodiment of the present application, data mining refers to a process of searching information hidden in a large amount of data through an algorithm. Data mining algorithms including, but not limited to, statistical, online analytical processing, intelligence retrieval, machine learning, expert systems, and pattern recognition methods are implemented via data mining scripts. The computer equipment comprises at least one processor, wherein one processor corresponds to one data mining script, one data mining script corresponds to one characteristic relevant to a driving scene, and different processors identify different characteristics through different data mining scripts so as to determine driving data corresponding to different driving scenes. In addition, since one data mining script corresponds to one feature and each feature corresponds to a corresponding driving scenario, but the same driving scenario may correspond to a plurality of features, the driving scenarios determined based on different features may be the same, that is, the driving data identified based on different data mining scripts may be the same. The data mining script includes multiple types, such as a parallel scene mining script for determining driving data of a parallel scene, a following scene mining script for determining driving data of a following scene, an uphill scene mining script for determining driving data of an uphill scene, and the like.
It should be noted that, the above are only a few exemplary data mining scripts, and in more possible implementation manners, different data mining scripts are designed according to requirements for data to obtain corresponding data, so as to perform analysis or training on an autonomous vehicle based on the obtained data in the following.
Optionally, before inputting the driving data into the at least one processor, the computer device pre-classifies the driving data according to the data type of the driving data, and inputs the driving data of the same data type into the same processor to execute a data mining algorithm, i.e., a data mining script. The driving data are subjected to pre-classification, namely, the driving data are subjected to primary screening, and in the subsequent processing process, the processor only needs to process the driving data of corresponding types, so that the data quantity required to be processed by the processor is reduced, the processing pressure of the processor is reduced, and the processing efficiency of the processor is improved.
For example, for instantaneous data such as vehicle speed, acceleration, oil quantity, oil consumption and positioning information and non-instantaneous data such as mileage, track, running time and parking time acquired by an automatic driving vehicle in the running process, the computer device can distinguish the instantaneous data from the non-instantaneous data according to the corresponding data types, and then processes the two types of running data through different processors.
It should be noted that the above is only an exemplary method for pre-classifying the traveling data information, and in a more possible implementation manner, the traveling data information is pre-classified in another manner, which is not limited in the embodiment of the present application.
303. Any processor in at least one processor of the computer equipment processes the driving data through a data mining script corresponding to the processor to obtain a group of driving data with associated characteristics, and the group of driving data obtained by processing of any processor has the characteristics corresponding to the data mining script corresponding to any processor.
In one possible implementation manner, any processor in the at least one processor of the computer device processes the driving data through the corresponding data mining script to obtain a set of driving data with the same characteristics as the characteristics corresponding to the data mining script corresponding to the any processor.
It should be noted that the computer device can input the driving data into a plurality of processors at the same time, and then the plurality of processors process the driving data in parallel through the corresponding data mining scripts to obtain a plurality of sets of driving data, and for each set of driving data, all driving data in the set of driving data have the characteristics of the data mining script corresponding to the processor that processes the driving data, that is, the characteristics corresponding to the data in each set of driving data are all related or the same, so that the plurality of sets of driving data have different characteristics.
For example, the computer device inputs the driving data into a processor for executing the merging scenario mining script, and the driving data is processed by the processor for executing the merging scenario mining script, so that a set of driving data with merging characteristics can be obtained, that is, in the set of driving data, the autonomous vehicle has a merging operation or another vehicle is merged into a lane in which the autonomous vehicle is driving; the driving data is input into a processor for executing the following scene mining script, and the driving data is processed through the processor for executing the following scene mining script, so that a group of driving data with following characteristics can be obtained, namely, the following operation of the automatic driving vehicle is performed in the group of driving data; the driving data is input into a processor for executing the uphill scene mining script, and the driving data is processed through the processor for executing the uphill scene mining script, so that a group of driving data with uphill characteristics can be obtained, namely, in the group of driving data, the gradient of the automatic driving vehicle in a certain section of road exceeds a threshold value, and the like, so that the driving data corresponding to different characteristics can be obtained.
304. The computer device determines a label for each set of travel data of the at least one set of travel data, the label indicating the characteristic in the each set of travel data.
Note that the tag is data for describing information, which is extracted from the travel data and used for describing the characteristics of the travel data.
In one possible implementation manner, after determining the driving data corresponding to different characteristics, the computer device determines the label corresponding to each set of driving data based on the characteristics corresponding to each set of driving data, and further obtains the label of each set of driving data.
305. The computer device obtains a scene condition parameter defining a particular driving scenario, identifies the tag corresponding to the scene condition parameter, and associates the at least one set of driving data tagged by the tag with the particular driving scenario.
In a possible implementation manner, a user inputs scene condition parameters for defining a specific driving scene into a computer device, the computer device acquires the scene condition parameters, further identifies a tag corresponding to the scene condition parameters, associates and stores at least one set of driving data marked by the tag corresponding to the scene condition parameters with the specific driving scene, and realizes association of the driving data with the specific driving scene.
For example, if a user wishes to extract driving data corresponding to a driving scene of "slow speed uphill at night", the user inputs scene condition parameters of "night", "slow speed", and "uphill" into the computer device, so that the computer device acquires the scene condition parameters, further identifies tags corresponding to the scene condition parameters, identifies that a tag corresponding to "night" is "ambient brightness lower than x", a tag corresponding to "slow speed" is "speed lower than x", and a tag corresponding to "uphill" is "gradient higher than x", and further stores the driving data corresponding to the tags in association with a specific driving scene of "slow speed uphill at night".
Optionally, after the driving data is associated with a specific driving scene, the computer device stores the driving data corresponding to the same specific driving scene in the driving data as a data set, so that when data of a specific driving scene is searched for later, the data set corresponding to the tag is directly determined according to the tag, and then all data corresponding to the tag are obtained, and data searching one by one is not needed, thereby improving data processing efficiency.
Referring to fig. 4, fig. 4 is a schematic diagram of a data processing process according to an embodiment of the present application, which shows the data processing process from step 301 to step 305. The computer equipment acquires driving data acquired by an automatic driving vehicle in the driving process, namely the collection of the driving data is realized, data cleaning processing is carried out on the data from a mobile phone, and data after the data cleaning processing is carried out through different processors (Processor1, Processor2, Processor3, procesor and ProcessorN), so that the association of the driving data and a specific driving scene is realized.
It should be noted that the scheme provided in the embodiment of the present application can be used for processing the driving data acquired by the commercial vehicle in the automatic driving process, and can also be used for processing the driving data acquired by the passenger vehicle in the automatic driving process.
According to the scheme provided by the embodiment of the application, after the driving data is acquired, at least one group of driving data with characteristics having relevance is identified from the acquired driving data through a data mining algorithm, so that the label corresponding to each group of driving data is obtained, the label can indicate the characteristics of the driving data, and further, after the scene condition parameters for limiting a specific driving scene are acquired, the driving data marked by the label corresponding to the scene condition parameters are correlated with the specific driving scene limited by the scene condition parameters, so that the driving data only corresponding to a specific driving scene can be acquired on the basis of the scene condition parameters, and the data processing efficiency is improved. According to the scheme provided by the embodiment of the application, according to a large amount of driving data collected in the automatic driving process, after the data are washed and enter a big data system, various types of off-line excavation are carried out on the data through various data excavation scripts, on the basis of the off-line excavation of the large amount of data of the automatic driving, corresponding labels are marked for the plurality of driving data, the automatic driving data can be subjected to depth analysis, problems and scenes in the data can be accurately positioned, the excavated data are subjected to classification processing, so that the driving data of a certain type can be directly acquired later, and the data acquisition efficiency is improved.
All the above optional technical solutions may be combined arbitrarily to form optional embodiments of the present application, and are not described herein again.
Fig. 5 is a schematic structural diagram of a data processing apparatus according to an embodiment of the present application, and referring to fig. 5, the apparatus includes:
an obtaining module 501, configured to obtain a plurality of driving data of an autonomous vehicle, where the driving data includes at least one characteristic related to a driving scene;
an identification module 502, configured to identify at least one set of driving data with associated features from the driving data through a data mining algorithm;
a determining module 503, configured to determine a label of each set of the at least one set of driving data, where the label is used to indicate the characteristic in each set of driving data;
an associating module 504 is configured to obtain a scene condition parameter for defining a specific driving scenario, identify the tag corresponding to the scene condition parameter, and associate the at least one set of driving data marked by the tag with the specific driving scenario.
According to the device provided by the embodiment of the application, after the running data is acquired, at least one set of running data with characteristics having relevance is identified from the acquired running data through a data mining algorithm, so that the label corresponding to each set of running data is obtained, the label can indicate the characteristics of the running data, and further, after the scene condition parameters for limiting a specific running scene are acquired, the running data marked by the label corresponding to the scene condition parameters is correlated with the specific running scene limited by the scene condition parameters, so that the running data only corresponding to a specific driving scene can be acquired directly on the basis of the scene condition parameters, and the data processing efficiency is improved.
In one possible implementation, the identification module 502 is configured to identify the at least one set of driving data having the same characteristics from the driving data through the data mining algorithm.
In one possible implementation, the identification module 502 is configured to input the driving data into at least one processor configured to execute a data mining script, wherein the data mining script corresponds to the feature; and processing the driving data by any processor in the at least one processor through the corresponding data mining script to obtain a group of driving data with relevant characteristics, wherein the group of driving data obtained by processing by any processor has the characteristics corresponding to the data mining script corresponding to any processor.
In one possible implementation, the apparatus further includes:
and the classification module is used for carrying out pre-classification on the driving data according to the data type of the driving data, and inputting the driving data of the same data type into the same processor to execute the data mining algorithm.
In a possible implementation manner, the obtaining module 501 is configured to at least one of:
acquiring the driving data on line in the driving process of the automatic driving vehicle;
and after the automatic driving vehicle finishes driving, downloading the driving data acquired in the driving process off line.
In one possible implementation, the apparatus further includes:
and the data cleaning module is used for carrying out data cleaning processing on the driving data, and the data cleaning processing is used for deleting wrong or illegal driving data in the driving data.
In one possible implementation, the apparatus further includes:
and the storage module is used for storing the running data corresponding to the same specific running scene in the running data into a data set.
In one possible implementation, the association module is configured to receive the scene condition parameters defining a specific driving scene input by a user.
It should be noted that: in the data processing apparatus provided in the above embodiment, only the division of the functional modules is illustrated when performing data processing, and in practical applications, the functions may be distributed by different functional modules as needed, that is, the internal structure of the computer device may be divided into different functional modules to complete all or part of the functions described above. In addition, the data processing apparatus and the data processing method provided by the above embodiments belong to the same concept, and specific implementation processes thereof are described in the method embodiments for details, which are not described herein again.
Fig. 6 is a schematic structural diagram of a server according to an embodiment of the present application, where the server 600 may generate a relatively large difference due to different configurations or performances, and may include one or more processors (CPUs) 601 and one or more memories 602, where at least one program code is stored in the one or more memories 602, and is loaded and executed by the one or more processors 601 to implement the methods provided by the method embodiments. Of course, the server 600 may also have components such as a wired or wireless network interface, a keyboard, and an input/output interface, so as to perform input and output, and the server 600 may also include other components for implementing the functions of the device, which is not described herein again.
In an exemplary embodiment, a computer-readable storage medium, such as a memory including program code, which is executable by a processor to perform the data processing method in the above-described embodiments, is also provided. For example, the computer-readable storage medium may be a Read-Only Memory (ROM), a Random Access Memory (RAM), a Compact Disc Read-Only Memory (CD-ROM), a magnetic tape, a floppy disk, an optical data storage device, and the like.
In an exemplary embodiment, a computer program product or a computer program is also provided, which comprises computer program code stored in a computer-readable storage medium, which is read by a processor of a computer device from the computer-readable storage medium, and which is executed by the processor such that the server performs the method steps of the data processing method provided in the above-mentioned embodiments.
It will be understood by those skilled in the art that all or part of the steps for implementing the above embodiments may be implemented by hardware, or may be implemented by hardware associated with program code, and the program may be stored in a computer readable storage medium, where the above mentioned storage medium may be a read-only memory, a magnetic or optical disk, etc.
The above description is only exemplary of the present application and should not be taken as limiting, as any modification, equivalent replacement, or improvement made within the spirit and principle of the present application should be included in the protection scope of the present application.
Claims (16)
1. A method of data processing, the method comprising:
obtaining a plurality of driving data of an autonomous vehicle, the driving data including at least one characteristic related to a driving scene;
inputting the travel data into at least one processor for executing a data mining script, wherein the data mining script corresponds to the feature;
any processor of the at least one processor processes the driving data through a data mining script corresponding to the processor to obtain a group of driving data with characteristics having relevance, and the group of driving data obtained by processing of any processor has characteristics corresponding to the data mining script corresponding to any processor;
determining a label for each set of travel data of the at least one set of travel data, the label indicating the characteristic in the each set of travel data;
the method comprises the steps of acquiring scene condition parameters for defining a specific driving scene, identifying the label corresponding to the scene condition parameters, and associating the at least one set of driving data marked by the label with the specific driving scene.
2. The method of claim 1, wherein the identifying, by a data mining algorithm, at least one set of travel data from the travel data having associated features comprises:
identifying, by the data mining algorithm, the at least one set of travel data having the same characteristics from the travel data.
3. The method of claim 1, wherein before identifying at least one set of travel data from the travel data having associated characteristics via a data mining algorithm, the method further comprises:
and carrying out pre-classification on the driving data according to the data type of the driving data, and inputting the driving data of the same data type into the same processor to execute the data mining algorithm.
4. The method of claim 1, wherein the obtaining a plurality of travel data for an autonomous vehicle comprises at least one of:
acquiring the driving data on line in the driving process of the automatic driving vehicle;
and after the automatic driving vehicle finishes driving, downloading the driving data acquired in the driving process off line.
5. The method of claim 1, wherein the step of identifying at least one set of travel data from the travel data having associated characteristics via a data mining algorithm is preceded by the method further comprising:
and carrying out data cleaning processing on the driving data, wherein the data cleaning processing is used for deleting wrong or illegal driving data in the driving data.
6. The method of claim 1, wherein after associating the at least one set of driving data labeled by the label with the particular driving scenario, the method further comprises:
and storing the driving data corresponding to the same specific driving scene in the driving data as a data set.
7. The method of claim 1, wherein said obtaining scene condition parameters defining a particular driving scenario comprises:
receiving the scene condition parameters defining a particular driving scene input by a user.
8. A data processing apparatus, characterized in that the apparatus comprises:
an acquisition module for acquiring a plurality of driving data of an autonomous vehicle, the driving data including at least one characteristic related to a driving scenario;
an identification module to input the travel data into at least one processor for executing a data mining script, wherein the data mining script corresponds to the feature; any processor of the at least one processor processes the driving data through a data mining script corresponding to the processor to obtain a group of driving data with characteristics having relevance, and the group of driving data obtained by processing of any processor has characteristics corresponding to the data mining script corresponding to any processor;
a determination module for determining a label for each set of travel data of the at least one set of travel data, the label being indicative of the characteristic in the each set of travel data;
the association module is used for acquiring scene condition parameters for limiting a specific driving scene, identifying the label corresponding to the scene condition parameters, and associating the at least one set of driving data marked by the label with the specific driving scene.
9. The apparatus of claim 8, wherein the identification module is configured to identify the at least one set of travel data having the same characteristics from the travel data via the data mining algorithm.
10. The apparatus of claim 8, further comprising:
and the classification module is used for carrying out pre-classification on the driving data according to the data type of the driving data, and inputting the driving data of the same data type into the same processor to execute the data mining algorithm.
11. The apparatus of claim 8, wherein the obtaining module is configured to at least one of:
acquiring the driving data on line in the driving process of the automatic driving vehicle;
and after the automatic driving vehicle finishes driving, downloading the driving data acquired in the driving process off line.
12. The apparatus of claim 8, further comprising:
and the data cleaning module is used for carrying out data cleaning processing on the driving data, and the data cleaning processing is used for deleting wrong or illegal driving data in the driving data.
13. The apparatus of claim 8, further comprising:
and the storage module is used for storing the running data corresponding to the same specific running scene in the running data into a data set.
14. The apparatus of claim 8, wherein the correlation module is configured to receive the scene condition parameters defining a particular driving scene input by a user.
15. A computer device comprising one or more processors and one or more memories having at least one program code stored therein, the program code being loaded and executed by the one or more processors to perform operations performed by the data processing method of any one of claims 1 to 7.
16. A computer-readable storage medium having stored therein at least one program code, the program code being loaded and executed by a processor to perform operations performed by a data processing method according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011285120.2A CN112269827B (en) | 2020-11-17 | 2020-11-17 | Data processing method and device, computer equipment and computer readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011285120.2A CN112269827B (en) | 2020-11-17 | 2020-11-17 | Data processing method and device, computer equipment and computer readable storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112269827A CN112269827A (en) | 2021-01-26 |
| CN112269827B true CN112269827B (en) | 2022-09-20 |
Family
ID=74340031
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011285120.2A Active CN112269827B (en) | 2020-11-17 | 2020-11-17 | Data processing method and device, computer equipment and computer readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112269827B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112905849A (en) * | 2021-02-18 | 2021-06-04 | 中国第一汽车股份有限公司 | Vehicle data processing method and device |
| CN117149860A (en) * | 2023-10-31 | 2023-12-01 | 安徽中科星驰自动驾驶技术有限公司 | Driving data mining method and system for automatic driving vehicle |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111367968A (en) * | 2020-03-16 | 2020-07-03 | 斑马网络技术有限公司 | Driving data processing method, device, equipment and storage medium |
| CN111923919A (en) * | 2019-05-13 | 2020-11-13 | 广州汽车集团股份有限公司 | Vehicle control method, vehicle control device, computer equipment and storage medium |
-
2020
- 2020-11-17 CN CN202011285120.2A patent/CN112269827B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111923919A (en) * | 2019-05-13 | 2020-11-13 | 广州汽车集团股份有限公司 | Vehicle control method, vehicle control device, computer equipment and storage medium |
| CN111367968A (en) * | 2020-03-16 | 2020-07-03 | 斑马网络技术有限公司 | Driving data processing method, device, equipment and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112269827A (en) | 2021-01-26 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112269827B (en) | Data processing method and device, computer equipment and computer readable storage medium | |
| CN108229567B (en) | Driver identity recognition method and device | |
| CN111121797B (en) | Road screening method, device, server and storage medium | |
| CN114418021B (en) | Model optimization method, device and computer program product | |
| CN111507226A (en) | Road image recognition model modeling method, image recognition method and electronic equipment | |
| CN110717918A (en) | Pedestrian detection method and device | |
| CN113780435A (en) | Vehicle damage detection method, device, equipment and storage medium | |
| CN112598192A (en) | Method and device for predicting vehicle entering logistics park, storage medium and terminal | |
| CN115830399A (en) | Classification model training method, apparatus, device, storage medium, and program product | |
| CN112529116B (en) | Scene element fusion processing method, device and equipment and computer storage medium | |
| CN113111689A (en) | Sample mining method, device, equipment and storage medium | |
| CN113344121A (en) | Method for training signboard classification model and signboard classification | |
| CN114639076A (en) | Target object detection method, target object detection device, storage medium, and electronic device | |
| CN114238740A (en) | Method and device for determining agent brand of agent main body | |
| CN113609956B (en) | Training method, recognition device, electronic equipment and storage medium | |
| CN116958176B (en) | Image segmentation method, device, computer equipment and medium | |
| US20240135705A1 (en) | Utilizing machine learning models to classify vehicle trajectories and collect road use data in real-time | |
| CN117194698B (en) | Task processing system and method based on OAR semantic knowledge base | |
| CN117218515B (en) | Target detection method, device, computing equipment and storage medium | |
| CN115311690B (en) | End-to-end pedestrian structural information and dependency relationship detection method thereof | |
| Govender et al. | Effect of hyperparameter tuning on classical machine learning models in detecting potholes | |
| CN116958856A (en) | Road section image providing method, device, terminal and storage medium | |
| CN117708379A (en) | Video retrieval method, device, system, electronic equipment and storage medium | |
| CN116959026A (en) | Object detection method, device, apparatus, storage medium and computer program product | |
| CN115481036A (en) | Driving model testing method, device, equipment and medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |