CN112085178B

CN112085178B - Vehicle behavior data labeling method and device

Info

Publication number: CN112085178B
Application number: CN201910510173.0A
Authority: CN
Inventors: 王宇舟
Original assignee: Momenta Suzhou Technology Co Ltd
Current assignee: Momenta Suzhou Technology Co Ltd
Priority date: 2019-06-13
Filing date: 2019-06-13
Publication date: 2022-06-10
Anticipated expiration: 2039-06-13
Also published as: CN112085178A

Abstract

The embodiment of the invention discloses a method and a device for marking vehicle behavior data. The method comprises the following steps: the method comprises the steps of obtaining first vehicle behavior data to be labeled, including vehicle information and road information, calculating statistical characteristics of the vehicle information based on a pre-established target mathematical statistical model, determining target statistical rules met by the statistical characteristics, obtaining behavior types corresponding to the first vehicle behavior data according to the target statistical rules and the road information, labeling the first vehicle behavior data based on the obtained behavior types to obtain corresponding labeling information when the accuracy rates of the behavior types corresponding to all the first vehicle behavior data in an obtained target data set are larger than a preset threshold value, and labeling second vehicle behavior data in a new target data set based on a pre-established target network model to obtain corresponding labeling information when the obtained accuracy rates are not larger than the preset threshold value. By applying the scheme provided by the embodiment of the invention, the marking can be automatically carried out, and the marking efficiency is improved.

Description

Vehicle behavior data labeling method and device

Technical Field

The invention relates to the technical field of intelligent driving, in particular to a method and a device for marking vehicle behavior data.

Background

In the automatic driving scenario, predicting the behavior of other vehicles in advance and giving an alarm can avoid the occurrence of traffic accidents, and therefore, the prediction of the future behavior of other vehicles is very necessary.

At present, the behaviors of other vehicles are mainly predicted in a machine learning mode. In order to establish the behavior prediction machine learning model, a large amount of vehicle behavior data and labeled information labeled on the vehicle behavior data are needed, and for example, when the vehicle information in the vehicle behavior data is left turn signal flashing, and the road information is a left turn road, the labeled information may be left turn of the vehicle.

At present, manual labeling modes such as crowdsourcing and outsourcing are mostly adopted for data labeling work at home and abroad. The above approach requires a significant amount of human resources to be employed to meet the labeling requirements during the machine learning iteration, which is inefficient. In order to solve the efficiency problem in vehicle behavior data labeling, a labeling method for vehicle behavior data, which improves labeling efficiency, is urgently needed.

Disclosure of Invention

The invention provides a method and a device for labeling vehicle behavior data, which are used for improving the efficiency of labeling the vehicle behavior data. The specific technical scheme is as follows.

In a first aspect, an embodiment of the present invention provides a method for labeling vehicle behavior data, where the method includes:

acquiring first vehicle behavior data to be labeled in a target data set, wherein the first vehicle behavior data comprises vehicle information and road information;

for each first vehicle behavior data, calculating the statistical characteristics of vehicle information included in the first vehicle behavior data based on a pre-established target mathematical statistical model, wherein the statistical characteristics reflect the behavior type corresponding to the first vehicle behavior data;

determining a target statistical rule met by the statistical characteristics, and obtaining a behavior type corresponding to the first vehicle behavior data according to the target statistical rule, a preset corresponding relation between the statistical rule and the statistical behavior and the road information;

when the accuracy of the behavior types corresponding to all the first vehicle behavior data in the target data set is larger than a preset threshold value, marking the first vehicle behavior data based on the obtained behavior types to obtain corresponding marking information;

when the accuracy of the behavior types corresponding to all the first vehicle behavior data in the target data set is not larger than a preset threshold value, acquiring second vehicle behavior data in a new target data set, determining the behavior type corresponding to the second vehicle behavior data based on a pre-established target network model, and labeling the second vehicle behavior data based on the determined behavior type to obtain corresponding labeling information,

wherein the target network model is: and training the initial network model to obtain a network model based on the vehicle behavior sample data and the corresponding behavior type as model training data, wherein the target network model is used for enabling the vehicle behavior sample data and the corresponding behavior type to be correlated, and the behavior type corresponding to the vehicle behavior sample data is input by a marker in advance aiming at the vehicle behavior sample data.

Optionally, the step of determining the target statistical rule satisfied by the statistical characteristic, and obtaining the behavior type corresponding to the first vehicle behavior data according to the target statistical rule, the preset corresponding relationship between the statistical rule and the statistical behavior, and the road information includes:

determining a target statistical rule met by the statistical characteristics, and determining a statistical behavior corresponding to the target statistical rule according to a preset corresponding relation between the statistical rule and the statistical behavior;

judging whether the statistical behavior is matched with the road information;

and if so, determining the behavior type corresponding to the first vehicle behavior data as the statistical behavior.

Optionally, the training process of the target network model includes:

obtaining vehicle behavior sample data in a training set;

receiving a behavior type input by a annotator aiming at the vehicle behavior sample data;

and taking the vehicle behavior sample data and the corresponding behavior type as model training data, and training the initial network model to obtain a target network model, wherein the target network model is used for enabling the vehicle behavior sample data to be associated with the corresponding behavior type.

Optionally, the step of training an initial network model by using the vehicle behavior sample data and the corresponding behavior type as model training data to obtain a target network model includes:

inputting the vehicle behavior sample data and the corresponding behavior type into an initial network model, wherein the initial network model comprises a feature extraction layer and a regression layer;

determining a feature vector in the vehicle behavior sample data through the first model parameter of the feature extraction layer, and performing regression on the feature vector through the second model parameter of the regression layer to obtain an initial behavior type;

calculating a difference value between the initial behavior type and a behavior type corresponding to the vehicle behavior sample data;

when the difference value is larger than a preset difference threshold value, adjusting the first model parameter and the second model parameter according to the difference value, and returning to execute the step of inputting the vehicle behavior sample data and the corresponding behavior type into an initial network model;

and when the difference value is not greater than a preset difference threshold value, determining that the training of the initial network model is finished to obtain a target network model.

Optionally, the step of obtaining the first vehicle behavior data to be labeled in the target data set includes:

receiving a target data set selected by a user from a preset data set;

and receiving a target data set selected by a user from the target data set, and taking the vehicle behavior data included in the target data set as first vehicle behavior data to be labeled.

Optionally, after the step of labeling the second vehicle behavior data based on the determined behavior type to obtain corresponding labeling information, the method further includes:

storing the second vehicle behavior data and corresponding labeling information;

detecting whether the unmarked vehicle behavior data exist in the new target data set or not;

and if the target data set exists, selecting a data set from the new target data set according to a preset data set interval rule, taking the vehicle behavior data included in the selected data set as second vehicle behavior data, returning to execute the step of determining the behavior type corresponding to the second vehicle behavior data based on the pre-established target network model.

Optionally, after the step of detecting whether there is any unlabeled vehicle behavior data in the new target data set, the method further includes:

when the unlabelled vehicle behavior data do not exist, determining a target data set in which the new target data set is located;

selecting a data set from the determined target data set according to a preset selection rule;

and receiving a data set selected by a user from the selected data set, taking the vehicle behavior data included in the selected data set as the second vehicle behavior data, and returning to the step of executing the step of determining the behavior type corresponding to the second vehicle behavior data based on the pre-established target network model.

and highlighting the vehicle behavior corresponding to the marking information in a vehicle behavior list.

In a second aspect, an embodiment of the present invention provides a device for labeling vehicle behavior data, including:

the system comprises an acquisition module, a storage module and a display module, wherein the acquisition module is used for acquiring first vehicle behavior data to be labeled in a target data set, and the first vehicle behavior data comprises vehicle information and road information;

the calculation module is used for calculating the statistical characteristics of the vehicle information included in the first vehicle behavior data based on a pre-established target mathematical statistical model for each first vehicle behavior data, wherein the statistical characteristics reflect the behavior type corresponding to the first vehicle behavior data;

the behavior type determining module is used for determining a target statistical rule met by the statistical characteristics and obtaining a behavior type corresponding to the first vehicle behavior data according to the target statistical rule, a preset corresponding relation between the statistical rule and the statistical behavior and the road information;

the first labeling module is used for labeling the first vehicle behavior data based on the obtained behavior types to obtain corresponding labeling information when the accuracy of the behavior types corresponding to all the first vehicle behavior data in the obtained target data set is greater than a preset threshold;

a second labeling module, configured to, when the accuracy of the behavior types corresponding to all first vehicle behavior data in the obtained target data set is not greater than a preset threshold, obtain second vehicle behavior data in a new target data set, determine a behavior type corresponding to the second vehicle behavior data based on a pre-established target network model, and label the second vehicle behavior data based on the determined behavior type to obtain corresponding labeling information,

Optionally, the behavior type determining module includes:

the statistical behavior determining submodule is used for determining a target statistical rule met by the statistical characteristics and determining a statistical behavior corresponding to the target statistical rule according to a preset corresponding relation between the statistical rule and the statistical behavior;

the judging submodule is used for judging whether the statistical behavior is matched with the road information or not, and if so, the first determining submodule is triggered;

the first determining submodule is used for determining that the behavior type corresponding to the first vehicle behavior data is the statistical behavior.

Optionally, the method further includes a training module, where the training module is configured to train to obtain the target network model, and the training module includes:

the acquisition submodule is used for acquiring vehicle behavior sample data in a training set;

the receiving submodule is used for receiving the behavior type input by the annotator aiming at the vehicle behavior sample data;

and the target network model generation submodule is used for training the initial network model by taking the vehicle behavior sample data and the corresponding behavior type as model training data to obtain a target network model, wherein the target network model is used for enabling the vehicle behavior sample data and the corresponding behavior type to be mutually associated.

Optionally, the target network model generation sub-module includes:

the input unit is used for inputting the vehicle behavior sample data and the corresponding behavior type into an initial network model, wherein the initial network model comprises a feature extraction layer and a regression layer;

the initial labeling information generating unit is used for determining a feature vector in the vehicle behavior sample data through the first model parameter of the feature extraction layer, and regressing the feature vector through the second model parameter of the regression layer to obtain an initial behavior type;

the difference value calculating unit is used for calculating a difference value between the initial behavior type and the behavior type corresponding to the vehicle behavior sample data;

the adjusting unit is used for adjusting the first model parameter and the second model parameter according to the difference value and triggering the input unit when the difference value is larger than a preset difference threshold value;

and the training completion unit is used for determining that the initial network model is trained to be completed to obtain a target network model when the difference value is not greater than a preset difference threshold value.

Optionally, the obtaining module includes:

the target data set receiving submodule is used for receiving a target data set selected by a user from a preset data set;

and the target vehicle behavior data determining submodule is used for receiving a target data group selected by a user from the target data set and taking the vehicle behavior data included in the target data group as first vehicle behavior data to be labeled.

Optionally, the apparatus further comprises:

the storage module is used for storing the second vehicle behavior data and the corresponding marking information after the second vehicle behavior data are marked based on the determined behavior types to obtain the corresponding marking information;

the detection module is used for detecting whether the unmarked vehicle behavior data exist in the new target data set or not, and if so, the selection module is triggered;

the selecting module is used for selecting a data set from the new target data set according to a preset data set interval rule, using vehicle behavior data included in the selected data set as second vehicle behavior data, and triggering the second labeling module.

Optionally, the apparatus further comprises:

a target data set determining module, configured to determine, after the detecting whether the unlabeled vehicle behavior data exists in the new target data set, a target data set in which the new target data set exists when the unlabeled vehicle behavior data does not exist;

the data set selection module is used for selecting a data set from the determined target data set according to a preset selection rule;

and the receiving module is used for receiving a data set selected from the selected data set by the user, taking the vehicle behavior data included in the selected data set as the second vehicle behavior data, and triggering the second labeling module.

Optionally, the apparatus further comprises:

and the display module is used for highlighting and displaying the vehicle behavior corresponding to the marking information in a vehicle behavior list after the marking is carried out on the second vehicle behavior data based on the determined behavior type to obtain the corresponding marking information.

As can be seen from the above, in this embodiment, first vehicle behavior data to be labeled including vehicle information and road information may be obtained, for each first vehicle behavior data, first, a statistical characteristic of the vehicle information is calculated based on a pre-established target mathematical statistical model, then, a target statistical rule satisfied by the statistical characteristic is determined, a behavior type corresponding to the first vehicle behavior data is obtained according to the target statistical rule, a preset corresponding relationship between the statistical rule and the statistical behavior, and the road information, when an accuracy rate of a behavior type corresponding to all first vehicle behavior data in an obtained target data set is greater than a preset threshold value, the first vehicle behavior data is labeled based on the obtained behavior type to obtain corresponding labeling information, when an accuracy rate of a behavior type corresponding to all first vehicle behavior data in the obtained target data set is not greater than the preset threshold value, and acquiring second vehicle behavior data in the new target data set, determining a behavior type corresponding to the second vehicle behavior data based on a pre-established target network model, and labeling the second vehicle behavior data based on the determined behavior type to obtain corresponding labeling information. Therefore, the statistical characteristics are calculated through the target mathematical statistical model, the marking is carried out based on the statistical characteristics, and when the accuracy requirement cannot be met on the target data set in a statistical data-based mode, the new target data set is marked through the target network model, so that the aim of automatically marking the vehicle behavior data is fulfilled. Of course, not all of the advantages described above need to be achieved at the same time in the practice of any one product or method of the invention.

The innovation points of the embodiment of the invention comprise:

1. the statistical characteristics are calculated through the target mathematical statistical model, the marking is carried out based on the statistical characteristics, and when the accuracy requirement cannot be met on the target data set based on the statistical data, the marking is carried out on a new target data set through the target network model, the purpose of automatically marking the vehicle behavior data is achieved, compared with the mode of marking through manpower, the automatic marking by a machine can save manpower, and the marking efficiency is improved.

2. The statistical behavior corresponding to the target statistical rule is determined by determining the target statistical rule met by the statistical characteristics and according to the preset corresponding relation between the statistical rule and the statistical behavior, the behavior type corresponding to the first vehicle behavior data is obtained based on the statistical behavior and the road information, and then the first vehicle behavior data is labeled based on the behavior type corresponding to the first vehicle behavior data, so that the purpose of automatically labeling based on the statistical characteristics is achieved.

3. The initial network model is trained to obtain a target network model used for enabling vehicle behavior sample data and corresponding behavior types to be mutually associated, the behavior type corresponding to second vehicle behavior data can be obtained through the target network model, then the second vehicle behavior data is labeled based on the behavior type to obtain corresponding labeling information, and the target network model is obtained by training based on a large amount of vehicle behavior sample data, so that the accuracy rate of the target network model is higher than that of the behavior types obtained by a statistical characteristic-based mode, and the purposes of automatically labeling the vehicle behavior data and higher labeling accuracy rate are achieved.

4. Through the mode of highlighting, make the user can be striking look over the vehicle action that has marked, be convenient for the user to distinguish the vehicle action, also promoted user's concentration degree.

Drawings

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the embodiments or the prior art descriptions will be briefly described below. It is to be understood that the drawings in the following description are merely exemplary of some embodiments of the invention. For a person skilled in the art, without inventive effort, further figures can be obtained from these figures.

Fig. 1 is a schematic flow chart of a method for labeling vehicle behavior data according to an embodiment of the present invention;

FIG. 2 is a schematic flow chart of step S130 in FIG. 1;

FIG. 3 is a flowchart illustrating a process of training a target network model according to an embodiment of the present invention;

FIG. 4 is a schematic interface diagram of the annotation software according to the embodiment of the present invention;

fig. 5 is a schematic structural diagram of a vehicle behavior data labeling apparatus according to an embodiment of the present invention.

Detailed Description

The technical solution in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. It is to be understood that the described embodiments are merely a few embodiments of the invention, and not all embodiments. All other embodiments, which can be obtained by a person skilled in the art without inventive effort based on the embodiments of the present invention, are within the scope of the present invention.

It is to be noted that the terms "comprises" and "comprising" and any variations thereof in the embodiments and drawings of the present invention are intended to cover non-exclusive inclusions. For example, a process, method, system, article, or apparatus that comprises a list of steps or elements is not limited to only those steps or elements listed, but may alternatively include other steps or elements not listed, or inherent to such process, method, article, or apparatus.

The embodiment of the invention discloses a method and a device for marking vehicle behavior data, which can automatically mark the vehicle behavior data without manual marking, and improve marking efficiency. The following provides a detailed description of embodiments of the invention.

Fig. 1 is a schematic flow chart of a method for labeling vehicle behavior data according to an embodiment of the present invention. The method is applied to the electronic equipment. The method may specifically include the following steps S110 to S150.

S110: and acquiring first vehicle behavior data to be labeled in the target data set.

In order to label the vehicle behavior data, first vehicle behavior data to be labeled in the target data set needs to be acquired.

Step S110 may specifically include:

receiving a target data set selected by a user from a preset data set; and receiving a target data group selected from the target data set by a user, and taking the vehicle behavior data included in the target data group as first vehicle behavior data to be labeled.

Generally, the vehicle behavior data is collected in real time and continuously, and one data set is generated by one collection, and the time period of one collection is not fixed, and may be 4 hours, 8 hours, or 24 hours, so that the continuous collection generates a plurality of data sets, forming a data set, for example: assuming that the time period of one-time collection is one day, ten data sets are generated by collecting the behavior of the vehicle passing through the intersection a within ten days to form a data set a, and ten data sets, a shape data set B, are generated by collecting the behavior of the vehicle passing through the intersection B within ten days.

Since many vehicles pass through the collected roads during each collection, each data set includes vehicle behavior data of many vehicles and corresponding time period information. Wherein the collected vehicle behavior data for each vehicle may include: vehicle information and road information.

Illustratively, the vehicle information may include at least one of 3D coordinate information, angular velocity, linear velocity, acceleration, turn signal information, brake signal information, or double jump signal information.

The road information may include at least one of a straight road, a left turn road, a right turn road, an intersection, a ramp junction, a parking space, zebra stripes, or red street light information.

Since each data set includes vehicle behavior data of a plurality of vehicles and corresponding time period information, in order to label the vehicle behavior data of each vehicle, one data set may be divided into a plurality of data sets, each data set may include vehicle behavior data of one vehicle, corresponding time period information, and label data, where the vehicle behavior data may include observation information and corresponding future information, for example: and if the time period position corresponding to the vehicle behavior data is 10s, the observation information is the vehicle behavior data corresponding to 1-5s, the future information is the vehicle behavior data corresponding to 6-10s, and when the vehicle behavior data included in the data group is not marked, the marked data is empty and has no numerical value.

Since the vehicle behavior data is labeled to build the behavior prediction machine learning model, a large amount of vehicle behavior data needs to be labeled. Therefore, when the vehicle behavior data is labeled, a preset data set exists, the user can select a data set to be labeled from the preset data set, and at this time, the electronic device receives a target data set selected by the user from the preset data set.

The electronic device receives the target data set selected by the user from the target data set, and uses the vehicle behavior data included in the target data set as first vehicle behavior data to be labeled, wherein the first vehicle behavior data includes vehicle information and road information.

S120: for each first vehicle behavior data, a statistical feature of vehicle information included in the first vehicle behavior data is calculated based on a target mathematical statistical model established in advance.

Since the essence of labeling the first vehicle behavior data is to label what behavior type the movement track of the first vehicle behavior data corresponds to in the corresponding time period, after the first vehicle behavior data is obtained, for each first vehicle behavior data, the behavior type corresponding to the first vehicle behavior data needs to be determined. For example, the determining of the behavior type corresponding to the first vehicle behavior data may be determining the behavior type corresponding to future information included in the first vehicle behavior data.

Since most driving behaviors have certain rules and patterns on data values, the behavior type can be determined by establishing a mathematical model of the driving behavior, i.e., vehicle information. Illustratively, the mathematical model may be a statistical model based on mean, variance, standard deviation, or cross entropy. Thus, a target mathematical statistical model is established in advance based on the vehicle information.

After the first vehicle behavior data are obtained, for each first vehicle behavior data, calculating the statistical characteristics of the vehicle information included in the first vehicle behavior data based on a pre-established target mathematical statistical model, wherein the statistical characteristics reflect the behavior type corresponding to the first vehicle behavior data. Illustratively, the statistical feature may be one of a mean, a variance, a standard deviation, or a cross entropy.

In one implementation, when the vehicle information includes an angular velocity and a linear velocity, and the statistical characteristic is an average value, the step of calculating the statistical characteristic of the vehicle information included in the first vehicle behavior data based on a pre-established target mathematical statistical model may include:

and calculating the average value of the angular speed in the corresponding time period and the average value of the linear speed in the corresponding time period based on the time period information corresponding to the first vehicle behavior data.

S130: and determining a target statistical rule met by the statistical characteristics, and obtaining a behavior type corresponding to the first vehicle behavior data according to the target statistical rule, a preset corresponding relation between the statistical rule and the statistical behavior and the road information.

Since the choice of behavior type may vary in different application scenarios, for example: in the expressway scene, the behavior types can comprise an entrance ramp behavior, an exit ramp behavior, a lane change behavior, a lane snatching behavior and an overtaking behavior; in the urban road scene, the behavior types may include parking behavior, straight behavior, left turn behavior, right turn behavior, reverse behavior, traffic light behaviors such as traffic lights, and obstacle avoidance behavior. And because the road information can reflect the application scene, after the statistical characteristics are calculated, the target statistical rule met by the statistical characteristics needs to be determined, and the behavior type corresponding to the first vehicle behavior data is obtained according to the target statistical rule, the preset corresponding relation between the statistical rule and the statistical behavior and the road information.

Referring to fig. 2, step S130 may include the steps of:

s1301: and determining a target statistical rule met by the statistical characteristics, and determining a statistical behavior corresponding to the target statistical rule according to a preset corresponding relation between the statistical rule and the statistical behavior.

In order to determine the statistical behavior, a corresponding relation between the statistical rule and the statistical behavior is preset, after the statistical characteristic is calculated, the target statistical rule met by the statistical characteristic can be determined, and then the statistical behavior corresponding to the target statistical rule is determined according to the preset corresponding relation between the statistical rule and the statistical behavior.

In one implementation, when the vehicle information includes an angular velocity and a linear velocity, and the statistical characteristic is an average value, the step of determining a target statistical rule that the statistical characteristic satisfies, and determining a statistical behavior corresponding to the target statistical rule according to a preset corresponding relationship between the statistical rule and the statistical behavior may include:

determining the statistical behavior corresponding to the target statistical rule according to a preset formula:

wherein, Behavior_statFor statistical behavior, FORWARD is FORWARD behavior, YIELD is YIELD behavior, RIGHT is RIGHT behavior, w_meanIs the average value of angular velocity, V_meanIs the average linear velocity, K_wAs statistical coefficient of angular velocity, K_vIs the linear velocity statistical coefficient, w is the angular velocity, v is the linear velocity.

S1302: and judging whether the statistical behavior is matched with the road information or not, and if so, executing the step S1303.

Because the behavior types are selected differently in different application scenarios and the road information can reflect the application scenario, after the statistical behavior corresponding to the target statistical rule is determined, it is necessary to determine whether the statistical behavior matches the road information, and perform subsequent steps according to the determination result.

For example: when the right-turn behavior of the statistical behavior corresponding to the target statistical rule is determined and the road information is a right-turn road, the step of determining whether the statistical behavior is matched with the road information may include:

and judging whether the right turning behavior is matched with the right turning lane.

S1303: and determining the behavior type corresponding to the first vehicle behavior data as statistical behavior.

When the statistical behavior is judged to be matched with the road information, it is described that the vehicle corresponding to the target vehicle behavior data normally runs in the scene corresponding to the road information, and at this time, the statistical behavior may be used as the behavior type corresponding to the first vehicle behavior data.

For example: in the example of step S1302, it is determined that the right-turn behavior matches the right-turn lane, and the right-turn behavior is used as the behavior type corresponding to the first vehicle behavior data.

And when the statistical behavior is judged not to be matched with the road information, the vehicle corresponding to the target vehicle behavior data is not normally driven in the scene corresponding to the road information, and at the moment, no processing is performed.

Therefore, the statistical behavior corresponding to the target statistical rule is determined by determining the target statistical rule met by the statistical characteristics and according to the preset corresponding relation between the statistical rule and the statistical behavior, the behavior type corresponding to the first vehicle behavior data is obtained based on the statistical behavior and the road information, and then the first vehicle behavior data is labeled based on the behavior type corresponding to the first vehicle behavior data, so that the purpose of automatically labeling based on the statistical characteristics is achieved. This is one of the innovative points of the embodiments of the present invention.

S140: and when the accuracy of the behavior types corresponding to all the first vehicle behavior data in the obtained target data set is greater than a preset threshold value, labeling the first vehicle behavior data based on the obtained behavior types to obtain corresponding labeling information.

Since an error may occur in the process of obtaining the behavior type based on the statistical characteristics, which may cause the obtained behavior type to be inaccurate, after obtaining the behavior types corresponding to all the first vehicle behavior data in the target data set, the accuracy rates of the behavior types corresponding to all the first vehicle behavior data need to be determined.

The electronic equipment determines the accuracy by calculating the proportion of the number of the corrected behavior types to the total number of the first vehicle behavior data.

When the accuracy is greater than the preset threshold, it is indicated that the proportion of correct behavior types in the obtained behavior types is large, and the accuracy is within an acceptable degree, and at this time, the first vehicle behavior data may be labeled based on the obtained behavior types to obtain corresponding labeling information.

The marking information obtained by marking the first vehicle behavior data based on the obtained behavior type may be the marking information corresponding to the obtained behavior type as the first vehicle behavior data.

For example: in the example of the adapting step S1303, the right-turn behavior is used as the label information corresponding to the first vehicle behavior data.

S150: when the accuracy of the behavior types corresponding to all the first vehicle behavior data in the obtained target data set is not larger than a preset threshold value, obtaining second vehicle behavior data in a new target data set, determining the behavior type corresponding to the second vehicle behavior data based on a pre-established target network model, and labeling the second vehicle behavior data based on the determined behavior type to obtain corresponding labeling information.

Wherein, the target network model is: and training the initial network model to obtain a network model based on the vehicle behavior sample data and the corresponding behavior type as model training data, wherein the target network model is used for enabling the vehicle behavior sample data and the corresponding behavior type to be correlated with each other, and the behavior type corresponding to the vehicle behavior sample data is input by a marker in advance aiming at the vehicle behavior sample data.

When the accuracy is not greater than the preset threshold, it is indicated that the proportion of correct behavior types in the obtained behavior types is small, and if the behavior types corresponding to the second vehicle behavior data in the new target data set are still determined in this way, a great number of wrong behavior types are obtained, so that, for this situation, the embodiment of the invention establishes the target network model in advance, determines the behavior types corresponding to the second vehicle behavior data through the pre-established target network model after obtaining the second vehicle behavior data in the new target data set, and labels the second vehicle behavior data based on the determined behavior types to obtain corresponding labeling information.

Referring to fig. 3, the training process of the target network model may be:

s301: vehicle behavior sample data in a training set is obtained.

When a target network model is established, vehicle behavior sample data in a training set needs to be acquired.

S302: and receiving the behavior type input by the annotator aiming at the vehicle behavior sample data.

After the vehicle behavior sample data is obtained, the vehicle behavior sample data is labeled in a manual mode, and the electronic equipment receives the behavior type input by a label operator aiming at the vehicle behavior sample data.

S303: and taking the vehicle behavior sample data and the corresponding behavior type as model training data, and training the initial network model to obtain a target network model.

After receiving the behavior type input by the annotator aiming at the vehicle behavior sample data, the vehicle behavior sample data and the corresponding behavior type can be used as model training data to train the initial network model to obtain a target network model, wherein the target network model is used for enabling the vehicle behavior sample data to be correlated with the corresponding behavior type.

Step S303 may specifically include:

inputting vehicle behavior sample data and a corresponding behavior type into an initial network model, wherein the initial network model comprises a feature extraction layer and a regression layer;

determining a feature vector in the vehicle behavior sample data through a first model parameter of the feature extraction layer, and performing regression on the feature vector through a second model parameter of the regression layer to obtain an initial behavior type;

when the difference value is larger than a preset difference threshold value, adjusting the first model parameter and the second model parameter according to the difference, and returning to execute the step of inputting the vehicle behavior sample data and the corresponding behavior type into the initial network model;

and when the difference value is not greater than the preset difference threshold value, determining that the training of the initial network model is finished, and obtaining the target network model.

It can be understood that the electronic device first needs to construct an initial network model and then train the initial network model to obtain the target network model. In one implementation, an initial network model including a feature extraction layer and a regression layer may be constructed using a caffe tool. For example, the initial Network model may be an SVM (Support Vector Machine) RNN (Recurrent Neural Network), LSTM (Long Short-Term Memory Network), GRU (Gated Recurrent Unit), or the like.

After the initial network model is built, vehicle behavior sample data and corresponding behavior types are input into the initial network model for training.

Specifically, vehicle behavior sample data is input into the feature extraction layer, and a feature vector in the vehicle behavior sample data is determined through a first model parameter of the feature extraction layer. And then inputting the determined feature vector into a regression layer, and performing regression on the feature vector through a second model parameter of the regression layer to obtain an initial behavior type.

After the initial behavior type is obtained, comparing the initial behavior type with the behavior type corresponding to the vehicle behavior sample data, calculating a difference value between the initial behavior type and the behavior type through a predefined objective function, and when the difference value is larger than a preset difference threshold value, indicating that the initial network model at the moment is not suitable for most vehicle behavior data, adjusting the first model parameter and the second model parameter through a back propagation method according to the difference value, and returning to execute the step of inputting the vehicle behavior sample data and the corresponding behavior type into the initial network model.

In the training process, all vehicle behavior sample data can be circularly traversed, and the first model parameter and the second model parameter of the initial network model are continuously adjusted. And when the difference value is not greater than the preset difference threshold value, the initial network model at the moment can be suitable for most vehicle behavior data to obtain an accurate result, and at the moment, the training completion of the initial network model is determined to obtain the target network model. It will be appreciated that the trained target network model is used to correlate vehicle behaviour sample data with corresponding behaviour types.

In addition, in the model training process, if errors occur, the errors can be corrected in a manual mode, and then the corrected behavior types are input into the initial network model, so that the accuracy of model training is improved in the manual correction mode.

It can be seen that the initial network model is trained through the training mode, a target network model used for enabling vehicle behavior sample data and corresponding behavior types to be mutually associated can be obtained, the behavior type corresponding to second vehicle behavior data can be obtained through the target network model, then the second vehicle behavior data is labeled based on the behavior type to obtain corresponding labeling information, and the target network model is obtained through training based on a large amount of vehicle behavior sample data, so that the accuracy rate of the target network model is higher than that of the behavior types obtained through a statistical characteristic-based mode, and the purposes that the vehicle behavior data can be automatically labeled and the labeling accuracy rate is higher are achieved. This is one of the innovative points of the embodiment of the present invention.

Generally, the above target network model works well for some common and easily distinguishable scenarios, such as: a straight-road driving scene, a lane-changing driving scene, a curve turning scene, a ramp-in scene, a ramp-out scene, a multi-lane ramp-in scene, a multi-lane ramp-out scene, a traffic light scene such as an intersection, a roadside parking scene, a lane-robbing scene or a sudden braking scene, etc.

For the situation that more than three vehicles interact or the vehicle behaviors are not easy to distinguish, a deep neural network with stronger expression capacity can be used during training, namely, an initial network model can be constructed as the deep neural network model.

As can be seen from the above, in this embodiment, first vehicle behavior data to be labeled including vehicle information and road information may be obtained, for each first vehicle behavior data, first, a statistical characteristic of the vehicle information is calculated based on a pre-established target mathematical statistical model, then, a target statistical rule satisfied by the statistical characteristic is determined, a behavior type corresponding to the first vehicle behavior data is obtained according to the target statistical rule, a preset corresponding relationship between the statistical rule and the statistical behavior, and the road information, when an accuracy rate of a behavior type corresponding to all first vehicle behavior data in an obtained target data set is greater than a preset threshold value, the first vehicle behavior data is labeled based on the obtained behavior type to obtain corresponding labeling information, when an accuracy rate of a behavior type corresponding to all first vehicle behavior data in the obtained target data set is not greater than the preset threshold value, and acquiring second vehicle behavior data in the new target data set, determining a behavior type corresponding to the second vehicle behavior data based on a pre-established target network model, and labeling the second vehicle behavior data based on the determined behavior type to obtain corresponding labeling information. Therefore, the statistical characteristics are calculated through the target mathematical statistical model, the marking is carried out based on the statistical characteristics, and when the accuracy requirement cannot be met on the target data set in a statistical data-based mode, the new target data set is marked through the target network model, so that the aim of automatically marking the vehicle behavior data is fulfilled.

In another embodiment of the present invention, on the basis of the method shown in fig. 1, after step S150, the method for annotating vehicle behavior data provided by the embodiment of the present invention may further include:

and storing the second vehicle behavior data and the corresponding marking information.

if the target data group exists, selecting a data group from the new target data set according to a preset data group interval rule, taking the vehicle behavior data included in the selected data group as second vehicle behavior data, and returning to execute the step of determining the behavior type corresponding to the second vehicle behavior data based on a preset target network model;

if the unlabelled vehicle behavior data do not exist, determining a target data set in which the new target data set is located;

and receiving a data set selected by a user from the selected data set, taking the vehicle behavior data included in the selected data set as second vehicle behavior data, and returning to execute the step of determining the behavior type corresponding to the second vehicle behavior data based on a pre-established target network model.

After the second vehicle behavior data is labeled, the obtained labeling information may be repeatedly used as model training data to construct various types of network models for labeling, so that the second vehicle behavior data and the corresponding labeling information need to be saved after the second vehicle behavior data is labeled.

In order to facilitate subsequent repeated utilization, the second vehicle behavior data and the corresponding labeling information can be stored in a universal format, so that the second vehicle behavior data and the corresponding labeling information can be repeatedly used for different application scenes, and the data utilization rate is improved. This is one of the innovative points of the embodiments of the present invention.

After the second vehicle behavior data is labeled, other vehicle data can be labeled continuously, at this time, it is necessary to detect whether the new target data set has the vehicle behavior data that is not labeled, and execute corresponding steps according to the detection result.

And when the unmarked vehicle behavior data exist, indicating that the new target data set also contains the vehicle behavior data which can be marked, selecting a data set from the new target data set according to a preset data set interval rule, taking the vehicle behavior data included in the selected data set as second vehicle behavior data, returning to execute the step of determining the behavior type corresponding to the second vehicle behavior data based on the pre-established target network model.

The preset data group interval rule may be: every 0.1s interval, preset second intervals, such as 0.1s interval, 0.2s interval, 0.5s interval, then 0.1s interval, 0.2s interval, 0.5s interval … …, every frame interval, preset frame intervals, such as 1 frame interval, 2 frame interval, 5 frame interval, then 1 frame interval, 2 frame interval, 5 frame interval … …, which is not limited in this embodiment of the invention.

Of course, when there is any unlabeled vehicle behavior data, a data set may be selected from the new target data set by manual selection.

Therefore, when the second vehicle behavior data is labeled and the unmarked vehicle behavior data exists in the new target data set, other vehicle behavior data in the new target data set are continuously labeled in a cyclic searching mode in the new target data set, and the purpose of labeling all vehicle behavior data in the new target data set is achieved.

When the vehicle behavior data which are not labeled do not exist, it is indicated that all the vehicle behavior data in the new target data set are labeled, at this time, a target data set where the new target data set is located can be determined, a data set is selected from the determined target data set according to a preset selection rule, a data group selected from the selected data set by a user is received, the vehicle behavior data included in the selected data set are used as second vehicle behavior data, and the step of determining the behavior type corresponding to the second vehicle behavior data based on a pre-established target network model is returned to be executed.

Wherein, the preset selection rule can be as follows: the numbering is in the order of small to large, and the embodiment of the present invention is not limited in any way.

Therefore, when the second vehicle behavior data is labeled and the target data set does not have the vehicle behavior data which is not labeled, the vehicle behavior data in other data sets in the target data set are continuously labeled in a cyclic searching mode in the target data set, and the purpose of labeling the vehicle behavior data except for the new target data set is achieved.

and highlighting the vehicle behavior corresponding to the marking information in the vehicle behavior list.

For convenience of viewing, in one implementation, after the annotation information is obtained, the vehicle behavior corresponding to the annotation information may be highlighted in the vehicle behavior list.

The highlighting manner may be various, and the vehicle behavior corresponding to the label information may be highlighted, and the vehicle behavior corresponding to the label information is selected by using a frame.

In another implementation manner, the vehicle behavior corresponding to the obtained labeled information and the vehicle behavior corresponding to the un-obtained labeled information may be displayed in different highlighting manners, and for convenience of description, the labeled vehicle behavior and the un-labeled vehicle behavior are referred to below for short. For example: the marked vehicle behavior is shown in yellow and the unmarked vehicle behavior is shown in red.

Therefore, the marked vehicle behaviors can be checked and seen by the user in a highlighting mode, the user can distinguish the vehicle behaviors conveniently, and the concentration degree of the user is improved. This is one of the innovative points of the embodiments of the present invention.

Based on the above method for labeling vehicle behavior data, an embodiment of the present invention further provides a labeling software, referring to fig. 4, where an interface of the labeling software is shown in fig. 4, and the method includes: loading a data frame window, a storage window, a vehicle ID selection window, a data visualization window, a behavior list window and a selection frame window.

The following describes the usage flow of the software:

after the software is started, a user selects a data set to be marked by clicking a loading data frame, and the software loads the selected data set;

and the user selects the data group to be marked according to the selection frame by clicking, and the software receives the selected data group and displays the information contained in the data group in a data visualization window in a visualized manner. The pose information of the vehicle can be displayed as a 3D model, and the speed, the acceleration, the ground information, the car light information and the like can be displayed through the suspension window;

the user selects the vehicle to be marked through the vehicle ID selection window, the software acquires the vehicle behavior data corresponding to the vehicle ID, marks the vehicle behavior data through the marking method of the vehicle behavior data provided by the embodiment of the invention to obtain the marking information, and displays the vehicle behavior corresponding to the marking information in the behavior list in a highlighting mode.

Through the software, under different application scenes, marking of the vehicle behavior data can be completed within 3 seconds on average, compared with manual marking, the efficiency is improved, and meanwhile, the accuracy can reach 40% -65%.

Fig. 5 is a schematic structural diagram of a vehicle behavior data labeling apparatus according to an embodiment of the present invention. The apparatus may include:

an obtaining module 510, configured to obtain first vehicle behavior data to be labeled in a target data set, where the first vehicle behavior data includes vehicle information and road information;

a calculating module 520, configured to calculate, for each first vehicle behavior data, a statistical characteristic of vehicle information included in the first vehicle behavior data based on a pre-established target mathematical statistical model, where the statistical characteristic reflects a behavior type corresponding to the first vehicle behavior data;

a behavior type determining module 530, configured to determine a target statistical rule that is satisfied by the statistical characteristic, and obtain a behavior type corresponding to the first vehicle behavior data according to the target statistical rule, a preset correspondence between the statistical rule and the statistical behavior, and the road information;

the first labeling module 540 is configured to label the first vehicle behavior data based on the obtained behavior types to obtain corresponding labeling information when the accuracy of the behavior types corresponding to all the first vehicle behavior data in the obtained target data set is greater than a preset threshold;

a second labeling module 550, configured to, when the accuracy of the behavior types corresponding to all the first vehicle behavior data in the obtained target data set is not greater than a preset threshold, obtain second vehicle behavior data in a new target data set, determine a behavior type corresponding to the second vehicle behavior data based on a pre-established target network model, and label the second vehicle behavior data based on the determined behavior type to obtain corresponding labeling information,

In another embodiment of the present invention, the behavior type determining module 530 may include:

In another embodiment of the present invention, the present invention further includes a training module, where the training module is configured to train to obtain the target network model, and the training module includes:

In another embodiment of the present invention, the target network model generation sub-module may include:

In another embodiment of the present invention, the obtaining module 510 may include:

In another embodiment of the present invention, the apparatus may further include:

The above device embodiment corresponds to the method embodiment, and has the same technical effect as the method embodiment, and for the specific description, refer to the method embodiment. The device embodiment is obtained based on the method embodiment, and for specific description, reference may be made to the method embodiment section, which is not described herein again.

Those of ordinary skill in the art will understand that: the figures are merely schematic representations of one embodiment, and the blocks or flow diagrams in the figures are not necessarily required to practice the present invention.

Those of ordinary skill in the art will understand that: modules in the devices in the embodiments may be distributed in the devices in the embodiments according to the description of the embodiments, or may be located in one or more devices different from the embodiments with corresponding changes. The modules of the above embodiments may be combined into one module, or further split into multiple sub-modules.

Finally, it should be noted that: the above examples are only intended to illustrate the technical solution of the present invention, but not to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions of the embodiments of the present invention.

Claims

1. A method for labeling vehicle behavior data, comprising:

2. The method of claim 1, wherein the step of determining the target statistical rule satisfied by the statistical characteristics, and obtaining the behavior type corresponding to the first vehicle behavior data according to the target statistical rule, the preset correspondence between the statistical rule and the statistical behavior, and the road information comprises:

judging whether the statistical behavior is matched with the road information;

and if so, determining that the behavior type corresponding to the first vehicle behavior data is the statistical behavior.

3. The method of claim 1, wherein the training process of the target network model is:

obtaining vehicle behavior sample data in a training set;

and training the initial network model by taking the vehicle behavior sample data and the corresponding behavior type as model training data to obtain a target network model, wherein the target network model is used for enabling the vehicle behavior sample data to be correlated with the corresponding behavior type.

4. The method according to claim 3, wherein the step of training an initial network model to obtain a target network model using the vehicle behavior sample data and the corresponding behavior type as model training data comprises:

5. The method of claim 1, wherein the step of obtaining first vehicle behavior data to be labeled in a target data set comprises:

receiving a target data set selected by a user from a preset data set;

6. The method of claim 1, wherein after the step of labeling the second vehicle behavior data based on the determined behavior type to obtain corresponding labeling information, further comprising:

7. The method of claim 6, wherein after the step of detecting whether there is unlabeled vehicle behavior data in the new target dataset, further comprising:

8. The method of claim 1, wherein after the step of labeling the second vehicle behavior data based on the determined behavior type to obtain corresponding labeling information, further comprising:

9. A device for labeling vehicle behavior data, comprising:

wherein the target network model is: and training the initial network model to obtain a network model based on the vehicle behavior sample data and the corresponding marking information as model training data, wherein the target network model is used for enabling the vehicle behavior sample data to be correlated with the corresponding behavior type, and the behavior type corresponding to the vehicle behavior sample data is input by a marker in advance aiming at the vehicle behavior sample data.

10. The apparatus of claim 9, wherein the behavior type determination module comprises: