CN112346999A

CN112346999A - Scene-independent unmanned driving simulation test evaluation method and device

Info

Publication number: CN112346999A
Application number: CN202110028045.XA
Authority: CN
Inventors: 何丰; 胡大林; 杨强; 丁佳佳
Original assignee: Zhejiang Saimu Technology Co ltd; Beijing Saimu Technology Co ltd
Current assignee: Beijing Saimu Technology Co ltd
Priority date: 2021-01-11
Filing date: 2021-01-11
Publication date: 2021-02-09
Anticipated expiration: 2041-01-11
Also published as: CN112346999B; CN113111003A; CN113111004A; CN113111004B; CN113111003B

Abstract

The embodiment of the invention discloses a scene-independent unmanned simulation test evaluation method and device. According to the method, for each frame of data in to-be-tested data, a safety evaluation score result of the frame is obtained according to a preset RSS model, a regulation evaluation score result of the frame is obtained according to information of a traffic sign object, position information and running information of a to-be-tested vehicle, a comfort evaluation score result is calculated according to acceleration of the to-be-tested vehicle in a target time period according to each target time period in the to-be-tested data, the to-be-tested data is subjected to cycle test until preset simulation times are reached, and all obtained evaluation score results are sorted to obtain a safety evaluation total result, a regulation evaluation total result and a comfort evaluation total result of the to-be-tested vehicle. Therefore, the type of the data-independent scene used for evaluation is not limited to the specific type of scene file, and the simulation test evaluation can be performed on any scene file, so that the compatibility is high and the robustness is good.

Description

Scene-independent unmanned driving simulation test evaluation method and device

Technical Field

The invention relates to the technical field of unmanned simulation, in particular to a scene-independent unmanned simulation test evaluation method and device.

Background

At present, the unmanned simulation technology is generally adopted for testing and evaluating the unmanned algorithm set for the unmanned vehicle. In the process of testing and evaluating the unmanned algorithm by using the unmanned simulation technology, the driving scene of the automatic driving vehicle is simulated by using manually set scene data or scene data acquired aiming at an actual driving scene, and then simulation test evaluation is performed.

The existing unmanned simulation test evaluation method is specially designed for a specific type of scene file, so that the simulation test evaluation can only be implemented based on the specific type of scene file, and is incompatible with other types of scene files, so that the unmanned simulation test evaluation cannot be performed on other scene files, and the robustness is poor.

Disclosure of Invention

The invention provides a scene-independent unmanned driving simulation test evaluation method and device, which can be used for carrying out simulation test evaluation on any scene file and have high compatibility and good robustness. The specific technical scheme is as follows.

In a first aspect, the invention provides a scene-independent unmanned simulation test evaluation method, which comprises the following steps:

receiving data to be tested for unmanned driving simulation test evaluation of a vehicle to be tested;

aiming at each frame of data in the data to be tested, assembling the frame of data according to a preset data assembling rule to generate a plurality of assembling data, inputting each assembling data into a preset RSS model to obtain a safety evaluation score result of the vehicle to be tested in the frame, extracting information of a traffic sign object and position information and driving information of the vehicle to be tested when the traffic sign object appears in the frame of data, judging whether the vehicle to be tested violates rules or not according to the extracted information, and obtaining a regularity evaluation score result of the vehicle to be tested in the frame;

aiming at each target time period in the data to be tested, extracting the acceleration of the vehicle to be tested in the target time period, and calculating a comfort evaluation score result of the vehicle to be tested in the target time period according to the extracted acceleration, wherein each target time period is a time period in which the vehicle to be tested has two continuous acceleration behaviors and has one deceleration behavior after the acceleration behaviors, the starting time point of each target time period is the time point of the first acceleration behavior in the two continuous acceleration behaviors, and the ending time of each target time period is the time point of the deceleration behavior;

and circularly testing the data to be tested until the preset simulation times are reached, sorting all the obtained safety evaluation score results to obtain a total safety evaluation result of the vehicle to be tested, sorting all the obtained regulation evaluation score results to obtain a total regulation evaluation result of the vehicle to be tested, and sorting all the obtained comfort evaluation score results to obtain a total comfort evaluation result of the vehicle to be tested.

Optionally, the step of assembling the frame data according to a preset data assembly rule for each frame of data in the to-be-tested data to generate a plurality of assembly data includes:

and aiming at each frame of data in the data to be tested, extracting the vehicle data to be tested and the target object data in the frame of data, and assembling the vehicle data to be tested and each target object data to generate a plurality of assembling data, wherein the type of the target object data at least comprises the type of the vehicle.

Optionally, the step of inputting each assembly data into a preset RSS model to obtain the safety evaluation score result of the vehicle to be tested in the frame includes:

aiming at each assembly data, calculating an actual distance and a safe distance between a target object and the vehicle to be tested according to the dynamic parameters of the vehicle to be tested and the position information of the target object in the assembly data, wherein the types of the distances comprise a longitudinal distance, a left transverse distance and a right transverse distance;

and calculating a difference value between the actual distance and the safe distance, and taking the actual distance, the safe distance and the difference value as a safety evaluation score result of the vehicle to be tested in the frame.

Optionally, the step of determining whether the vehicle to be tested violates the rule according to the extracted information to obtain a rule evaluation score result of the vehicle to be tested in the frame includes:

when the traffic sign object is a speed limit sign, judging whether the speed of the vehicle to be tested accords with the speed standard shown by the speed limit sign, and if not, determining that the vehicle to be tested does not run according to the speed limit standard;

when the traffic sign object is a traffic signal lamp and a stop line, judging whether the vehicle to be tested runs through a red light or not according to the position information of the vehicle to be tested and the position information of the stop line, and if so, determining that the vehicle to be tested runs through the red light;

when the traffic sign object is a traffic warning sign, judging whether the position of the vehicle to be tested exceeds the position of the traffic warning sign, and if so, determining that the vehicle to be tested runs off-line;

when the traffic sign is a humanoid crosswalk line or a deceleration yielding sign, judging whether the vehicle to be tested stops at the position of the humanoid crosswalk line and starts to run in a preset time period, and if not, determining that the vehicle to be tested runs off the line.

Optionally, the step of calculating a comfort evaluation score result of the vehicle to be tested in the target time period according to the extracted acceleration includes:

and determining the average acceleration, the average deceleration, the deceleration time, the acceleration time, the maximum deceleration, the maximum acceleration, the deceleration number and the acceleration number of the vehicle to be tested in the target time period according to the extracted acceleration.

Optionally, the step of collating all the obtained safety evaluation sub-results to obtain a total safety evaluation result of the vehicle to be tested includes:

aiming at the safety evaluation score result of the same frame data, taking the proportion of the times that the actual distance and the safe distance are both 0 to the simulation times as the collision rate of the vehicle to be tested in the frame;

taking the maximum difference value in the longitudinal difference values smaller than a preset longitudinal threshold value as the maximum deviation value of the vehicle to be tested in the longitudinal unsafe state of the frame, wherein the longitudinal difference value is the difference value between the longitudinal actual distance and the longitudinal safe distance;

taking the maximum difference value of the left lateral difference values smaller than a preset left lateral threshold value as the maximum deviation value of the vehicle to be tested in the left lateral unsafe state of the frame, wherein the left lateral difference value is the difference value between the left lateral actual distance and the left lateral safe distance;

taking the maximum difference value of right lateral difference values smaller than a preset right lateral threshold value as the maximum deviation value of the vehicle to be tested in the unsafe state of the right lateral of the frame, wherein the right lateral difference value is the difference value between the actual right lateral distance and the safe right lateral distance;

taking the minimum difference value in the longitudinal difference values which are not less than the preset longitudinal threshold value as the minimum deviation value of the vehicle to be tested in the longitudinal safety state of the frame;

taking the minimum difference value in the left transverse difference values which are not smaller than the preset left transverse threshold value as the minimum deviation value of the vehicle to be tested in the left transverse safety state of the frame;

and taking the minimum difference value in the right transverse difference values which are not less than the preset right transverse threshold value as the minimum deviation value of the vehicle to be tested in the right transverse safety state of the frame.

Optionally, the step of collating all the obtained regulatory evaluation score results to obtain a regulatory evaluation total result of the vehicle to be tested includes:

aiming at the regularity evaluation score result of the same frame data, taking the ratio of the number of the vehicle to be tested driving over the line to the simulation number as the line crossing rate of the vehicle to be tested in the frame;

taking the ratio of the red light running times of the vehicle to be tested to the simulation times as the red light running rate of the vehicle to be tested in the frame;

and taking the proportion of the times of the vehicle to be tested not driving according to the speed limit standard to the simulation times as the rate of the vehicle to be tested not driving according to the speed limit standard in the frame.

Optionally, the step of collating all the obtained comfort evaluation sub-results to obtain a total comfort evaluation result of the vehicle to be tested includes:

and counting the average acceleration, the average deceleration, the deceleration time, the acceleration time, the maximum deceleration, the maximum acceleration, the deceleration times and the acceleration times obtained by each simulation aiming at the comfort evaluation score result in the same target time period to obtain the distribution rate of the average acceleration, the distribution rate of the average deceleration, the distribution rate of the deceleration time, the distribution rate of the acceleration time, the distribution rate of the maximum deceleration, the distribution rate of the maximum acceleration, the distribution rate of the deceleration times and the distribution rate of the acceleration times of the vehicle to be tested in the target time period.

In a second aspect, an embodiment of the present invention provides a scene-independent unmanned simulation test evaluation apparatus, including:

the system comprises a receiving module, a judging module and a judging module, wherein the receiving module is used for receiving to-be-tested data used for carrying out unmanned driving simulation test evaluation on a to-be-tested vehicle;

the first evaluation module is used for assembling the frame data according to a preset data assembly rule aiming at each frame of data to be tested to generate a plurality of assembly data, inputting each assembly data to a preset RSS (really simple syndication) model to obtain a safety evaluation score result of the vehicle to be tested in the frame, extracting information of a traffic sign object and position information and driving information of the vehicle to be tested when the traffic sign object appears in the frame data, judging whether the vehicle to be tested violates rules or not according to the extracted information, and obtaining a regularity evaluation score result of the vehicle to be tested in the frame;

the second evaluation module is used for extracting the acceleration of the vehicle to be tested in the target time period according to each target time period in the data to be tested, and calculating a comfort evaluation score result of the vehicle to be tested in the target time period according to the extracted acceleration, wherein each target time period is a time period when the vehicle to be tested has two continuous acceleration behaviors and one deceleration behavior after the acceleration behaviors, the starting time point of each target time period is the time point when the first acceleration behavior in the two continuous acceleration behaviors occurs, and the ending time of each target time period is the time point when the deceleration behavior occurs;

and the total evaluation result determining module is used for circularly testing the data to be tested until the preset simulation times are reached, sorting all the obtained safety evaluation score results to obtain the total safety evaluation result of the vehicle to be tested, sorting all the obtained regulatory evaluation score results to obtain the total regulatory evaluation result of the vehicle to be tested, and sorting all the obtained comfort evaluation score results to obtain the total comfort evaluation result of the vehicle to be tested.

Optionally, the first evaluation module is specifically configured to:

Optionally, the first evaluation module includes:

the distance calculation unit is used for calculating the actual distance and the safe distance between the target object and the vehicle to be tested according to the dynamic parameters of the vehicle to be tested and the position information of the target object in the assembly data aiming at each assembly data, wherein the types of the distances comprise a longitudinal distance, a left transverse distance and a right transverse distance;

and the safety evaluation score result determining unit is used for calculating a difference value between the actual distance and the safe distance, and taking the actual distance, the safe distance and the difference value as a safety evaluation score result of the vehicle to be tested in the frame.

Optionally, the first evaluation module is specifically configured to:

Optionally, the second evaluation module is specifically configured to:

Optionally, the total evaluation result determining module is specifically configured to:

As can be seen from the above, in the scene-independent unmanned simulation test and evaluation method provided in the embodiments of the present invention, each frame of data in the to-be-tested data is assembled according to the preset data assembly rule to generate a plurality of assembly data, each assembly data is input to the preset RSS model to obtain the safety evaluation score result of the to-be-tested vehicle in the frame, when a traffic sign object appears in the frame of data, information of the traffic sign object and position information and driving information of the to-be-tested vehicle are extracted, whether the to-be-tested vehicle violates a rule is determined according to the extracted information to obtain the regularity evaluation score result of the to-be-tested vehicle in the frame, the acceleration of the to-be-tested vehicle in the target time period is extracted for each target time period in the to-be-tested data, the comfort evaluation score result of the to-be-tested vehicle in the target time period is calculated according to the extracted, and circularly testing the data to be tested until the preset simulation times are reached, sorting all the obtained safety evaluation score results to obtain a total safety evaluation result of the vehicle to be tested, sorting all the obtained regulation evaluation score results to obtain a total regulation evaluation result of the vehicle to be tested, and sorting all the obtained comfort evaluation score results to obtain a total comfort evaluation result of the vehicle to be tested. Therefore, in the embodiment of the invention, the safety evaluation, the regulation evaluation and the comfort evaluation are carried out in a mode of directly extracting data from the data to be tested, and the extracted data is independent of the type of the scene, so that the method is not limited to carrying out simulation test evaluation only on the scene files of a specific type, can carry out simulation test evaluation on any scene file, and has high compatibility and good robustness. Of course, not all of the advantages described above need to be achieved at the same time in the practice of any one product or method of the invention.

The innovation points of the embodiment of the invention comprise:

1. in the embodiment of the invention, the safety evaluation, the regulation evaluation and the comfort evaluation are carried out in a mode of directly extracting data from the data to be tested, and the extracted data is independent of the type of the scene, so that the method is not limited to carrying out simulation test evaluation only on the scene files of a specific type, can carry out simulation test evaluation on any scene file, and has high compatibility and good robustness.

2. And aiming at each frame of data in the data to be tested, assembling the frame of data according to a preset data assembling rule to generate a plurality of assembling data, and inputting each assembling data into a preset RSS model to obtain a safety evaluation score result of the vehicle to be tested in the frame.

3. And aiming at each frame of data in the data to be tested, obtaining a rule evaluation score result of the vehicle to be tested in the frame by extracting the information of the traffic sign object, the position information and the driving information of the vehicle to be tested and judging whether the vehicle to be tested violates the rule according to the extracted information.

4. And aiming at each target time period in the data to be tested, calculating a comfort evaluation score result of the vehicle to be tested in the target time period according to the extracted acceleration by extracting the acceleration of the vehicle to be tested in the target time period.

Drawings

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below. It is to be understood that the drawings in the following description are merely exemplary of some embodiments of the invention. For a person skilled in the art, without inventive effort, further figures can be obtained from these figures.

Fig. 1 is a schematic flow chart of a scene-independent unmanned simulation test evaluation method according to an embodiment of the present invention;

fig. 2 is a schematic structural diagram of a scene-independent unmanned simulation test evaluation device according to an embodiment of the present invention.

Detailed Description

The technical solution in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. It is to be understood that the described embodiments are merely a few embodiments of the invention, and not all embodiments. All other embodiments, which can be obtained by a person skilled in the art without inventive effort based on the embodiments of the present invention, are within the scope of the present invention.

It is to be noted that the terms "comprises" and "comprising" and any variations thereof in the embodiments and drawings of the present invention are intended to cover non-exclusive inclusions. For example, a process, method, system, article, or apparatus that comprises a list of steps or elements is not limited to only those steps or elements listed, but may alternatively include other steps or elements not listed, or inherent to such process, method, article, or apparatus.

The embodiment of the invention discloses a scene-independent unmanned simulation test evaluation method and device, which can be used for carrying out simulation test evaluation on any scene file and have high compatibility and robustness. The following provides a detailed description of embodiments of the invention.

Fig. 1 is a schematic flow chart of a scene-independent unmanned simulation test evaluation method according to an embodiment of the present invention. The method specifically comprises the following steps.

S110: and receiving to-be-tested data for unmanned driving simulation test evaluation of the to-be-tested vehicle.

The scene-independent unmanned simulation test evaluation method provided by the embodiment of the invention can be applied to any electronic equipment with computing capability, and the electronic equipment can be a terminal or a server. In one implementation, the functional software for implementing the scene-independent unmanned simulation test evaluation method may exist in the form of separate client software, or may exist in the form of a plug-in of currently-related client software, which is all possible.

The vehicle to be tested is: and the virtual vehicle is provided with the tested automatic driving test algorithm.

The electronic equipment receives data to be tested for unmanned simulation test evaluation of vehicles to be tested. The data to be tested can be any data containing a vehicle to be tested, the data to be tested can be specific type of scene data of a pre-component for the vehicle to be tested, and the data to be tested can also be random data generated based on the specific type of scene data in a random generalization mode.

S120: the method comprises the steps of assembling frame data according to a preset data assembling rule aiming at each frame of data to be tested to generate a plurality of assembling data, inputting each assembling data to a preset RSS model to obtain a safety evaluation score result of a vehicle to be tested in the frame, extracting information of a traffic sign object and position information and driving information of the vehicle to be tested when the traffic sign object appears in the frame data, judging whether the vehicle to be tested violates rules or not according to the extracted information, and obtaining a regularity evaluation score result of the vehicle to be tested in the frame.

After receiving the data to be tested, the electronic equipment performs security evaluation, regulatory evaluation and comfort evaluation on the data to be tested in a simulation test mode. The safety evaluation refers to detecting the running safety of the vehicle to be tested, the regulatory evaluation refers to detecting whether the vehicle to be tested violates the road passing regulation during running, and the comfort evaluation refers to detecting the driving comfort of the vehicle to be tested.

The following describes the security assessment:

after receiving the data to be tested, the electronic equipment assembles each frame of data in the data to be tested according to a preset data assembly rule to generate a plurality of assembly data.

For each frame of data in the data to be tested, assembling the frame of data according to a preset data assembling rule to generate a plurality of assembling data, which may include:

and aiming at each frame of data in the data to be tested, extracting the vehicle data to be tested and the target object data in the frame of data, assembling the vehicle data to be tested and each target object data to generate a plurality of assembling data, wherein the type of the target object data at least comprises the type of the vehicle.

The target object is an object which may cause a potential safety hazard with the vehicle to be tested in the data to be tested, except for the vehicle to be tested, and since the vehicle is most likely to cause the potential safety hazard with the vehicle to be tested, the type of the target object data at least includes a vehicle type, and of course, may also include a pedestrian type or an obstacle type.

In the embodiment of the invention, aiming at each frame of data in the data to be tested, the data of the vehicle to be tested and each target object are assembled to generate a plurality of assembling data, namely, how many target objects in the frame of data generate how many assembling data.

For example, the frame data includes a vehicle a to be tested, a vehicle B, and a pedestrian C, the vehicle a data and the vehicle B data to be tested are assembled to generate an assembly data, and the vehicle a data and the pedestrian C data to be tested are assembled to generate an assembly data.

After a plurality of assembly data are generated, inputting each assembly data into a preset RSS (Responsibility-Sensitive Safety) model to obtain a Safety evaluation score result of the vehicle to be tested in the frame.

Specifically, inputting each assembly data into a preset RSS model to obtain a safety evaluation score result of the vehicle to be tested in the frame, which may include:

aiming at each assembly data, calculating the actual distance and the safe distance between a target object and a vehicle to be tested according to the dynamic parameters of the vehicle to be tested and the position information of the target object in the assembly data, wherein the types of the distances comprise a longitudinal distance, a left transverse distance and a right transverse distance;

Wherein the dynamic parameters of the vehicle to be tested are parameters related to the performance of the vehicle, such as: the maximum acceleration of the vehicle or the large deceleration of the vehicle, and the deceleration is determined when the acceleration is negative. Aiming at each assembly data, the actual distance and the safe distance between the target object and the vehicle to be tested can be calculated according to the dynamic parameters of the vehicle to be tested and the position information of the target object in the assembly data, the actual distance is the distance between the vehicle to be tested and the target object in the simulation process, the safe distance is the minimum distance when no collision occurs between the vehicle to be tested and the target object, and the target object possibly appears in the longitudinal direction and the transverse direction of the vehicle to be tested, so the types of the distances comprise the longitudinal distance, the left transverse distance and the right transverse distance.

After the actual distance and the safe distance are obtained, calculating a difference value between the actual distance and the safe distance, if the difference value is smaller, it is indicated that the distance between the vehicle to be tested and the target object is smaller, collision is easy to occur, if the difference value is larger, it is indicated that the distance between the vehicle to be tested and the target object is smaller, collision is not easy to occur, because safety evaluation needs to be performed on each frame of data in the data to be tested, and the same frame of data needs to be tested circularly, after the difference value corresponding to the frame of data is obtained, the actual distance, the safe distance and the difference value are used as safety evaluation scores of the vehicle to be tested in the frame, and subsequently, each safety evaluation score needs to be integrated.

Therefore, for each frame of data in the data to be tested, assembling the frame of data according to a preset data assembling rule to generate a plurality of assembling data, and inputting each assembling data into a preset RSS model to obtain a safety evaluation score result of the vehicle to be tested in the frame.

The following is a description of the assessment of regularity:

and aiming at each frame of data in the data to be tested, when a traffic sign object appears in the frame of data, extracting the information of the traffic sign object and the position information and the driving information of the vehicle to be tested, and judging whether the vehicle to be tested violates rules or not according to the extracted information to obtain a rule evaluation score result of the vehicle to be tested in the frame.

Since the vehicle to be tested may have the illegal behavior only when the traffic sign object appears in the frame data, the information of the traffic sign object and the position information and the driving information of the vehicle to be tested are extracted when the traffic sign object appears in the frame data. Wherein the traffic sign object is an object related to traffic regulations. The driving information of the vehicle to be tested includes, but is not limited to, speed.

For example, the determining whether the vehicle to be tested violates the rule according to the extracted information to obtain the rule evaluation score result of the vehicle to be tested in the frame may include:

when the traffic sign objects are traffic signal lamps and stop lines, judging whether the vehicle to be tested runs through the red light or not according to the position information of the vehicle to be tested and the position information of the stop line, and if so, determining that the vehicle to be tested runs through the red light;

when the traffic sign is a humanoid crosswalk line or a deceleration traffic-giving sign, judging whether the vehicle to be tested stops at the position of the humanoid crosswalk line or not and starts to run in a preset time period, and if not, determining that the vehicle to be tested runs off the line.

Since the speed limit signs include the highest speed limit sign and the lowest speed limit sign, the vehicle to be tested does not travel according to the speed limit standard including overspeed travel and low speed travel, wherein the low speed travel is travel at a speed lower than the lowest speed indicated by the speed limit signs.

And when the speed limit sign is the highest speed limit sign, judging whether the speed of the vehicle to be tested exceeds the highest speed shown by the speed limit sign, if so, determining that the vehicle to be tested runs at an overspeed, when the speed limit sign is the lowest speed limit sign, judging whether the speed of the vehicle to be tested is lower than the lowest speed shown by the speed limit sign, and if so, determining that the vehicle to be tested runs at a low speed.

When the traffic sign objects are traffic lights and stop lines, judging whether the vehicles to be tested exceed the stop lines when the traffic lights are red or not according to the position information of the vehicles to be tested and the position information of the stop lines, and if so, determining that the vehicles to be tested run the red lights.

When the traffic sign object is a traffic warning sign, such as an entrance prohibition sign, judging whether the position of the vehicle to be tested exceeds the position of the traffic warning sign, and if so, determining that the vehicle to be tested runs off the line.

When the traffic sign is a crosswalk line or a deceleration yielding sign, it is indicated that the vehicle to be tested needs to stop and yield, and continues to run after yielding but cannot stop, so that whether the vehicle to be tested stops at the position of the crosswalk line and starts to run within a preset time period needs to be judged, and if not, the vehicle to be tested is determined to run off the line, wherein the duration of the preset time period is shorter, and is generally 3 s.

Therefore, for each frame of data in the data to be tested, the rule evaluation score result of the vehicle to be tested in the frame is obtained by extracting the information of the traffic sign object, the position information and the driving information of the vehicle to be tested and judging whether the vehicle to be tested violates the rule or not according to the extracted information.

Whether the obtained vehicle to be tested does not run according to the speed limit standard, whether the vehicle runs through the red light and whether the vehicle runs off the line is the result of the regularity evaluation score of the vehicle to be tested in the frame. The same as the safety evaluation, the regularity evaluation is required for each frame of data in the data to be tested, the same frame of data is required to be tested circularly, and the subsequent regularity evaluation sub-results are required to be integrated.

S130: the method comprises the steps of extracting the acceleration of a vehicle to be tested in a target time period according to the target time period in data to be tested, and calculating a comfort evaluation score result of the vehicle to be tested in the target time period according to the extracted acceleration, wherein each target time period is a time period when the vehicle to be tested has two continuous acceleration behaviors and one deceleration behavior after the acceleration behaviors, the starting time point of each target time period is the time point when the first acceleration behavior in the two continuous acceleration behaviors occurs, and the ending time of each target time period is the time point when the deceleration behavior occurs.

Comfort evaluation is described below:

the driving comfort is required to be evaluated for continuous driving, so that when the comfort evaluation is performed, each continuous driving behavior of the vehicle to be tested in the data to be tested needs to be evaluated, wherein a time period of the one-time continuous driving behavior of the vehicle to be tested is a target time period, the target time period is a time period in which two continuous acceleration behaviors of the vehicle to be tested exist and one deceleration behavior exists after the acceleration behaviors, a starting time point of the target time period is a time point at which a first acceleration behavior in the two continuous acceleration behaviors occurs, and an ending time of the target time period is a time point at which the deceleration behavior occurs.

For example, the vehicle to be tested is accelerated at the 1 st frame and the 5 th frame, decelerated at the 10 th frame, accelerated at the 12 th frame and the 16 th frame, and decelerated at the 20 th frame, then the 1 st frame to the 10 th frame are a target time period, and the 12 th frame to the 20 th frame are a target time period.

And aiming at each target time period in the data to be tested, extracting the acceleration of the vehicle to be tested in the target time period, and calculating the comfort evaluation score result of the vehicle to be tested in the target time period according to the extracted acceleration.

Calculating a comfort evaluation score result of the vehicle to be tested in the target time period according to the extracted acceleration, wherein the comfort evaluation score result may include:

The vehicle to be tested may be accelerated and decelerated for a target period of time several times, and thus, the average acceleration, the average deceleration, the deceleration time, the acceleration time, the maximum deceleration, the maximum acceleration, the number of decelerations, and the number of accelerations of the vehicle to be tested for the target period of time may be determined according to the extracted acceleration and speed time basic calculation formula.

The greater the difference between the maximum acceleration and the average acceleration, the worse the comfort, the greater the difference between the maximum deceleration and the average deceleration, the worse the comfort, the greater the number of decelerations and the number of accelerations, the worse the comfort, within a target time period.

Therefore, aiming at each target time period in the data to be tested, the comfort evaluation score result of the vehicle to be tested in the target time period can be calculated according to the extracted acceleration by extracting the acceleration of the vehicle to be tested in the target time period.

Each target time period corresponds to a comfort evaluation score, and each comfort evaluation score needs to be integrated subsequently.

S140: and circularly testing the data to be tested until the preset simulation times are reached, sorting all the obtained safety evaluation score results to obtain a total safety evaluation result of the vehicle to be tested, sorting all the obtained regulation evaluation score results to obtain a total regulation evaluation result of the vehicle to be tested, and sorting all the obtained comfort evaluation score results to obtain a total comfort evaluation result of the vehicle to be tested.

In the embodiment of the invention, the data to be tested needs to be tested circularly until reaching the preset simulation times, and then each evaluation sub-result is sorted to obtain the total evaluation result. The obtained safety evaluation score results are sorted to obtain a total safety evaluation result of the vehicle to be tested, all the obtained regulation evaluation score results are sorted to obtain a total regulation evaluation result of the vehicle to be tested, and all the obtained comfort evaluation score results are sorted to obtain a total comfort evaluation result of the vehicle to be tested.

Under the condition that the actual distance, the safety distance and the difference value are used as the safety evaluation score results of the vehicle to be tested in the frame, sorting all the obtained safety evaluation score results to obtain a total safety evaluation result of the vehicle to be tested, which may include:

taking the maximum difference value of the longitudinal difference values smaller than the preset longitudinal threshold value as the maximum deviation value of the vehicle to be tested in the longitudinal unsafe state of the frame, wherein the longitudinal difference value is the difference value between the longitudinal actual distance and the longitudinal safe distance;

taking the maximum difference value of the left lateral difference values smaller than the preset left lateral threshold value as the maximum deviation value of the vehicle to be tested in the left lateral unsafe state of the frame, wherein the left lateral difference value is the difference value between the left lateral actual distance and the left lateral safe distance;

taking the maximum difference value of the right transverse difference values smaller than the preset right transverse threshold value as the maximum deviation value of the vehicle to be tested in the unsafe state of the right transverse direction of the frame, wherein the right transverse difference value is the difference value between the actual distance of the right transverse direction and the safe distance of the right transverse direction;

taking the minimum difference value in the longitudinal difference values not less than the preset longitudinal threshold value as the minimum deviation value of the vehicle to be tested in the longitudinal safety state of the frame;

taking the minimum difference value in the left transverse difference values which are not less than the preset left transverse threshold value as the minimum deviation value of the vehicle to be tested in the left transverse safety state of the frame;

And aiming at the safety evaluation score result of the same frame data, if the actual distance and the safe distance are both 0, indicating that the vehicle to be tested collides with the target object, and taking the proportion of the times of which the actual distance and the safe distance are both 0 to the simulation times as the collision rate of the vehicle to be tested in the frame. For example, the simulation times is 10 times, the times that the actual distance and the safe distance are both 0 for the 1 st frame data is 3 times, and the times that the actual distance and the safe distance are both 0 for the 2 nd frame data is 2 times, so that the collision rate of the vehicle to be tested in the 1 st frame is three tenths, and the collision rate of the vehicle to be tested in the 2 nd frame is one fifth.

The unsafe state is a state where the actual distance and the safe distance have a small difference but no collision occurs, and the safe state is a state where the actual distance and the safe distance have a large difference and no collision occurs.

Since the types of distances include a longitudinal distance, a left lateral distance, and a right lateral distance, the unsafe states include a longitudinal unsafe state, a left lateral unsafe state, and a right lateral unsafe state, and the safe states include a longitudinal safe state, a left lateral safe state, and a right lateral safe state.

Specifically, the maximum difference value of longitudinal difference values smaller than a preset longitudinal threshold value is taken as the maximum deviation value of the vehicle to be tested in the longitudinal unsafe state of the frame, wherein the longitudinal difference value is the difference value between the longitudinal actual distance and the longitudinal safe distance, the maximum difference value of left lateral difference values smaller than a preset left lateral threshold value is taken as the maximum deviation value of the vehicle to be tested in the left lateral unsafe state of the frame, wherein the left lateral difference value is the difference value between the left lateral actual distance and the left lateral safe distance, the maximum difference value of right lateral difference values smaller than a preset right lateral threshold value is taken as the maximum deviation value of the vehicle to be tested in the right lateral unsafe state of the frame, wherein the right lateral difference value is the difference value between the right lateral actual distance and the right lateral safe distance, and the minimum difference value of the longitudinal difference values not smaller than the preset longitudinal threshold value is taken as the minimum deviation value of the vehicle to be tested in the longitudinal safe state of the, and taking the minimum difference value in the left transverse difference values not less than the preset left transverse threshold value as the minimum deviation value of the vehicle to be tested in the left transverse safety state of the frame, and taking the minimum difference value in the right transverse difference values not less than the preset right transverse threshold value as the minimum deviation value of the vehicle to be tested in the right transverse safety state of the frame.

And determining whether the vehicle to be tested has danger or not according to the maximum deviation value and the minimum deviation value under each state.

In order to clearly view the obtained total result of the security evaluation, a histogram may be used for displaying, which is not limited in this embodiment of the present invention.

The sorting of all the obtained regulatory evaluation score results to obtain a regulatory evaluation total result of the vehicle to be tested may include:

aiming at the regularity evaluation score result of the same frame data, taking the ratio of the number of the vehicles to be tested to the simulation number of the off-line driving as the off-line rate of the vehicles to be tested in the frame;

For example, the simulation times is 10 times, for the 1 st frame data, the number of times that the vehicle to be tested travels across the line is 3 times, the number of times that the vehicle to be tested runs the red light is 1 time, the number of times that the vehicle to be tested does not travel according to the speed limit standard is 0, for the 2 nd frame data, the number of times that the vehicle to be tested travels across the line is 2 times, the number of times that the vehicle to be tested runs the red light is 2 times, and the number of times that the vehicle to be tested does not travel according to the speed limit standard is 1 time, the line crossing rate of the vehicle to be tested in the 1 st frame is three tenths, the red light running rate is one tenth, the speed limit standard running rate is not 0, the line crossing rate of the vehicle to be tested in the 2 nd frame is one fifth, the red light running rate is one fifth.

The running according to the speed limit standard comprises overspeed running and low-speed running, so that the ratio of the number of times of overspeed running of the vehicle to be tested to the number of times of simulation can be used as the overspeed running rate of the vehicle to be tested in the frame, and the ratio of the number of times of low-speed running of the vehicle to be tested to the number of times of simulation can be used as the low-speed running rate of the vehicle to be tested in the frame.

In order to clearly view the obtained overall result of the regulatory assessment, a histogram may be used for displaying, which is not limited in this embodiment of the present invention.

The sorting of all the obtained comfort evaluation sub-results to obtain a total comfort evaluation result of the vehicle to be tested may include:

In the embodiment of the invention, the average acceleration, the average deceleration, the deceleration time, the acceleration time, the maximum deceleration, the maximum acceleration, the deceleration times and the acceleration times obtained by each simulation are counted in a statistical manner, so that the distribution rate of the average acceleration, the distribution rate of the average deceleration, the distribution rate of the deceleration time, the distribution rate of the acceleration time, the distribution rate of the maximum deceleration, the distribution rate of the maximum acceleration, the distribution rate of the deceleration times and the distribution rate of the acceleration times of the vehicle to be tested in the target time period are obtained.

In summary, according to the scene-independent unmanned simulation test evaluation method provided by the embodiment of the present invention, for each frame of data in the to-be-tested data, the frame of data is assembled according to the preset data assembly rule to generate a plurality of assembly data, each assembly data is input to the preset RSS model to obtain the safety evaluation score result of the to-be-tested vehicle in the frame, when a traffic sign object appears in the frame of data, information of the traffic sign object and position information and driving information of the to-be-tested vehicle are extracted, whether the to-be-tested vehicle violates a rule is judged according to the extracted information, the regularity evaluation score result of the to-be-tested vehicle in the frame is obtained, for each target time slot in the to-be-tested data, the acceleration of the to-be-tested vehicle in the target time slot is extracted, the comfort evaluation score result of the to-be-tested vehicle in the target time slot, and circularly testing the data to be tested until the preset simulation times are reached, sorting all the obtained safety evaluation score results to obtain a total safety evaluation result of the vehicle to be tested, sorting all the obtained regulation evaluation score results to obtain a total regulation evaluation result of the vehicle to be tested, and sorting all the obtained comfort evaluation score results to obtain a total comfort evaluation result of the vehicle to be tested. Therefore, in the embodiment of the invention, the safety evaluation, the regulation evaluation and the comfort evaluation are carried out in a mode of directly extracting data from the data to be tested, and the extracted data is independent of the type of the scene, so that the method is not limited to carrying out simulation test evaluation only on the scene files of a specific type, can carry out simulation test evaluation on any scene file, and has high compatibility and good robustness.

Corresponding to the above method embodiment, an embodiment of the present invention provides a scene-independent unmanned simulation test evaluation apparatus, and as shown in fig. 2, the apparatus may include:

the system comprises a receiving module 201, a data processing module and a data processing module, wherein the receiving module is used for receiving to-be-tested data used for carrying out unmanned simulation test evaluation on a to-be-tested vehicle;

the first evaluation module 202 is configured to assemble each frame of data in the to-be-tested data according to a preset data assembly rule to generate a plurality of assembly data, input each assembly data to a preset RSS model to obtain a safety evaluation score result of the to-be-tested vehicle in the frame, extract information of a traffic sign object and position information and driving information of the to-be-tested vehicle when the traffic sign object appears in the frame of data, judge whether the to-be-tested vehicle violates a rule according to the extracted information, and obtain a regularity evaluation score result of the to-be-tested vehicle in the frame;

the second evaluation module 203 is configured to extract, for each target time period in the to-be-tested data, an acceleration of the to-be-tested vehicle in the target time period, and calculate a comfort evaluation score result of the to-be-tested vehicle in the target time period according to the extracted acceleration, where each target time period is a time period in which the to-be-tested vehicle has two consecutive acceleration behaviors and a deceleration behavior after the acceleration behavior, a starting time point of each target time period is a time point at which a first acceleration behavior in the two consecutive acceleration behaviors occurs, and an ending time of each target time period is a time point at which the deceleration behavior occurs;

the total evaluation result determining module 204 is configured to perform a cycle test on the data to be tested until a preset simulation frequency is reached, sort all the obtained safety evaluation score results to obtain a total safety evaluation result of the vehicle to be tested, sort all the obtained regulatory evaluation score results to obtain a total regulatory evaluation result of the vehicle to be tested, and sort all the obtained comfort evaluation score results to obtain a total comfort evaluation result of the vehicle to be tested.

The scene-independent unmanned simulation test evaluation device provided by the embodiment of the invention assembles each frame of data in the to-be-tested data according to a preset data assembly rule to generate a plurality of assembly data, inputs each assembly data into a preset RSS model to obtain a safety evaluation score result of the to-be-tested vehicle in the frame, extracts information of a traffic sign object, position information and running information of the to-be-tested vehicle when the traffic sign object appears in the frame of data, judges whether the to-be-tested vehicle violates rules according to the extracted information to obtain a regularity evaluation score result of the to-be-tested vehicle in the frame, extracts acceleration of the to-be-tested vehicle in a target time period according to each target time period in the to-be-tested data, calculates a comfort evaluation score result of the to-be-tested vehicle in the target time period according to the extracted acceleration, and circularly testing the data to be tested until the preset simulation times are reached, sorting all the obtained safety evaluation score results to obtain a total safety evaluation result of the vehicle to be tested, sorting all the obtained regulation evaluation score results to obtain a total regulation evaluation result of the vehicle to be tested, and sorting all the obtained comfort evaluation score results to obtain a total comfort evaluation result of the vehicle to be tested. Therefore, in the embodiment of the invention, the safety evaluation, the regulation evaluation and the comfort evaluation are carried out in a mode of directly extracting data from the data to be tested, and the extracted data is independent of the type of the scene, so that the method is not limited to carrying out simulation test evaluation only on the scene files of a specific type, can carry out simulation test evaluation on any scene file, and has high compatibility and good robustness.

In one implementation, the first evaluation module 202 may be specifically configured to:

In one implementation, the first evaluation module 202 may include:

Optionally, the first evaluation module 202 may be specifically configured to:

Optionally, the second evaluation module 203 may be specifically configured to:

Optionally, the total evaluation result determining module 204 may be specifically configured to:

The above device embodiment corresponds to the method embodiment, and has the same technical effect as the method embodiment, and for the specific description, refer to the method embodiment. The device embodiment is obtained based on the method embodiment, and for specific description, reference may be made to the method embodiment section, which is not described herein again.

Those of ordinary skill in the art will understand that: the figures are merely schematic representations of one embodiment, and the blocks or flow diagrams in the figures are not necessarily required to practice the present invention.

Those of ordinary skill in the art will understand that: modules in the devices in the embodiments may be distributed in the devices in the embodiments according to the description of the embodiments, or may be located in one or more devices different from the embodiments with corresponding changes. The modules of the above embodiments may be combined into one module, or further split into multiple sub-modules.

Finally, it should be noted that: the above examples are only intended to illustrate the technical solution of the present invention, but not to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions of the embodiments of the present invention.

Claims

1. A scene-independent unmanned simulation test evaluation method is characterized by comprising the following steps:

2. The method of claim 1, wherein the step of assembling each frame of data to be tested according to a preset data assembling rule for the frame of data to generate a plurality of assembling data comprises:

3. The method according to claim 1 or 2, wherein the step of inputting each assembly data into a preset RSS model to obtain the safety evaluation score of the vehicle to be tested in the frame comprises:

4. The method of claim 1, wherein the step of determining whether the vehicle to be tested violates the rule according to the extracted information to obtain the rule evaluation score of the vehicle to be tested in the frame comprises:

5. The method of claim 1, wherein the step of calculating a comfort assessment score for the vehicle under test over the target time period based on the extracted acceleration comprises:

6. The method of claim 3, wherein the step of collating all the safety assessment partial results to obtain a total safety assessment result of the vehicle to be tested comprises:

7. The method of claim 4, wherein the step of collating all of the obtained regulatory assessment score results to obtain a total regulatory assessment result for the vehicle under test comprises:

8. The method of claim 5, wherein the step of collating all of the obtained comfort assessment scores to obtain a total comfort assessment result for the vehicle under test comprises:

9. A scene-independent unmanned simulation test evaluation device is characterized by comprising:

10. The apparatus of claim 9, wherein the first evaluation module is specifically configured to: