CN111290370B - Automatic driving performance detection method and device - Google Patents

Automatic driving performance detection method and device Download PDF

Info

Publication number
CN111290370B
CN111290370B CN202010140452.5A CN202010140452A CN111290370B CN 111290370 B CN111290370 B CN 111290370B CN 202010140452 A CN202010140452 A CN 202010140452A CN 111290370 B CN111290370 B CN 111290370B
Authority
CN
China
Prior art keywords
automatic driving
data
target
driving
preset
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202010140452.5A
Other languages
Chinese (zh)
Other versions
CN111290370A (en
Inventor
胡太群
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Tencent Technology Shenzhen Co Ltd
Original Assignee
Tencent Technology Shenzhen Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Tencent Technology Shenzhen Co Ltd filed Critical Tencent Technology Shenzhen Co Ltd
Priority to CN202010140452.5A priority Critical patent/CN111290370B/en
Publication of CN111290370A publication Critical patent/CN111290370A/en
Application granted granted Critical
Publication of CN111290370B publication Critical patent/CN111290370B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05B—CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
    • G05B23/00—Testing or monitoring of control systems or parts thereof
    • G05B23/02—Electric testing or monitoring
    • G05B23/0205—Electric testing or monitoring by means of a monitoring system capable of detecting and responding to faults
    • G05B23/0218—Electric testing or monitoring by means of a monitoring system capable of detecting and responding to faults characterised by the fault detection method dealing with either existing or incipient faults
    • G05B23/0243—Electric testing or monitoring by means of a monitoring system capable of detecting and responding to faults characterised by the fault detection method dealing with either existing or incipient faults model based detection method, e.g. first-principles knowledge model
    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05B—CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
    • G05B2219/00—Program-control systems
    • G05B2219/20—Pc systems
    • G05B2219/24—Pc safety
    • G05B2219/24065—Real time diagnostics

Abstract

The embodiment of the invention discloses a method and a device for detecting automatic driving performance; the method comprises the steps of acquiring a running data set of an automatic driving program for simulation running after the automatic driving program to be detected is loaded to a preset automatic driving system, identifying target running data in the running data set based on the type of the automatic driving program, performing feature extraction on the target running data to obtain an initial characteristic value of the target running data, classifying the initial characteristic value, determining a weight corresponding to the initial characteristic value based on a classification result, screening out a target characteristic value from the initial characteristic value according to the weight, analyzing the target characteristic value to obtain a comprehensive driving performance parameter of the preset automatic driving system loaded with the automatic driving program to be detected, and determining that the automatic driving performance detection of the automatic driving program to be detected is qualified when the comprehensive driving performance parameter exceeds a preset parameter threshold value; the scheme can greatly improve the test efficiency and accuracy of the automatic driving vehicle.

Description

Automatic driving performance detection method and device
Technical Field
The invention relates to the technical field of communication, in particular to an automatic driving performance detection method and device.
Background
In recent years, with the rapid development of internet technology, autonomous vehicles have come to be transported. The core of the automatic driving vehicle is an automatic driving system for controlling the automatic driving of the vehicle, and the automatic driving system comprises a plurality of automatic driving programs such as software algorithms and the like, for example, automatic driving programs such as perception, positioning, navigation, control and the like. Before the development of the autonomous vehicle is completed, it is necessary to detect the autonomous driving performance of each autonomous driving program. The existing test method mainly comprises the steps of independently testing a single automatic driving program in an automatic driving system, or directly deploying all automatic driving programs in the automatic driving system to a real vehicle for testing.
In the research and practice process of the prior art, the inventor of the invention finds that the automatic driving programs in the automatic driving system have an incidence relation, one automatic driving program is tested independently, the automatic driving performance of the automatic driving program cannot be determined accurately according to the feedback of other automatic driving programs in the automatic driving system, the test accuracy is greatly influenced, the test is directly carried out in a real vehicle, the test time and the test cost are greatly increased, and therefore, the test efficiency and the accuracy of the automatic driving vehicle are reduced.
Disclosure of Invention
The embodiment of the invention provides an automatic driving performance detection method and device. The efficiency and accuracy of testing of the autonomous vehicle can be improved.
An automatic drivability detection method comprising:
acquiring a driving data set of each automatic driving program for simulation driving after the automatic driving program to be detected is loaded to a preset automatic driving system, and identifying target driving data in the driving data set based on the type of the automatic driving program;
performing feature extraction on the target driving data to obtain an initial feature value of the target driving data;
classifying the initial characteristic values, and determining weights corresponding to the initial characteristic values based on classification results;
screening out a target characteristic value from the initial characteristic values according to the weight;
analyzing the target characteristic value to obtain comprehensive driving performance parameters of a preset automatic driving system loaded with the automatic driving program to be detected;
and when the comprehensive driving performance parameter exceeds a preset parameter threshold value, determining that the automatic driving performance of the automatic driving program to be detected is qualified.
Correspondingly, an embodiment of the present invention provides an automatic drivability detection apparatus, including:
the system comprises an acquisition unit, a simulation unit and a processing unit, wherein the acquisition unit is used for acquiring a driving data set of each automatic driving program for simulation driving after the automatic driving program to be detected is loaded to a preset automatic driving system, and identifying target driving data in the driving data set based on the type of the automatic driving program;
the extraction unit is used for carrying out feature extraction on the target driving data to obtain an initial feature value of the target driving data;
the classification unit is used for classifying the initial characteristic values and determining the weight corresponding to the initial characteristic values based on the classification result;
the screening unit is used for screening a target characteristic value from the initial characteristic values according to the weight;
the analysis unit is used for analyzing the target characteristic value to obtain the comprehensive driving performance parameters of the preset automatic driving system loaded with the automatic driving program to be detected;
and the determining unit is used for determining that the automatic driving performance detection of the automatic driving program to be detected is qualified when the comprehensive driving performance parameter exceeds a preset parameter threshold value.
Optionally, in some embodiments, the classification unit may be specifically configured to identify a data type of the initial feature value; classifying the initial characteristic values according to the identification result; and determining the weight corresponding to the initial characteristic value based on the classification result.
Optionally, in some embodiments, the classifying unit may be specifically configured to classify an initial feature value corresponding to the data type into a precise initial feature value when the data type is calculation accuracy data, where the calculation accuracy data is used to indicate accuracy of calculation of the target driving data; when the data type is calculation performance data, classifying the initial characteristic value corresponding to the data type into a time delay type initial characteristic value, wherein the calculation performance data is used for indicating the calculation speed of the target driving data; and when the data type is feedback control data, classifying the initial characteristic value corresponding to the data type into a feedback type initial characteristic value, wherein the feedback control data is data corresponding to the feedback control action executed in the target driving data.
Optionally, in some embodiments, the classifying unit may be specifically configured to determine a weight corresponding to the initial feature value according to preset delay-type initial feature values, precision-type initial feature values, and weights corresponding to feedback-type initial feature values, where the delay-type weight corresponding to the delay-type initial feature value is the largest among the three weights, the precision-type weight corresponding to the precision-type initial feature value is the second among the three weights, and the feedback-type weight corresponding to the feedback-type initial feature value is the smallest among the three weights.
Optionally, in some embodiments, the parsing unit may be specifically configured to weight the target feature value; normalizing the weighted target characteristic value, and performing format conversion on the normalized target characteristic value to obtain first target characteristic data; and analyzing the first target characteristic data by adopting a discrete model sub-model of the trained analysis model to obtain the comprehensive driving performance parameters of the preset automatic driving system loaded with the automatic driving program to be detected.
Optionally, in some embodiments, the parsing unit may be specifically configured to perform multi-scale feature extraction on the first target feature to obtain local feature information of the first target feature data; fusing the local information to obtain fused characteristic information; and calculating the comprehensive driving performance parameters of the preset automatic driving system loaded with the automatic driving program to be detected according to the fused characteristic information.
Optionally, in some embodiments, the determining unit may be specifically configured to weight the target characteristic value, and convert a data format of the weighted target characteristic value to obtain second target characteristic data; acquiring a mapping relation between the second target characteristic data and the manual intervention parameters according to the continuous submodel of the trained analytical model; and determining the manual intervention parameters corresponding to the second target characteristic data according to the mapping relation.
Optionally, in some embodiments, the analysis unit may be specifically configured to collect historical target feature data of real driving of the preset automatic driving system on the automatic driving vehicle; marking corresponding actual manual intervention parameters on the historical target characteristic data, and taking the historical target characteristic data marked with the actual manual intervention parameters as a target data sample; analyzing the target data sample by adopting a preset analysis model to obtain a predicted manual intervention parameter of the preset automatic driving system; and converging the preset analytical model according to the predicted manual intervention parameters and the marked actual manual intervention parameters to obtain a trained analytical model.
Optionally, in some embodiments, the obtaining unit may be specifically configured to obtain the to-be-detected automatic driving program; loading the automatic driving program to be detected to the preset automatic driving system; carrying out simulation driving test on the loaded automatic driving system to obtain a driving data set of each automatic driving program; target travel data is identified in the set of travel data based on the type of the autonomous driving program.
In addition, the embodiment of the invention also provides electronic equipment which comprises a processor and a memory, wherein the memory stores an application program, and the processor is used for running the application program in the memory to realize the automatic driving performance detection method provided by the embodiment of the invention.
In addition, an embodiment of the present invention further provides a computer-readable storage medium, where a plurality of instructions are stored, where the instructions are suitable for being loaded by a processor to perform steps in any one of the automatic drivability detection methods provided by the embodiments of the present invention.
The method comprises the steps that after the automatic driving program to be detected is loaded to a preset automatic driving system, a driving data set of each automatic driving program for simulation driving is obtained, target driving data are identified in the driving data set based on the type of the automatic driving program, and feature extraction is carried out on the target driving data to obtain an initial feature value of the target driving data; classifying the initial characteristic values, and determining weights corresponding to the initial characteristic values based on classification results; screening out a target characteristic value from the initial characteristic values according to the weight, then analyzing the target characteristic value to obtain a comprehensive drivability parameter of a preset automatic driving system loaded with the automatic driving program to be detected, and determining that the automatic driving performance detection of the automatic driving program to be detected is qualified when the comprehensive drivability parameter exceeds a preset parameter threshold value; because the scheme is used for carrying out simulation driving test on the whole automatic driving system and fully considering the influence of different types of simulation data on the automatic driving performance, the processing mode of classifying and weighting the data in the simulation driving is adopted, and the test efficiency and the accuracy of the automatic driving vehicle can be greatly improved.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings needed to be used in the description of the embodiments will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.
Fig. 1 is a scene schematic diagram of an automatic driving performance detection method according to an embodiment of the present invention;
FIG. 2 is a schematic flow chart of an automatic driveability detection method according to an embodiment of the invention;
FIG. 3 is a schematic diagram of the discrete model submodel provided by the embodiment of the invention calculating initial drivability parameters;
FIG. 4 is another schematic flow chart diagram illustrating an exemplary method for automatic drivability detection according to an embodiment of the present invention;
fig. 5 is a schematic structural diagram of an automatic drivability detecting apparatus according to an embodiment of the present invention;
fig. 6 is a schematic structural diagram of an acquisition unit of an automatic drivability detection apparatus according to an embodiment of the present invention;
fig. 7 is a schematic structural diagram of a classification unit of an automatic drivability detection apparatus according to an embodiment of the present invention;
fig. 8 is a schematic structural diagram of an analysis unit of an automatic drivability detection apparatus according to an embodiment of the present invention;
fig. 9 is another schematic structural diagram of an automatic drivability detecting apparatus according to an embodiment of the present invention;
fig. 10 is a schematic structural diagram of an electronic device according to an embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The embodiment of the invention provides an automatic driving performance detection method, an automatic driving performance detection device and a computer readable storage medium. The automatic driving performance detection device may be integrated in an electronic device, and the electronic device may be a server or the like.
For example, referring to fig. 1, taking an example that an automatic driving device is integrated in an electronic device, the electronic device acquires a driving data set of each automatic driving program for performing simulation driving after an automatic driving program to be detected is loaded into a preset automatic driving system, identifies target driving data in the driving data set based on the type of the automatic driving program, performs feature extraction on the target driving data to obtain an initial feature value of the target driving data, classifies the initial feature value, determines a weight corresponding to the initial feature value based on a classification result, screens out a target feature value from the initial feature value according to the weight, then analyzes the target feature value to obtain a comprehensive driving performance parameter of the preset automatic driving system loaded with the automatic driving program to be detected, and when the comprehensive driving performance parameter exceeds a preset parameter threshold, and determining that the automatic driving performance of the automatic driving program to be detected is qualified, deploying the automatic driving program to be detected on the automatic driving vehicle with the real deployment value, and determining that the automatic driving performance of the automatic driving program to be detected is unqualified and cannot be integrated on the real automatic driving vehicle when the comprehensive driving performance parameter does not exceed the preset first parameter threshold.
The preset automatic driving system can be an integration of a software program and a hardware part for automatic driving on the automatic driving vehicle, and the automatic driving system controls the driving of the vehicle according to a planned path and by combining actual information detected on a road.
The automatic driving program may be each algorithm module in the automatic driving system, the algorithm module is a computer program for collecting operation data, calculating operation data and controlling the automatic driving vehicle during automatic driving of the automatic driving vehicle, for example, a sensing program, a positioning program, a map or navigation program, a decision program, a planning program, a control program and the like, and the operation of the automatic driving vehicle is controlled by receiving real-time data of the automatic driving vehicle sent by the hardware device.
The following are detailed below. It should be noted that the following description of the embodiments is not intended to limit the preferred order of the embodiments.
The present embodiment will be described in terms of an automatic drivability detection apparatus, which may be specifically integrated in an electronic device, which may be a server or the like.
An automatic drivability detection method comprising:
the method comprises the steps of obtaining a driving data set of each automatic driving program which loads an automatic driving program to be detected to a preset automatic driving system and then carries out simulation driving, identifying target driving data in the driving data set based on the type of the automatic driving program, carrying out feature extraction on the target driving data to obtain an initial characteristic value of the target driving data, classifying the initial characteristic value, determining a weight corresponding to the initial characteristic value based on a classification result, screening out a target characteristic value from the initial characteristic value according to the weight, analyzing the target characteristic value to obtain a comprehensive driving performance parameter of the preset automatic driving system loaded with the automatic driving program to be detected, and determining that the automatic driving performance detection of the automatic driving program to be detected is qualified when the comprehensive driving performance parameter exceeds a preset parameter threshold value.
As shown in fig. 2, the specific flow of the automatic driveability detection method is as follows:
101. the method comprises the steps of obtaining a driving data set of each automatic driving program for simulation driving after the automatic driving program to be detected is loaded to a preset automatic driving system, and identifying target driving data in the driving data set based on the type of the automatic driving program.
For example, the method includes the steps of obtaining an automatic driving program to be detected, loading the automatic driving program to be detected to a preset automatic driving system, carrying out simulation test on the loaded automatic driving system to obtain a driving data set of each automatic driving program, and identifying target driving data in the driving data set based on the type of the automatic driving program. Specifically, the following may be mentioned:
and S1, acquiring the automatic driving program to be detected.
For example, the autonomous driving programs to be detected submitted or uploaded by the user may be received directly. The automatic driving program to be detected can also be indirectly received by receiving a detection request sent by a user, for example, the detection request sent by the user can be directly obtained, the detection request can directly carry the automatic driving program to be detected, and the automatic driving program to be detected is extracted from the obtained detection request. When the memory of the to-be-detected automatic driving program is large, the user can store the to-be-detected automatic driving program in a third-party database, and the storage address of the to-be-detected automatic driving program is carried in the detection request. After receiving the detection request carrying the storage address, the autopilot performance detection device acquires the autopilot to be detected from the third-party database according to the storage address, and after successfully acquiring the autopilot to be detected, the autopilot performance detection device can also send feedback information to the user or a terminal corresponding to the user so as to prompt the user or the terminal.
And S2, loading the automatic driving program to be detected to a preset automatic driving system.
The loading can be understood as adding the autopilot system to be detected to the autopilot package of the preset autopilot system, and therefore, the loading can also be understood as updating the autopilot program in the preset autopilot system. The automatic driving program package can be the integration or collection of all software programs in the automatic driving system of the automatic driving vehicle, and the various automatic driving programs in the automatic driving program package jointly form the software part of the automatic driving system and cooperate with the hardware of the automatic driving system (a camera, a laser radar, a millimeter wave radar, a switch and the like) to jointly complete the real-time control and operation of the automatic driving vehicle.
For example, an identity of an automatic driving program to be detected is obtained, an automatic driving program which is not updated and corresponds to the identity is obtained in an automatic driving program package according to the identity, the automatic driving program which is not updated is replaced by the automatic driving program to be detected, updating of the automatic driving program package is completed, the updated automatic driving program package is obtained, for example, the automatic driving program to be detected is a perception type automatic driving program a, the perception type automatic driving program a which is not updated is inquired in the automatic driving program package, the automatic driving program a which is not updated in the automatic driving program package is replaced by the automatic driving program a to be detected, and updating of the automatic driving program package is completed at this moment. And if the perception type automatic driving program A which is not updated and corresponds to the identity identification is not inquired in the automatic driving program package, the perception type automatic driving program A to be detected is the automatic driving program which is newly added in the automatic driving program package, the automatic driving program to be detected is directly added into the automatic driving program package, the updating of the automatic driving program package is completed, and the updated automatic driving program package is obtained.
And S3, carrying out simulation driving test on the loaded automatic driving system to obtain a driving data set of each automatic driving program.
The simulation driving test can be understood as that in a virtual test environment, the automatic driving system loaded with the automatic driving program to be detected is tested in various test items to obtain a driving data set of the automatic driving system in simulation driving in various test items. The driving data set may include a set of data of a driving speed, a driving position, a recognition result of recognizing a traffic signal, a recognition result of recognizing an obstacle, a time taken to recognize a traffic signal and an obstacle, and/or a feedback operation action size when a traffic signal is recognized, of the vehicle under various test items.
(1) And determining the test items of the automatic driving program to be detected according to the type of the automatic driving program to be detected.
For example, a program type corresponding to the to-be-detected automatic driving program is determined, and according to the determined program type, a test item corresponding to the program type is screened from a preset test item set, for example, the to-be-detected automatic driving program is a traffic light detection program, the type of the to-be-detected automatic driving program can be determined to be a perception detection type automatic driving program, and then a test item corresponding to a traffic light detection of the perception detection type is screened from the preset test item set, for example, the test item can be a test item for identifying a color of a traffic light.
(2) And adding the test items into the historical test item set of the updated automatic driving program package to obtain the current test item set of the updated automatic driving program package.
The historical test item set may be a set of test items tested by the updated autopilot package before updating. For example, the autopilot package needs to be deployed to a real autopilot vehicle before it is updated, and the set of historical test items is the set of all test items tested at that time, which must be tested.
For example, the test item is added to the historical test item set of the updated autopilot package, for example, the autopilot to be detected is used as the autopilot corresponding to the traffic light detection, the test item such as the identification of the color of the traffic light is added to the historical test item set, whether the test item to be added exists in the historical test item set is detected, if the same test item exists, the test item is not added, and if the same test item does not exist, the test item is added to the historical test item set, so that the current test item set of the updated autopilot package is obtained.
(3) And screening out a test scene corresponding to the current test item set from a preset test scene set.
The test scene may be a virtual environment or a virtual road simulating the operation of the autonomous vehicle, for example, the test scene may be in rainy days, nights or daytime, the wind speed, the rainfall and/or the road surface condition may be set, and the test scene may also be a traffic sign and a signal lamp on the road.
For example, a test scenario corresponding to the current test item set is screened from a preset test scenario set, for example, for the detection of traffic lights as test items, traffic lights such as traffic lights on roads need to be screened out from a preset scene set, if the test items are the classification of the obstacles, the obstacles such as pedestrians, vehicles or sundries need to be screened out from the preset scene set, if the test item is the speed in the rain, the driving environment needs to be screened from the preset scene set to be the rainy day, all the test items are screened from the preset test scene set to be corresponding test scenes, and the test scenes are combined to obtain the test scene corresponding to the current test item set, for example, the driving environment is the rainy day, one or more traffic lights exist on a road, and partial obstacles such as pedestrians and other vehicles need to exist.
(4) And under a test scene, according to the test items in the current test item set, carrying out simulation driving test on the loaded automatic driving system to obtain a driving data set of each automatic driving program.
For example, in a test scene, according to a test item in a current test item set, a simulation driving test is performed on the loaded automatic driving system to obtain a driving data set, for example, a driving environment of the screened test scene may be rainy day, one or more traffic lights exist on a road, and some obstacles such as pedestrians and other vehicles need to exist. In the test environment, the loaded automatic driving system is subjected to a simulation driving test according to test items in a current test item set, for example, the test items are taken as test items for identifying traffic light signals, the loaded automatic driving system is subjected to the simulation driving test, when a camera in hardware detects the traffic light signals, various information such as current vehicle speed, vehicle position, identification results of the traffic lights, time spent in identification, feedback actions made by the vehicle according to the identification results after the traffic lights are identified, corresponding control quantity and the like can be collected, the data are taken as driving data in the driving data set, and the driving data corresponding to each automatic driving program are combined into a corresponding driving data set according to the type of each automatic driving program.
The feedback action may be that when the recognition result is a red light, the vehicle needs to be decelerated, the feedback action is used to operate a brake pedal, and the corresponding control amount is a pedal variation amount.
And S4, identifying target running data in the running data set based on the automatic driving program type.
The target travel data may be necessary data that greatly affects the automatic driving performance of the automatic driving system. The automatic driving system is most important in safety that determines the accuracy and response time of each automatic driving program, and thus the most important indexes are the accuracy of calculation, calculation performance (i.e., calculation speed), and feedback operation of each automatic driving program, and thus, the necessary data may be data related to the accuracy of calculation, calculation performance, and feedback operation of detecting a driving signal of the automatic driving program. For example, the identification result, the identification time and/or the feedback action after the identification of the traffic light output by the traffic light detection type automatic driving program, the classification information (such as vehicle type, size, color, and the like), the position information (longitudinal and latitudinal coordinates) and/or the identification time of the object which is promised to be output by the obstacle detection type automatic driving, and the accuracy index of the positioning type automatic driving program comprises the data of errors (standard deviation, variance, and the like) of six degrees of freedom (transverse, longitudinal, height, roll angle, pitch angle, heading angle). The corresponding unnecessary data may be data that is not related to the accuracy of the calculation of the automatic driving program, the calculation performance, and the feedback operation of detecting the driving signal, such as data format, the number of data, and the version number of the preset automatic driving system, among other information.
For example, based on the type of the automatic driving program and the data type of the driving data, the target driving data is identified in the driving data set, for example, the necessary data is mainly data related to the accuracy, the time-ductility (calculation performance) and the feedback action of the automatic driving calculation in each automatic driving program, so that the data for obtaining the accuracy, the time-ductility (calculation performance) and the feedback action of the automatic driving calculation, for example, the detection accuracy, the recall rate, the calculation speed and the pedal variation, etc., taking the target driving data of the automatic driving program of the traffic light detection type to be identified as an example, the target data to be identified can be the identification result, the number and the real result corresponding to the detection accuracy, and the target data to be identified can be the time taken from the start of identification to the end of identification for a single traffic light from the time delay (calculation performance), And identifying the time required by all the traffic lights and the number of the traffic lights, wherein for the feedback action, the data required to be identified can be the feedback action of the vehicle when the red light is identified, the feedback action of the vehicle when the green light is identified and the feedback action of the vehicle when the yellow light is identified. For example, after the red light is identified, the vehicle needs to be decelerated, and the feedback action corresponding to the deceleration of the vehicle can generate the pedal variation amount of the deceleration for the corresponding movement of the accelerator pedal. Since the type of the target data corresponding to each of the automatic driving programs is different, it is necessary to identify data reflecting the accuracy, the time-lag (calculation performance), and the feedback operation of the automatic driving calculation corresponding to each of the automatic driving programs in the travel data set according to the type of the automatic driving program, and use these data as the target travel data for each of the automatic driving programs.
102. And performing feature extraction on the target driving data to obtain an initial feature value of the target driving data.
The initial characteristic value may be an index value corresponding to accuracy (precision) of the automatic driving program calculation, time-lapse performance (calculation performance), and a feedback action, for example, an index value such as a detection accuracy, a detection recall rate, a detection performance, an identification performance, an accelerator/brake pedal variation, a steering wheel rotation angle, and a steering wheel rotation direction.
For example, the feature extraction may be understood as calculating an index value corresponding to accuracy, time delay and feedback action according to the collected raw data, for example, for target driving data corresponding to a traffic light detection automatic driving program, feature extraction is performed on the target driving data, and the index value corresponding to the accuracy, time delay and feedback action is extracted. For example, the index value corresponding to the accuracy may be a traffic light identification accuracy and an identification recall rate, and to extract the identification accuracy and the identification recall rate, the number of times of correct identification and the number of times of incorrect identification need to be determined according to the identification result, and the identification accuracy can be obtained by dividing the number of times of correct identification by the total number of times of identification. Similar calculations may be used to identify recall. The index value corresponding to the time-ductility can be the recognition performance (recognition speed), and the recognition performance (recognition speed) needs to be calculated based on the total number of times of recognition and the time taken for each or all recognition operations. The index value corresponding to the feedback action can be the brake/accelerator pedal variation, the steering wheel rotation angle and the steering wheel rotation direction, and when the index value corresponding to the feedback exists in the target driving data corresponding to the red and green light detection automatic driving program, the index value corresponding to the feedback is directly screened from the target driving data. The target travel data corresponding to the other automatic driving programs is calculated in the same manner as described above. The characteristic extraction is mainly to calculate or screen out index values corresponding to accuracy, time delay and feedback actions according to collected target driving data, and the index values are used as initial characteristic values.
103. And classifying the initial characteristic values, and determining the weight corresponding to the initial characteristic values based on the classification result.
(1) The data type of the initial feature value is identified.
The data type can be a physical or mathematical meaning of the data, for example, the data types of the data such as identification accuracy, identification recall rate and the like can be understood as calculation accuracy data, the data such as calculation speed, calculation performance and the like can be understood as calculation performance data, and the calculation performance data is mainly used for indicating the calculation speed of the target driving data, for example, the traffic light is identified, and the faster the identification is, the faster the calculation speed of the traffic light detection automatic driving program is, the faster the corresponding calculation performance is. Data specific to automatic driving, such as the amount of pedal change and the steering wheel rotation angle/direction, mainly correspond to data corresponding to the magnitude of motion during feedback operation, and the data type of such data may be feedback control data.
For example, the data type of the initial feature values is identified, for example, when the initial feature values are the initial feature values such as detection accuracy, detection recall, identification accuracy, identification recall, classification accuracy and classification recall, the data type of the initial feature values is marked as the calculation accuracy data. When the initial characteristic values are the initial characteristic values such as classification performance, recognition performance and detection performance, the data types of the initial characteristic values are marked as calculation performance data. When the initial characteristic values are characteristic values such as accelerator/brake pedal variation and steering wheel rotation angle/direction, the data types of these initial characteristic values are marked as feedback control data.
(2) And classifying the initial characteristic values according to the identification result.
For example, the initial characteristic values are classified according to the identified data type of each initial characteristic value, for example, when the data type is calculation accuracy data, the initial characteristic value corresponding to the data type is classified as an accurate initial characteristic value, when the data type is calculation performance data, the initial characteristic value corresponding to the data type is classified as a delay type initial characteristic value, and when the data type is feedback control data, the initial characteristic value corresponding to the data type is classified as a feedback type initial characteristic value.
(3) And determining the weight corresponding to the initial characteristic value based on the classification result.
For example, the weight corresponding to the initial characteristic value is determined according to the preset weight corresponding to the delay type initial characteristic value, the accurate type initial characteristic value and the feedback type initial characteristic value. The time delay type weight corresponding to the time delay type initial characteristic value is the largest in the three weights, the precision type weight corresponding to the precision type initial characteristic value is the second in the three weights, and the feedback type weight corresponding to the feedback type initial characteristic value is the smallest in the three weights. For example, the preset delay-type weight corresponding to the initial characteristic value of delay is X, the precision-type weight Y corresponding to the precision-type initial characteristic value, and the feedback-type weight Z corresponding to the feedback-type initial characteristic value, X > Y > Z, then when the type of the initial characteristic value a is the delay-type initial characteristic value, the weight of the initial characteristic value a is determined to be X, when the type of the initial characteristic value a is the precision-type initial characteristic value, the weight of the initial characteristic value a is determined to be Y, and when the type of the initial characteristic value a is the feedback-type initial characteristic value, the weight of the initial characteristic value a is determined to be Z.
104. And screening out a target characteristic value from the initial characteristic values according to the weight.
For example, according to the time-delay type weight, the precise type weight and the feedback type weighting information, the target characteristic value is screened out from the initial characteristic values, for example, the initial characteristic values extracted by the characteristic extraction of the target driving data corresponding to the automatic driving program for detecting the traffic lights have identification accuracy, detection accuracy, identification performance, detection performance and pedal variation, the time-delay type weight X corresponding to the preset time-delay type initial characteristic value, the precise type weight Y corresponding to the precise type initial characteristic value and the feedback type weight Z corresponding to the feedback type initial characteristic value are taken as examples, since the time-delay type weight X is greater than the precise type weight Y and is also greater than the feedback type weight Z, the target characteristic value can be screened out from the initial characteristic values according to the weight magnitude, for example, for each automatic driving program, 10 characteristic values can be screened out from the initial characteristic values as the target characteristic values, according to the weight, more initial eigenvalues can be selected from the initial eigenvalues corresponding to the delay-type weight X as target eigenvalues, and less initial eigenvalues can be selected from the initial eigenvalues corresponding to the feedback-type weight Z as target eigenvalues, for example, 5 initial eigenvalues can be selected from the delay-type initial eigenvalues as target eigenvalues, 3 initial eigenvalues can be selected from the accurate initial eigenvalues as target eigenvalues, and 2 initial eigenvalues can be selected from the feedback-type eigenvalues as target eigenvalues.
105. And analyzing the target characteristic value to obtain the comprehensive driving performance parameters of the preset automatic driving system loaded with the automatic driving program to be detected.
The comprehensive driving performance parameter is a comprehensive quantitative index for measuring control accuracy, intelligence or sensitivity of an automatic driving system of the automatic driving vehicle, and generally, the index value is a value interval, such as a positive integer of [1,10], and the higher the value is, the stronger the comprehensive driving performance is.
For example, the target feature values may be weighted according to weighting coefficients corresponding to the automatic driving programs, the weighted target feature values may be normalized to obtain normalized target feature values, the format of the normalized target feature values may be converted to obtain first target feature data, for example, the target feature values corresponding to the automatic driving programs may be weighted, the weighted target feature values may be normalized, the normalized array may be converted into an N × N-dimensional input matrix, and the N × N-dimensional input matrix may be used as the first target feature data.
Analyzing the first target characteristic data by adopting a discrete model sub-model of the trained analysis model, for example, performing multi-dimensional characteristic extraction on the first target characteristic data to obtain local characteristic information corresponding to the first target characteristic data, fusing the local characteristic information to obtain fused characteristic information, and calculating comprehensive driving performance parameters of a preset automatic driving system loaded with the automatic driving program to be detected according to the fused characteristic information. For example, taking a matrix with 32 × 32 dimensions as the first target test data as an example, the whole analysis process is as shown in fig. 3, a feature extraction module composed of one or more convolution layers and a sampling layer (pooling layer) may be used to perform multi-scale feature extraction on the 32 × 32 dimensional matrix to obtain local feature information of the matrix, a full connection layer may also be used to fuse the local feature information to obtain fused feature information, and according to the fused feature information, a value area of a comprehensive drivability parameter may be preset in a discrete model sub-model of the analysis model after training, for example, the value area may be 1 to 10, and the larger the value of the comprehensive drivability parameter is, the better the comprehensive drivability of the automatic driving system is. Analyzing the fused characteristic information, judging the comprehensive drivability parameter corresponding to the fused characteristic information in the value range of the preset comprehensive drivability parameter, outputting the comprehensive drivability parameter in the value range of the preset comprehensive drivability parameter, and using the comprehensive drivability parameter as the comprehensive drivability parameter of the preset automatic driving system loaded with the automatic driving program to be detected. 106. And when the comprehensive driving performance parameter exceeds a preset first parameter threshold value, determining that the automatic driving performance detection of the automatic driving program to be detected is qualified.
For example, when the comprehensive drivability parameter exceeds a preset first parameter threshold, it is determined that the autopilot performance of the autopilot to be detected is detected as being qualified. And after the automatic driving performance is detected to be qualified, analyzing the target characteristic value by adopting a continuous model submodel of the trained analysis model to obtain the manual intervention parameters of the preset automatic driving system loaded with the automatic driving program to be detected. Specifically, the following may be mentioned:
and C1, when the comprehensive driving performance parameter exceeds a preset parameter threshold value, determining that the automatic driving performance detection of the automatic driving program to be detected is qualified.
For example, the comprehensive drivability parameter of the preset autopilot system loaded with the autopilot to be detected obtained by the analysis is compared with a preset parameter threshold, and when the comprehensive drivability parameter exceeds the preset parameter threshold, it is determined that the autopilot to be detected of the autopilot to be detected is qualified, and the autopilot to be detected can be deployed in an autopilot system of an actual vehicle. And when the comprehensive driving performance parameter does not exceed the preset parameter threshold value, determining that the automatic driving performance of the automatic driving program to be detected is unqualified, the automatic driving performance of the automatic driving program to be detected cannot be directly deployed in an automatic system of an actual vehicle, and the automatic driving performance of the automatic driving program to be detected can be continuously detected after the modification or the adjustment is finished.
And C2, when the automatic driving performance of the automatic driving program to be detected is qualified, analyzing the target characteristic value by adopting a continuous model submodel of the trained analysis model to obtain the manual intervention parameters of the preset automatic driving system loaded with the automatic driving program to be detected.
The manual intervention parameter may be understood as a disturbance frequency (MPI) of manual intervention on the autonomous driving system, and the disturbance frequency may include a coefficient of how many miles the autonomous driving system needs to intervene on average, and may include an average of a ratio of the number of manual interventions to a running time of the autonomous driving vehicle during running.
For example, when the comprehensive driving performance parameter exceeds a preset parameter threshold value and it is determined that the automatic driving performance of the automatic driving program to be detected is qualified, the target characteristic values are weighted according to the weighting coefficients corresponding to the automatic driving program, the format of the weighted target characteristic values is converted to obtain second target characteristic data, for example, the number group of the weighted characteristic values corresponding to each automatic driving program can be converted into an n × k dimensional matrix, the matrix can represent n times of sampled comprehensive test data of k automatic driving programs, and the n × k dimensional matrix is used as the second target characteristic data.
And determining a mapping relation between the second target characteristic data and the manual intervention parameters (MPI) according to the continuous model submodel of the trained analytic model, for example, fitting a curve by adopting a least square method based on test data collected by a preset automatic driving system on a real vehicle before loading the automatic driving program to be detected, and constructing the continuous model submodel according to the fitted curve. Determining a mapping relationship between the second target test data and the manual intervention parameter according to the continuous model submodel, for example, the mapping relationship may be:
X*A＝Y
wherein, X is a matrix of n X k dimension, A is a coefficient vector of k X1, Y is a vector of n X1, where n is the number of actual sampling times, and when n is 1, it refers in particular to an adjustment coefficient of the automatic driving performance of the automatic driving vehicle.
And calculating the manual intervention parameter (MPI) of the preset automatic driving system loaded with the automatic driving program to be detected according to the mapping relation and the second target characteristic data.
Among them, the continuous model submodel predicts the actual performance of the automatic driving system based on the initial characteristic values (index values) of the respective automatic driving programs, and focuses on predicting the future performance of the driving comprehensive ability. The discrete model submodel emphasizes on evaluating the comprehensive driving performance of the automatic driving system based on various initial characteristic values of respective automatic driving programs, aims to find the influence relationship of each automatic driving program on the driving performance parameter indexes of the automatic driving system, and emphasizes on evaluating and searching the influence relationship.
The relation between the MPI and the comprehensive driving performance parameters, the MPI is one of the index values of the comprehensive automatic driving capacity of the whole automatic driving system, and the driving performance parameters of a single automatic driving program in the automatic driving system to the whole system can be objectively evaluated. Therefore, when the target characteristic data is analyzed in a discrete mode, the obtained data is the automatic driving comprehensive capacity of the whole automatic driving system, the automatic driving comprehensive capacity is used as an initial driving performance parameter of the automatic driving program to be detected relative to the automatic driving system, the higher the automatic driving comprehensive capacity is, the larger the MPI value is, and otherwise, the lower the MPI value is; the lower the MPI value is, the lower the automated driving compatibility is certainly. When the MPI value is large, the comprehensive capacity is not necessarily high, and the complexity of a specific driving environment needs to be considered. The MPI value is consistent with the automated driving performance only when the driving environment is the same, in which case the greater the MPI value, the higher the automated driving performance, and vice versa.
The analysis model after training can be set according to actual needs. In addition, it should be noted that the analysis model after training may be set in advance by a maintenance person, or may be trained by the automatic driveability detection apparatus, that is, before the step "before analyzing the target characteristic value by using the discrete model sub-model of the analysis model after training", the automatic driveability detection method may further include:
(1) historical target characteristic data and MPI of the real driving of a preset automatic driving system on an automatic driving vehicle are collected.
For example, the preset autonomous driving system may be deployed on the autonomous vehicle before the to-be-detected autonomous driving program is loaded, and thus, historical target characteristic data of actual driving of the preset autonomous driving system on the autonomous vehicle may be collected.
(2) And marking the corresponding actual MPI on the historical target characteristic data, and taking the historical target characteristic data marked with the actual MPI as a target data sample.
For example, the historical target feature data is labeled with the corresponding actual MPI. And taking the historical target characteristic data marked with the actual MPI as a target data sample.
(3) And analyzing the target data sample by adopting a preset analysis model to obtain a predicted manual intervention parameter of a preset automatic driving system.
For example, according to the continuous model submodel, determining a mapping relation between the second target characteristic data and the MPI, and predicting the predicted MPI of the preset automatic driving system based on the second target characteristic data and the mapping relation;
for constructing the continuous submodel, when the coefficient vector A needs to be calculated in the mapping relation between the second target characteristic data and the MPI, the coefficient vector A can be calculated by collecting multiple test data and actual data of the automatic driving system. The specific calculation process may be as follows:
A＝(X′*X)-1*X′*Y
wherein X' and X are test data collected twice or more, and Y is actual data.
By the formula, the coefficient vector A can be solved, and then continuous model data can be fitted into a curve, so that a continuous model submodel is obtained.
(4) And converging the preset analytical model according to the predicted MPI and the actual MPI to obtain the trained analytical model.
For example, in the embodiment of the present application, the preset analytic model may be converged according to the predicted MPI and the labeled actual MPI through an interpolation loss function, so as to obtain a trained analytic model. For example, the following may be specifically mentioned:
and adjusting parameters for calculating the comprehensive performance parameters and the MPI of the preset automatic driving system in the preset analytical model according to the predicted MPI and the actual MPI in the target data sample by adopting a Dice function (a loss function), and adjusting the parameters for calculating the comprehensive performance parameters and the MPI of the preset automatic driving system in the preset analytical model according to the predicted MPI and the labeled actual MPI in the target data sample by adopting an interpolation loss function to obtain the trained analytical model.
Optionally, in order to improve the accuracy of the autopilot performance parameter, besides the rice function, other loss functions such as a cross entropy loss function may be used for convergence, which may specifically be as follows:
and adjusting parameters used for calculating the comprehensive performance parameters and the MPI of the preset automatic driving system in the preset analysis model according to the predicted MPI and the marked actual MPI in the target data sample by adopting a cross entropy loss function, and adjusting the parameters used for calculating the comprehensive performance parameters and the MPI of the preset automatic driving system in the preset analysis model according to the predicted MPI and the actual MPI in the target data sample by adopting an interpolation loss function to obtain the trained analysis model.
Because the MPI and the comprehensive driving performance parameters have an incidence relation, when the analytical model is trained, the actual MPI is acquired, which is equivalent to the actual MPI and the actual comprehensive driving performance parameters, so that the training of the discrete model submodel and the continuous model submodel of the analytical model can be completed only by acquiring the historical MPI value of the real vehicle in running.
As can be seen from the above, in the embodiment of the present invention, after the driving data set of each automatic driving program for performing simulation driving after the automatic driving program to be detected is loaded to the preset automatic driving system is acquired, the target driving data is identified in the driving data set based on the type of the automatic driving program, and the target driving data is subjected to feature extraction to obtain the initial feature value of the target driving data; classifying the initial characteristic values, and determining weights corresponding to the initial characteristic values based on classification results; screening out a target characteristic value from the initial characteristic values according to the weight, then analyzing the target characteristic value to obtain a comprehensive drivability parameter of a preset automatic driving system loaded with the automatic driving program to be detected, and determining that the automatic driving performance detection of the automatic driving program to be detected is qualified when the comprehensive drivability parameter exceeds a preset parameter threshold value; because the scheme is used for carrying out simulation driving test on the whole automatic driving system and fully considering the influence of different types of simulation data on the automatic driving performance, the processing mode of classifying and weighting the data in the simulation driving is adopted, and the test efficiency and the accuracy of the automatic driving vehicle can be greatly improved.
The method described in the above examples is further illustrated in detail below by way of example.
In this embodiment, the automatic drivability detection apparatus is specifically integrated into an electronic device, and the electronic device is a server as an example.
As shown in fig. 4, an automatic driveability detection method includes the following specific processes:
201. the terminal deploys a preset autonomous driving system to the autonomous driving vehicle.
For example, the terminal may deploy a preset autopilot system to the autopilot vehicle, where an autopilot package in the preset autopilot system has passed the detection, and may complete control of the autopilot vehicle, for example, the server may send the autopilot package in the preset autopilot system to the terminal, and the terminal deploys the autopilot program in the autopilot vehicle.
202. The server collects the MPI value of the automatic driving vehicle during running through a preset automatic driving system.
For example, the server may collect real-time data of the autonomous vehicle in the current operation process through an autonomous driving program in a preset autonomous driving system, and obtain an MPI value of the autonomous vehicle from the collected real-time data, for example, by detecting an autonomous driving program for manual intervention, recording an intervention condition of the autonomous vehicle by a human in real time, when the intervention condition exists, collecting mileage and the intervention condition that are traveled at the time, and after the current operation is completed, obtaining a final MPI value of the current operation through a total mileage and a total intervention number.
203. And the server trains a preset analytical model according to the MPI value and the historical operating data acquired at this time to obtain a trained analytical model.
For example, the server trains a preset analytic model according to the collected MPI value and historical operating data to obtain a trained analytic model, which may specifically be as follows:
(1) the server obtains historical target characteristic data of real driving of a preset automatic driving system on the automatic driving vehicle.
For example, the preset autonomous driving system may be deployed on the autonomous vehicle before the autonomous driving program to be detected is loaded, and thus, the server may obtain historical target feature data of the actual driving of the preset autonomous driving system on the autonomous vehicle.
(2) And the server marks the corresponding actual MPI for the historical target characteristic data, and takes the historical target characteristic data of the actual MPI as a target data sample.
For example, the server labels the corresponding actual MPI to the historical target feature data. And taking the historical target characteristic data marked with the actual MPI as a target data sample.
(3) And the server analyzes the target data sample by adopting a preset analysis model to obtain the predicted driving performance parameters and the predicted adjustment coefficients of the preset automatic driving system.
For example, the server determines a mapping relationship between the second target characteristic data and the MPI, and predicts the MPI of the preset automatic driving system based on the second target characteristic data and the mapping relationship.
For constructing the continuous submodel, when the coefficient vector A needs to be calculated in the mapping relation between the second target characteristic data and the MPI, the coefficient vector A can be calculated by collecting multiple test data and actual data of the automatic driving system. The specific calculation process may be as follows:
A＝(X′*X)-1*X′*Y
wherein X' and X are test data collected twice or more, and Y is actual data.
By the formula, the coefficient vector A can be solved, and then continuous model data can be fitted into a curve, so that a continuous model submodel is obtained.
(4) And the server converges the preset analytical model according to the predicted MPI and the marked actual MPI to obtain the trained analytical model.
For example, in the embodiment of the present application, the server may converge the preset analytic model according to the predicted MPI and the labeled actual MPI through an interpolation loss function, so as to obtain the trained analytic model. For example, the following may be specifically mentioned:
and adjusting parameters used for calculating the comprehensive performance parameters and the MPI of the preset automatic driving system in the preset analytical model according to the predicted MPI and the marked actual MPI in the target data sample by adopting a Dice function, and adjusting the parameters used for calculating the comprehensive performance parameters and the MPI of the preset automatic driving system in the preset analytical model according to the predicted MPI and the marked actual MPI in the target data sample by adopting an interpolation loss function to obtain the trained analytical model.
Optionally, in order to improve the accuracy of the autopilot performance parameter, besides the rice function, other loss functions such as a cross entropy loss function may be used for convergence, which may specifically be as follows:
and adjusting parameters used for calculating the comprehensive performance parameters and the MPI of the preset automatic driving system in the preset analytical model according to the predicted MPI and the marked actual MPI in the target data sample by adopting a cross entropy loss function, and adjusting the parameters used for calculating the comprehensive performance parameters and the MPI of the preset automatic driving system in the preset analytical model according to the predicted MPI and the marked actual MPI in the target data sample by adopting an interpolation loss function to obtain the trained analytical model.
204. The method comprises the steps that a server obtains a driving data set of each automatic driving program which loads the automatic driving program to be detected to a preset automatic driving system and then carries out simulation driving, and target driving data are identified in the driving data set based on the type of the automatic driving program.
For example, the server acquires the automatic driving programs to be detected, loads the automatic driving programs to be detected to a preset automatic driving system, performs simulation test on the loaded automatic driving system to obtain a driving data set of each automatic driving program, and identifies target driving data in the driving data set based on the type of the automatic driving program. Specifically, the following may be mentioned:
and B1, the server acquires the automatic driving program to be detected.
For example, the server may directly receive the to-be-detected autopilot submitted or uploaded by the user. The automatic driving program to be detected can also be indirectly received by receiving a detection request sent by a user, for example, the detection request sent by the user can be directly obtained, the detection request can directly carry the automatic driving program to be detected, and the automatic driving program to be detected is extracted from the obtained detection request. When the memory of the to-be-detected automatic driving program is large, the user can store the to-be-detected automatic driving program in a third-party database, and the storage address of the to-be-detected automatic driving program is carried in the detection request. After receiving the detection request carrying the storage address, the autopilot performance detection device acquires the autopilot to be detected from the third-party database according to the storage address, and after successfully acquiring the autopilot to be detected, the autopilot performance detection device can also send feedback information to the user or a terminal corresponding to the user so as to prompt the user or the terminal.
And B2, the server loads the automatic driving program to be detected to a preset automatic driving system.
For example, taking the to-be-detected automatic driving program as the perception type automatic driving program a as an example, the server queries the perception type automatic driving program a which is not updated in the automatic driving program package, replaces the perception type automatic driving program a which is not updated in the automatic driving program package with the to-be-detected automatic driving program a, and at this time, the updating of the automatic driving program package is completed. And if the perception type automatic driving program A which is not updated and corresponds to the identity identification is not inquired in the automatic driving program package, the perception type automatic driving program A to be detected is the automatic driving program which is newly added in the automatic driving program package, the automatic driving program to be detected is directly added into the automatic driving program package, the updating of the automatic driving program package is completed, and the updated automatic driving program package is obtained.
And B3, the server tests the simulation driving of the loaded automatic driving system to obtain a driving data set of each automatic driving program.
(1) And the server determines the test items of the automatic driving program to be detected according to the type of the automatic driving program to be detected.
For example, the server determines a program type corresponding to the to-be-detected automatic driving program, and screens out a test item corresponding to the program type from a preset test item set according to the determined program type, for example, the to-be-detected automatic driving program is a traffic light detection program, and the to-be-detected automatic driving program can be determined to be a perception detection type automatic driving program, and then screens out a test item corresponding to the perception detection type traffic light detection from the preset test item set, for example, the test item can be a test item for identifying the color of the traffic light.
(2) And the server adds the test items to the historical test item set of the updated automatic driving program package to obtain the current test item set of the updated automatic driving program package.
For example, the automatic driving program to be detected is used as the automatic driving program corresponding to the traffic light detection, the server adds test items such as the identification of the color of the traffic light in the test to the historical test item set, detects whether the test items to be added exist in the historical test item set, if the same test items exist, the test items are not added, if the same test items do not exist, the test items are added to the historical test item set, and the current test item set of the updated automatic driving program package is obtained.
(3) And the server screens out the test scene corresponding to the current test item set from the preset test scene set.
For example, the server screens out the test scenario corresponding to the current test item set from the preset test scenario set, for example, for example, when the test item is a traffic light, traffic lights such as traffic lights on roads need to be screened from the preset scene set, if the test items are the classification of the obstacles, the obstacles such as pedestrians, vehicles or sundries need to be screened out from the preset scene set, if the test item is the speed in the rain, the driving environment is screened from the preset scene set to be the rainy day, all the test items are screened from the preset test scene set to obtain corresponding test scenes, the test scenes are combined to obtain the test scene corresponding to the current test item set, for example, the driving environment can be rainy days, one or more traffic lights exist on the road, and partial obstacles such as pedestrians, other vehicles and the like need to exist.
(4) And under a test scene, the server performs simulation driving test on the loaded automatic driving system according to the test items in the current test item set to obtain a driving data set of each automatic driving program.
For example, the driving environment of the screened test scene is rainy day, one or more traffic lights exist on the road, and some obstacles such as pedestrians and other vehicles need to exist. In the test environment, the server performs a simulation driving test on the loaded automatic driving system according to the test items in the current test item set, for example, taking the test items as the identification items of the traffic lights, the simulation driving test on the loaded automatic driving system is performed, when a camera in hardware detects signals of the traffic lights, various information such as the current vehicle speed, the vehicle position, the identification results of the traffic lights, the time spent in the identification, the feedback action made by the vehicle according to the identification results after the identification results of the traffic lights and the corresponding control quantity can be collected, the data are taken as driving data in the driving data combination, and the driving data corresponding to each automatic driving program are combined into a corresponding driving data set according to the type of each automatic driving program.
And B4, the server identifies target running data in the running data set based on the type of the automatic driving program.
For example, the server identifies target driving data in the driving data set based on the type of the automatic driving program and the data type of the driving data, for example, the necessary data is mainly data related to the accuracy, the time-ductility (calculation performance) and the feedback action of the automatic driving calculation in each automatic driving program, so that the data for acquiring the accuracy, the time-ductility (calculation performance) and the feedback action of the automatic driving calculation, for example, the detection accuracy, the recall rate, the calculation speed, the pedal variation and the like, taking the target driving data of the automatic driving program of the traffic light detection type as an example, the target data to be identified can be the identification result, the number and the real result corresponding to the detection accuracy, and the time for the time delay (calculation performance), the target data to be identified can be the time for a single traffic light from the start of identification to the end of identification, And identifying the time required by all the traffic lights and the number of the traffic lights, wherein for the feedback action, the data required to be identified can be the feedback action of the vehicle when the red light is identified, the feedback action of the vehicle when the green light is identified and the feedback action of the vehicle when the yellow light is identified. For example, after the red light is identified, the vehicle needs to be decelerated, and the feedback action corresponding to the deceleration of the vehicle can generate the pedal variation amount of the deceleration for the corresponding movement of the accelerator pedal. Since the type of the target data corresponding to each of the automatic driving programs is different, it is necessary to identify data reflecting the accuracy, the time-lag (calculation performance), and the feedback operation of the automatic driving calculation corresponding to each of the automatic driving programs in the travel data set according to the type of the automatic driving program, and use these data as the target travel data for each of the automatic driving programs.
205. And the server performs characteristic extraction on the target driving data to obtain an initial characteristic value of the target driving data.
For example, taking the target driving data corresponding to the traffic light detection automatic driving program as an example, the server performs feature extraction on the target driving data to extract an index value corresponding to accuracy, time delay and feedback action. For example, the index value corresponding to the accuracy may be a traffic light identification accuracy and an identification recall rate, and to extract the identification accuracy and the identification recall rate, the number of times of correct identification and the number of times of incorrect identification need to be determined according to the identification result, and the identification accuracy can be obtained by dividing the number of times of correct identification by the total number of times of identification. Similar calculations may be used to identify recall. The index value corresponding to the time-ductility can be the recognition performance (recognition speed), and the recognition performance (recognition speed) needs to be calculated based on the total number of times of recognition and the time taken for each or all recognition operations. The index value corresponding to the feedback action can be the brake/accelerator pedal variation, the steering wheel rotation angle and the steering wheel rotation direction, and when the index value corresponding to the feedback exists in the target driving data corresponding to the red and green light detection automatic driving program, the index value corresponding to the feedback is directly screened from the target driving data. The target travel data corresponding to the other automatic driving programs is calculated in the same manner as described above. The characteristic extraction is mainly to calculate or screen out index values corresponding to accuracy, time delay and feedback actions according to collected target driving data, and the index values are used as initial characteristic values.
206. And the server classifies the initial characteristic values and determines the weight corresponding to the initial characteristic values based on the classification result.
(1) The server identifies the data type of the initial characteristic value.
For example, when the initial feature values are such initial feature values as detection accuracy, detection recall, identification accuracy, identification recall, classification accuracy, and classification recall, the server marks the data types of these initial feature values as calculation accuracy data. When the initial characteristic values are the initial characteristic values such as classification performance, recognition performance and detection performance, the data types of the initial characteristic values are marked as calculation performance data. When the initial characteristic values are characteristic values such as accelerator/brake pedal variation and steering wheel rotation angle/direction, the data types of these initial characteristic values are marked as feedback control data.
(2) And the server classifies the initial characteristic values according to the identification result.
For example, the initial characteristic values are classified according to the identified data type of each initial characteristic value, for example, when the data type is calculation accuracy data, the initial characteristic value corresponding to the data type is classified as an accurate initial characteristic value, when the data type is calculation performance data, the initial characteristic value corresponding to the data type is classified as a delay type initial characteristic value, and when the data type is feedback control data, the initial characteristic value corresponding to the data type is classified as a feedback type initial characteristic value.
(3) And determining the weight corresponding to the initial characteristic value based on the classification result.
For example, the weight corresponding to the initial characteristic value is determined according to the preset weight corresponding to the delay type initial characteristic value, the accurate type initial characteristic value and the feedback type initial characteristic value. The time delay type weight corresponding to the time delay type initial characteristic value is the largest in the three weights, the precision type weight corresponding to the precision type initial characteristic value is the second in the three weights, and the feedback type weight corresponding to the feedback type initial characteristic value is the smallest in the three weights. For example, the preset delay-type weight corresponding to the initial characteristic value of delay is X, the precision-type weight Y corresponding to the precision-type initial characteristic value, and the feedback-type weight Z corresponding to the feedback-type initial characteristic value, X > Y > Z, then when the type of the initial characteristic value a is the delay-type initial characteristic value, the weight of the initial characteristic value a is determined to be X, when the type of the initial characteristic value a is the precision-type initial characteristic value, the weight of the initial characteristic value a is determined to be Y, and when the type of the initial characteristic value a is the feedback-type initial characteristic value, the weight of the initial characteristic value a is determined to be Z.
207. And the server screens out the target characteristic value from the initial characteristic values according to the weight.
For example, the server screens target feature values from the initial feature values according to the time-delay type weight, the precise type weight and the feedback type weighting information, for example, the initial feature values extracted by characterizing the target driving data corresponding to the traffic light detection automatic driving program include identification accuracy, detection accuracy, identification performance, detection performance and pedal variation, the time-delay type weight X corresponding to the preset time-delay initial feature value, the precise type weight Y corresponding to the precise type initial feature value, and the feedback type weight Z corresponding to the feedback type initial feature value, since the time-delay type weight X is greater than the precise type weight Y and is also greater than the feedback type weight Z, the target feature values can be screened from the initial feature values according to the weight magnitude, for example, for each automatic driving program, 10 feature values can be screened from the initial feature values as the target feature values, according to the weight, more initial eigenvalues can be selected from the initial eigenvalues corresponding to the delay-type weight X as target eigenvalues, and less initial eigenvalues can be selected from the initial eigenvalues corresponding to the feedback-type weight Z as target eigenvalues, for example, 5 initial eigenvalues can be selected from the delay-type initial eigenvalues as target eigenvalues, 3 initial eigenvalues can be selected from the accurate initial eigenvalues as target eigenvalues, and 2 initial eigenvalues can be selected from the feedback-type eigenvalues as target eigenvalues.
208. And analyzing the target characteristic value by the server to obtain the comprehensive driving performance parameters of the preset automatic driving system loaded with the automatic driving program to be detected.
For example, the server may weight the target feature value according to a weighting coefficient corresponding to the automatic driving program, normalize the weighted target feature value, convert the normalized array into an N × N-dimensional input matrix, and use the N × N-dimensional input matrix as the first target feature data.
Then, the server analyzes the first target characteristic data by adopting a discrete model sub-model of the trained analysis model, for example, multi-dimensional characteristic extraction is carried out on the first target characteristic data to obtain local characteristic information corresponding to the first target characteristic data, the local characteristic information is fused to obtain fused characteristic information, and then, according to the fused characteristic information, comprehensive driving performance parameters of a preset automatic driving system loaded with the automatic driving program to be detected are calculated. For example, taking a matrix with the second target test data being 32 × 32 dimensions as an example, the whole analysis process is as shown in fig. 3, a feature extraction module composed of one or more convolution layers and a sampling layer (pooling layer) may be used to perform multi-scale feature extraction on the 32 × 32 dimensional matrix to obtain local feature information of the matrix, a full connection layer may also be used to fuse the local feature information to obtain fused feature information, and a value area of the comprehensive drivability parameter may be preset in the discrete model sub-model of the analysis model after training according to the fused feature information, for example, the value area may be 1 to 10. Analyzing the fused characteristic information, judging the comprehensive drivability parameter corresponding to the fused characteristic information in the value range of the preset comprehensive drivability parameter, outputting the comprehensive drivability parameter in the value range of the preset comprehensive drivability parameter, and using the comprehensive drivability parameter as the comprehensive drivability parameter of the preset automatic driving system loaded with the automatic driving program to be detected.
209. And when the comprehensive driving performance parameter exceeds a preset first parameter threshold value, the server determines that the automatic driving performance detection of the automatic driving program to be detected is qualified.
For example, when the comprehensive driving performance parameter exceeds a preset first parameter threshold, the server analyzes the target characteristic value by using a continuous model submodel of the trained analysis model to obtain the MPI of the preset automatic driving system loaded with the automatic driving program to be detected, and when the MPI exceeds a preset second parameter threshold, the automatic driving performance detection of the automatic driving program to be detected is determined to be qualified. Specifically, the following may be mentioned:
(1) and when the comprehensive driving performance parameter exceeds a preset parameter threshold value, the server determines that the automatic driving performance detection of the automatic driving program to be detected is qualified.
For example, the server compares the comprehensive drivability parameter of the preset automatic driving system loaded with the automatic driving program to be detected obtained by the analysis with a preset parameter threshold, and determines that the automatic drivability of the automatic driving program to be detected is qualified when the comprehensive drivability parameter exceeds the preset parameter threshold, and the automatic driving system can be deployed in an automatic driving system of an actual vehicle. And when the comprehensive driving performance parameter does not exceed the preset parameter threshold value, determining that the automatic driving performance of the automatic driving program to be detected is unqualified, the automatic driving performance of the automatic driving program to be detected cannot be directly deployed in an automatic system of an actual vehicle, and the automatic driving performance of the automatic driving program to be detected can be continuously detected after the modification or the adjustment is finished.
(2) And when the automatic driving performance of the automatic driving program to be detected is qualified, the server analyzes the target characteristic value by adopting the continuous model submodel of the trained analysis model to obtain the MPI of the preset automatic driving system loaded with the automatic driving program to be detected.
For example, when the comprehensive driving performance parameter exceeds a preset parameter threshold value and it is determined that the automatic driving performance of the automatic driving program to be detected is qualified, the target characteristic values are weighted according to the weighting coefficients corresponding to the automatic driving program, the format of the weighted target characteristic values is converted to obtain second target characteristic data, for example, the number group of the weighted characteristic values corresponding to each automatic driving program can be converted into an n × k dimensional matrix, the matrix can represent n times of sampled comprehensive test data of k automatic driving programs, and the n × k dimensional matrix is used as the second target characteristic data.
And determining the mapping relation between the second target characteristic data and the MPI according to the continuous model submodel of the trained analytic model, for example, fitting a curve by adopting a least square method based on test data acquired by a preset automatic driving system before loading the automatic driving program to be detected, and constructing the continuous model submodel according to the fitted curve. Determining a mapping relationship between the second target test data and the MPI according to the continuous model submodel, for example, the mapping relationship may be:
X*A＝Y
wherein, X is a matrix of n X k dimension, A is a coefficient vector of k X1, Y is a vector of n X1, where n is the number of actual sampling times, and when n is 1, it refers in particular to an adjustment coefficient of the automatic driving performance of the automatic driving vehicle.
And calculating the MPI of the preset automatic driving system loaded with the automatic driving program to be detected according to the mapping relation and the second target characteristic data.
As can be seen from the above, in the embodiment, the electronic device acquires the driving data set of each automatic driving program for performing simulation driving after the automatic driving program to be detected is loaded to the preset automatic driving system, identifies the target driving data in the driving data set based on the type of the automatic driving program, and performs feature extraction on the target driving data to obtain the initial feature value of the target driving data; classifying the initial characteristic values, and determining weights corresponding to the initial characteristic values based on classification results; screening out a target characteristic value from the initial characteristic values according to the weight, then analyzing the target characteristic value to obtain a comprehensive driving performance parameter of a preset automatic driving system to which the automatic driving program to be detected is to be loaded, and determining that the automatic driving performance detection of the automatic driving program to be detected is qualified when the comprehensive driving performance parameter exceeds a preset parameter threshold value; because the scheme is used for carrying out simulation driving test on the whole automatic driving system and fully considering the influence of different types of simulation data on the automatic driving performance, the processing mode of classifying and weighting the data in the simulation driving is adopted, and the test efficiency and the accuracy of the automatic driving vehicle can be greatly improved.
In order to better implement the above method, an embodiment of the present invention further provides an automatic driveability detection apparatus, which may be integrated in an electronic device, such as a server or the like.
For example, as shown in fig. 5, the text label generating apparatus may include an obtaining unit 301, an extracting unit 302, a classifying unit 303, a filtering unit 304, an analyzing unit 305, and a determining unit 306, as follows:
(1) an acquisition unit 301;
the acquiring unit 301 is configured to acquire a driving data set of each automatic driving program for performing simulated driving after the automatic driving program to be detected is loaded to a preset automatic driving system, and identify target driving data in the driving data set based on the type of the automatic driving program.
Optionally, the obtaining unit 301 may include an obtaining subunit 3011, a loading subunit 3012, a testing subunit 3013, and a first identifying subunit 3014, as shown in fig. 6, which may specifically be as follows:
an acquisition subunit 3011, configured to acquire an automatic driving program to be detected;
a loading subunit 3012, configured to load the to-be-detected autopilot to a preset autopilot system;
a test subunit 3013, configured to perform a simulation driving test on the loaded automatic driving system to obtain a driving data set of each automatic driving program;
a first identifying subunit 3014, configured to identify the target travel data from the travel data set based on the type of the automatic driving program.
For example, the obtaining subunit 3011 obtains an autopilot to be detected, the loading subunit 3012 loads the autopilot to be detected to a preset autopilot system, the testing subunit 3013 performs a simulation driving test on the loaded autopilot system to obtain a driving data set of each autopilot program, and the first identifying subunit 3014 identifies target driving data in the driving data set based on the type of the autopilot program.
(2) An extraction unit 302;
the extracting unit 302 is configured to perform feature extraction on the target driving data to obtain an initial feature value of the target driving data.
For example, the extracting unit 302 may be specifically configured to calculate, according to the collected raw data, an index value corresponding to the accuracy, the time delay, and the feedback action.
(3) A classification unit 303;
the classifying unit 303 is configured to classify the initial feature value, and determine a weight corresponding to the initial feature value based on a classification result.
Optionally, the classification unit 303 may further include a second identification subunit 3031, a classification subunit 3032, and a determination subunit 3033, as shown in fig. 7, which may specifically be as follows:
a second identifying subunit 3031, configured to identify a data type of the initial feature value;
a classification subunit 3032, configured to classify the initial feature values according to the identification result;
a determining subunit 3033, configured to determine, based on the classification result, a weight corresponding to the initial feature value.
For example, the second identifying subunit 3031 identifies the data type of the initial feature value, the classifying subunit 3032 classifies the initial feature value according to the identification result, and the determining subunit 3033 determines the weight corresponding to the initial feature value based on the classification result.
(4) A screening unit 304;
and a screening unit 304, configured to screen out a target feature value from the initial feature values according to the weight.
For example, the screening unit 304 may be specifically configured to screen the target feature value from the time-delay initial feature value, the precision initial feature value, and the feedback initial feature value according to the time-delay weight, the precision weight, and the feedback weight.
(5) An analysis unit 305;
and the analyzing unit 305 is configured to analyze the target characteristic value to obtain a comprehensive driving performance parameter of the preset autopilot system loaded with the autopilot to be detected.
Optionally, the parsing unit 305 may further include a weighting subunit 3051, a conversion subunit 3052, and a parsing subunit 3053, as illustrated in fig. 8, which may specifically be as follows:
a weighting subunit 3051, configured to weight the target feature value;
a conversion subunit 3052, configured to normalize the weighted target feature value, and perform format conversion on the normalized target feature value to obtain first target feature data;
the analysis subunit 3053 is configured to analyze the first target feature data by using the discrete model sub-model of the trained analysis model, so as to obtain the comprehensive driving performance parameter of the preset autopilot system loaded with the autopilot to be detected.
For example, the weighting subunit 3051 weights the target feature value, the converting subunit 3052 normalizes the weighted target feature value, performs format conversion on the normalized target feature value to obtain first target feature data, and the analyzing subunit 3053 analyzes the first target feature data by using the discrete model submodule of the trained analysis model to obtain the comprehensive drivability parameter of the preset autopilot system loaded with the autopilot program to be detected.
(6) A determination unit 306;
the determining unit 306 is configured to determine that the autopilot performance of the autopilot to be detected is detected to be qualified when the comprehensive drivability parameter exceeds a preset parameter threshold.
For example, the determining unit 306 may be specifically configured to determine that the detection of the automatic driving performance of the to-be-detected automatic driving program is qualified when the comprehensive driving performance parameter exceeds the preset parameter threshold, and analyze the target characteristic value by using a continuous model sub-model of the analysis model to obtain the manual intervention parameter of the preset automatic driving system loaded with the to-be-detected automatic driving program.
Optionally, the automatic driveability detection apparatus may further include an acquisition unit 307 and a training unit 308, as shown in fig. 9, specifically as follows:
the acquisition unit 307 is configured to acquire historical target feature data of a vehicle actually driven by a preset automatic driving system on the automatic driving vehicle, label corresponding actual manual intervention parameters for the historical target feature data, and use the historical target feature data labeled with the actual manual intervention parameters as a target data sample;
the training unit 308 is configured to analyze the target data sample by using a preset analysis model to obtain a predicted manual intervention parameter of the to-be-detected automatic driving program; and converging the preset analytical model according to the predicted manual intervention parameters and the marked actual manual intervention parameters to obtain the trained analytical model.
For example, the acquisition unit 307 acquires historical target feature data of a real driving of a preset automatic driving system on an automatic driving vehicle, labels corresponding actual manual intervention parameters on the historical target feature data, takes the historical target feature data labeled with the actual manual intervention parameters as a target data sample, and the training unit 308 analyzes the target data sample by using a preset analysis model to obtain predicted manual intervention parameters of the preset automatic driving system; and converging the preset analytical model according to the predicted manual intervention parameters and the marked actual manual intervention parameters to obtain the trained analytical model.
In a specific implementation, the above units may be implemented as independent entities, or may be combined arbitrarily to be implemented as the same or several entities, and the specific implementation of the above units may refer to the foregoing method embodiments, which are not described herein again.
As can be seen from the above, in the present embodiment, the obtaining unit 301 obtains the driving data set of each automatic driving program for performing simulated driving after the automatic driving program to be detected is loaded to the preset automatic driving system, based on the type of the automatic driving program, the target driving data is identified in the driving data set, and the extracting unit 302 performs feature extraction on the target driving data to obtain the initial feature value of the target driving data; then, the classification unit 303 classifies the initial characteristic values, determines a weight corresponding to the initial characteristic values based on a classification result, the screening unit 304 screens out a target characteristic value from the initial characteristic values according to the weight, the analysis unit 305 analyzes the target characteristic value to obtain a comprehensive drivability parameter of a preset automatic driving system loaded with the automatic driving program to be detected, and the determination unit 306 determines that the automatic driving performance detection of the automatic driving program to be detected is qualified when the comprehensive drivability parameter exceeds a preset parameter threshold; because the scheme is used for carrying out simulation driving test on the whole automatic driving system and fully considering the influence of different types of simulation data on the automatic driving performance, the processing mode of classifying and weighting the data in the simulation driving is adopted, and the test efficiency and the accuracy of the automatic driving vehicle can be greatly improved.
An embodiment of the present invention further provides an electronic device, as shown in fig. 10, which shows a schematic structural diagram of the electronic device according to the embodiment of the present invention, specifically:
the electronic device may include components such as a processor 401 of one or more processing cores, memory 402 of one or more computer-readable storage media, a power supply 403, and an input unit 404. Those skilled in the art will appreciate that the electronic device configuration shown in fig. 10 does not constitute a limitation of the electronic device and may include more or fewer components than those shown, or some components may be combined, or a different arrangement of components. Wherein:
the processor 401 is a control center of the electronic device, connects various parts of the whole electronic device by various interfaces and lines, performs various functions of the electronic device and processes data by running or executing software programs and/or modules stored in the memory 402 and calling data stored in the memory 402, thereby performing overall monitoring of the electronic device. Optionally, processor 401 may include one or more processing cores; preferably, the processor 401 may integrate an application processor, which mainly handles operating systems, user interfaces, application programs, etc., and a modem processor, which mainly handles wireless communications. It will be appreciated that the modem processor described above may not be integrated into the processor 401.
The memory 402 may be used to store software programs and modules, and the processor 401 executes various functional applications and data processing by operating the software programs and modules stored in the memory 402. The memory 402 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application program required by at least one function (such as a sound playing function, an image playing function, etc.), and the like; the storage data area may store data created according to use of the electronic device, and the like. Further, the memory 402 may include high speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other volatile solid state storage device. Accordingly, the memory 402 may also include a memory controller to provide the processor 401 access to the memory 402.
The electronic device further comprises a power supply 403 for supplying power to the various components, and preferably, the power supply 403 is logically connected to the processor 401 through a power management system, so that functions of managing charging, discharging, and power consumption are realized through the power management system. The power supply 403 may also include any component of one or more dc or ac power sources, recharging systems, power failure detection circuitry, power converters or inverters, power status indicators, and the like.
The electronic device may further include an input unit 404, and the input unit 404 may be used to receive input numeric or character information and generate keyboard, mouse, joystick, optical or trackball signal inputs related to user settings and function control.
Although not shown, the electronic device may further include a display unit and the like, which are not described in detail herein. Specifically, in this embodiment, the processor 401 in the electronic device loads the executable file corresponding to the process of one or more application programs into the memory 402 according to the following instructions, and the processor 401 runs the application program stored in the memory 402, thereby implementing various functions as follows:
acquiring a driving data set of each automatic driving program for carrying out simulation driving after the automatic driving program to be detected is loaded to a preset automatic driving system, identifying target driving data in the driving data set based on the type of the automatic driving program, and carrying out feature extraction on the target driving data to obtain an initial feature value of the target driving data; classifying the initial characteristic values, and determining weights corresponding to the initial characteristic values based on classification results; and screening out a target characteristic value from the initial characteristic values according to the weight, analyzing the target characteristic value to obtain a comprehensive driving performance parameter of a preset automatic driving system loaded with the automatic driving program to be detected, and determining that the automatic driving performance detection of the automatic driving program to be detected is qualified when the comprehensive driving performance parameter exceeds a preset parameter threshold value.
For example, acquiring an automatic driving program to be detected, loading the automatic driving program to be detected to a preset automatic driving system, performing simulation driving test on the loaded automatic driving system to obtain a driving data set of each automatic driving program, and identifying target driving data in the driving data set based on the type of the automatic driving program; performing feature extraction on the target driving data to obtain an initial feature value of the target driving data; the method comprises the steps of identifying the data type of an initial characteristic value, classifying the initial characteristic value corresponding to the data type into an accurate initial characteristic value when the data type is calculation accuracy data, classifying the initial characteristic value corresponding to the data type into a time delay type initial characteristic value when the data type is calculation performance data, classifying the initial characteristic value corresponding to the data type into a feedback type initial characteristic value when the data type is feedback control data, determining the weight corresponding to the initial characteristic value according to the preset time delay type initial characteristic value, the accurate initial characteristic value and the weight corresponding to the feedback type initial characteristic value, and screening out a target characteristic value from the time delay type initial characteristic value, the accurate initial characteristic value and the feedback type initial characteristic value respectively according to the time delay type weight, the accurate initial characteristic value and the feedback type weight. Analyzing the target characteristic value by adopting a discrete model sub-model of the analyzed model after training to obtain the comprehensive driving performance parameter of the preset automatic driving system loaded with the automatic driving program to be detected, and determining that the automatic driving performance detection of the automatic driving program to be detected is qualified when the comprehensive driving performance parameter exceeds a preset parameter threshold value. And when the comprehensive driving performance parameter does not exceed the preset parameter threshold value, determining that the automatic driving performance detection of the automatic driving program to be detected is unqualified. And when the automatic driving performance of the automatic driving program to be detected is detected to be qualified, analyzing the target characteristic value by adopting a continuous model submodel of the trained analysis model to obtain the manual intervention parameters of the preset automatic driving system loaded with the automatic driving program to be detected.
The above operations can be implemented in the foregoing embodiments, and are not described in detail herein.
As can be seen from the above, in the embodiment of the present invention, after the driving data set of each automatic driving program for performing simulation driving after the automatic driving program to be detected is loaded to the preset automatic driving system is acquired, the target driving data is identified in the driving data set based on the type of the automatic driving program, and the target driving data is subjected to feature extraction to obtain the initial feature value of the target driving data; classifying the initial characteristic values, and determining weights corresponding to the initial characteristic values based on classification results; screening out a target characteristic value from the initial characteristic values according to the weight, then analyzing the target characteristic value to obtain a comprehensive drivability parameter of a preset automatic driving system loaded with the automatic driving program to be detected, and determining that the automatic driving performance detection of the automatic driving program to be detected is qualified when the comprehensive drivability parameter exceeds a preset parameter threshold value; because the scheme is used for carrying out simulation driving test on the whole automatic driving system and fully considering the influence of different types of simulation data on the automatic driving performance, the processing mode of classifying and weighting the data in the simulation driving is adopted, and the test efficiency and the accuracy of the automatic driving vehicle can be greatly improved.
It will be understood by those skilled in the art that all or part of the steps of the methods of the above embodiments may be performed by instructions or by associated hardware controlled by the instructions, which may be stored in a computer readable storage medium and loaded and executed by a processor.
To this end, embodiments of the present invention provide a computer-readable storage medium, in which a plurality of instructions are stored, where the instructions can be loaded by a processor to execute steps in any one of the automatic drivability detection methods provided by the embodiments of the present invention. For example, the instructions may perform the steps of:
acquiring a driving data set of each automatic driving program for carrying out simulation driving after the automatic driving program to be detected is loaded to a preset automatic driving system, identifying target driving data in the driving data set based on the type of the automatic driving program, and carrying out feature extraction on the target driving data to obtain an initial feature value of the target driving data; classifying the initial characteristic values, and determining weights corresponding to the initial characteristic values based on classification results; and screening out a target characteristic value from the initial characteristic values according to the weight, analyzing the target characteristic value to obtain a comprehensive driving performance parameter of a preset automatic driving system loaded with the automatic driving program to be detected, and determining that the automatic driving performance detection of the automatic driving program to be detected is qualified when the comprehensive driving performance parameter exceeds a preset parameter threshold value.
For example, acquiring an automatic driving program to be detected, loading the automatic driving program to be detected to a preset automatic driving system, performing simulation driving test on the loaded automatic driving system to obtain a driving data set of each automatic driving program, and identifying target driving data in the driving data set based on the type of the automatic driving program; performing feature extraction on the target driving data to obtain an initial feature value of the target driving data; the method comprises the steps of identifying the data type of an initial characteristic value, classifying the initial characteristic value corresponding to the data type into an accurate initial characteristic value when the data type is calculation accuracy data, classifying the initial characteristic value corresponding to the data type into a time delay type initial characteristic value when the data type is calculation performance data, classifying the initial characteristic value corresponding to the data type into a feedback type initial characteristic value when the data type is feedback control data, determining the weight corresponding to the initial characteristic value according to the preset time delay type initial characteristic value, the accurate initial characteristic value and the weight corresponding to the feedback type initial characteristic value, and screening out a target characteristic value from the time delay type initial characteristic value, the accurate initial characteristic value and the feedback type initial characteristic value respectively according to the time delay type weight, the accurate initial characteristic value and the feedback type weight. And analyzing the target characteristic value by adopting a discrete model sub-model of the trained analysis model to obtain the comprehensive driving performance parameters of the preset automatic driving system loaded with the automatic driving program to be detected. And when the comprehensive driving performance parameter exceeds a preset first parameter threshold value, determining that the automatic driving performance detection of the automatic driving program to be detected is qualified. And when the comprehensive driving performance parameter does not exceed the preset parameter threshold value, determining that the automatic driving performance detection of the automatic driving program to be detected is unqualified. And when the automatic driving performance of the automatic driving program to be detected is detected to be qualified, analyzing the target characteristic value by adopting a continuous model submodel of the trained analysis model to obtain the manual intervention parameters of the preset automatic driving system loaded with the automatic driving program to be detected.
The above operations can be implemented in the foregoing embodiments, and are not described in detail herein.
Wherein the computer-readable storage medium may include: read Only Memory (ROM), Random Access Memory (RAM), magnetic or optical disks, and the like.
Since the instructions stored in the computer-readable storage medium can execute the steps in any of the automatic driving performance detection methods provided by the embodiments of the present invention, the beneficial effects that can be achieved by any of the automatic driving performance detection methods provided by the embodiments of the present invention can be achieved, which are detailed in the foregoing embodiments and will not be described again here.
The method, the device and the computer-readable storage medium for detecting the automatic driving performance provided by the embodiment of the invention are described in detail, a specific example is applied in the text to explain the principle and the implementation of the invention, and the description of the embodiment is only used for helping to understand the method and the core idea of the invention; meanwhile, for those skilled in the art, according to the idea of the present invention, there may be variations in the specific embodiments and the application scope, and in summary, the content of the present specification should not be construed as a limitation to the present invention.

Claims (10)

1. An automatic drivability detection method characterized by comprising:
acquiring a driving data set of each automatic driving program for simulation driving after the automatic driving program to be detected is loaded to a preset automatic driving system, and identifying target driving data in the driving data set based on the type of the automatic driving program;
performing feature extraction on the target driving data to obtain an initial feature value of the target driving data;
classifying the initial characteristic values, and determining weights corresponding to the initial characteristic values based on classification results;
screening out a target characteristic value from the initial characteristic values according to the weight;
analyzing the target characteristic value to obtain comprehensive driving performance parameters of a preset automatic driving system loaded with the automatic driving program to be detected;
and when the comprehensive driving performance parameter exceeds a preset parameter threshold value, determining that the automatic driving performance of the automatic driving program to be detected is qualified.
2. The automatic drivability detection method according to claim 1, wherein the classifying the initial feature value and determining the weight corresponding to the initial feature value based on the classification result includes:
identifying a data type of the initial characteristic value;
classifying the initial characteristic values according to the identification result;
and determining the weight corresponding to the initial characteristic value based on the classification result.
3. The automatic drivability detection method according to claim 2, wherein the classifying the initial feature value according to the recognition result includes:
when the data type is calculation accuracy data, classifying the initial characteristic value corresponding to the data type into an accurate initial characteristic value, wherein the calculation accuracy data is used for indicating the calculation accuracy of the target driving data;
when the data type is calculation performance data, classifying the initial characteristic value corresponding to the data type into a time delay type initial characteristic value, wherein the calculation performance data is used for indicating the calculation speed of the target driving data;
and when the data type is feedback control data, classifying the initial characteristic value corresponding to the data type into a feedback type initial characteristic value, wherein the feedback control data is data corresponding to the feedback control action executed in the target driving data.
4. The automatic drivability detection method according to claim 3, wherein the determining the weight corresponding to the initial feature value based on the classification result includes:
determining the weight corresponding to the initial characteristic value according to a preset time delay type initial characteristic value, a precise type initial characteristic value and the weight corresponding to the feedback type initial characteristic value, wherein the time delay type weight corresponding to the time delay type initial characteristic value is the largest in the three weights, the precise type weight corresponding to the precise type initial characteristic value is the second in the three weights, and the feedback type weight corresponding to the feedback type initial characteristic value is the smallest in the three weights.
5. The automatic driveability detection method according to claim 1, wherein the analyzing the target characteristic value to obtain the comprehensive driveability parameter of the preset automatic driveability system loaded with the automatic driveability program to be detected includes:
weighting the target characteristic value;
normalizing the weighted target characteristic value, and performing format conversion on the normalized target characteristic value to obtain first target characteristic data;
and analyzing the first target characteristic data by adopting a discrete model sub-model of the trained analysis model to obtain the comprehensive driving performance parameters of the preset automatic driving system loaded with the automatic driving program to be detected.
6. The automatic driveability detection method of claim 5, wherein the analyzing the first target characteristic data by using the discrete model sub-model of the trained analysis model to obtain the comprehensive driveability parameter of the preset automatic driveability system loaded with the automatic driveability program to be detected comprises:
performing multi-scale feature extraction on the first target feature to obtain local feature information of the first target feature data;
fusing the local information to obtain fused characteristic information;
and calculating the comprehensive driving performance parameters of the preset automatic driving system loaded with the automatic driving program to be detected according to the fused characteristic information.
7. The automatic driveability detection method according to claim 5, wherein after determining that the automatic driveability detection of the to-be-detected automatic driving program is qualified when the comprehensive driveability parameter exceeds a preset parameter threshold, the method further comprises:
weighting the target characteristic value, and converting the data format of the weighted target characteristic value to obtain second target characteristic data;
acquiring a mapping relation between the second target characteristic data and the manual intervention parameters according to the continuous model submodel of the trained analytical model;
and determining the manual intervention parameters corresponding to the second target characteristic data according to the mapping relation.
8. The automatic driveability detection method of claim 7, wherein before analyzing the first target feature data using the discrete model sub-model of the trained analytical model, the method further comprises:
acquiring historical target characteristic data of real driving of the preset automatic driving system on the automatic driving vehicle;
marking corresponding actual manual intervention parameters on the historical target characteristic data, and taking the historical target characteristic data marked with the actual manual intervention parameters as a target data sample;
analyzing the target data sample by adopting a preset analysis model to obtain a predicted manual intervention parameter of the preset automatic driving system;
and converging the preset analytical model according to the predicted manual intervention parameter and the actual manual intervention parameter to obtain a trained analytical model.
9. The autopilot performance detection method of claim 1, wherein the obtaining of a travel data set for each autopilot that performs a simulated travel after the autopilot to be detected is loaded into a preset autopilot system, and the identifying of target travel data in the travel data set based on a type of the autopilot comprises:
acquiring the automatic driving program to be detected;
loading the automatic driving program to be detected to the preset automatic driving system;
carrying out simulation driving test on the loaded automatic driving system to obtain a driving data set of each automatic driving program;
target travel data is identified in the set of travel data based on the type of the autonomous driving program.
10. An automatic drivability detection apparatus characterized by comprising:
the system comprises an acquisition unit, a simulation unit and a processing unit, wherein the acquisition unit is used for acquiring a driving data set of each automatic driving program for simulation driving after the automatic driving program to be detected is loaded to a preset automatic driving system, and identifying target driving data in the driving data set based on the type of the automatic driving program;
the extraction unit is used for carrying out feature extraction on the target driving data to obtain an initial feature value of the target driving data;
the classification unit is used for classifying the initial characteristic values and determining the weight corresponding to the initial characteristic values based on the classification result;
the screening unit is used for screening a target characteristic value from the initial characteristic values according to the weight;
the analysis unit is used for analyzing the target characteristic value and loading the comprehensive driving performance parameters of the preset automatic driving system of the automatic driving program to be detected;
and the determining unit is used for determining that the automatic driving performance detection of the automatic driving program to be detected is qualified when the comprehensive driving performance parameter exceeds a preset parameter threshold value.
CN202010140452.5A 2020-03-03 2020-03-03 Automatic driving performance detection method and device Active CN111290370B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202010140452.5A CN111290370B (en) 2020-03-03 2020-03-03 Automatic driving performance detection method and device

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202010140452.5A CN111290370B (en) 2020-03-03 2020-03-03 Automatic driving performance detection method and device

Publications (2)

Publication Number Publication Date
CN111290370A CN111290370A (en) 2020-06-16
CN111290370B true CN111290370B (en) 2021-07-23

Family

ID=71028564

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202010140452.5A Active CN111290370B (en) 2020-03-03 2020-03-03 Automatic driving performance detection method and device

Country Status (1)

Country Link
CN (1) CN111290370B (en)

Families Citing this family (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112559371B (en) * 2020-12-24 2023-07-28 北京百度网讯科技有限公司 Automatic driving test method and device and electronic equipment
CN112817873A (en) * 2021-03-02 2021-05-18 上海初斟智能科技有限公司 Millimeter wave radar alarm automatic testing method
CN113076237B (en) * 2021-03-05 2022-09-20 山东英信计算机技术有限公司 Memory performance testing method and system and computer readable storage medium
CN112987596B (en) * 2021-05-10 2021-09-07 北京三快在线科技有限公司 Test method and test device for unmanned equipment
CN116863430B (en) * 2023-07-31 2023-12-22 合肥海普微电子有限公司 Point cloud fusion method for automatic driving

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108647437A (en) * 2018-05-09 2018-10-12 公安部交通管理科学研究所 A kind of autonomous driving vehicle evaluation method and evaluation system
CN109358526A (en) * 2018-10-15 2019-02-19 清华大学苏州汽车研究院（吴江） The software of automatic Pilot tracking algorithm based on emulation is in ring test method
CN109435955A (en) * 2018-10-22 2019-03-08 百度在线网络技术（北京）有限公司 A kind of automated driving system performance estimating method, device, equipment and storage medium
CN109520744A (en) * 2018-11-12 2019-03-26 百度在线网络技术（北京）有限公司 The driving performance test method and device of automatic driving vehicle
CN109884916A (en) * 2019-02-26 2019-06-14 初速度（苏州）科技有限公司 A kind of automatic Pilot Simulation Evaluation method and device
CN109992884A (en) * 2019-04-01 2019-07-09 清华大学 Automatic driving vehicle Function Appraising method based on worst scene search
CN110160804A (en) * 2019-05-31 2019-08-23 中国科学院深圳先进技术研究院 A kind of test method of automatic driving vehicle, apparatus and system

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
DE102016220913A1 (en) * 2015-11-06 2017-05-11 Ford Global Technologies, Llc Method and device for generating test cases for autonomous vehicles
US10346564B2 (en) * 2016-03-30 2019-07-09 Toyota Jidosha Kabushiki Kaisha Dynamic virtual object generation for testing autonomous vehicles in simulated driving scenarios
DE102017213634A1 (en) * 2017-08-07 2019-02-07 Ford Global Technologies, Llc Method and apparatus for performing virtual tests in a virtual reality environment for an autonomous vehicle

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108647437A (en) * 2018-05-09 2018-10-12 公安部交通管理科学研究所 A kind of autonomous driving vehicle evaluation method and evaluation system
CN109358526A (en) * 2018-10-15 2019-02-19 清华大学苏州汽车研究院（吴江） The software of automatic Pilot tracking algorithm based on emulation is in ring test method
CN109435955A (en) * 2018-10-22 2019-03-08 百度在线网络技术（北京）有限公司 A kind of automated driving system performance estimating method, device, equipment and storage medium
CN109520744A (en) * 2018-11-12 2019-03-26 百度在线网络技术（北京）有限公司 The driving performance test method and device of automatic driving vehicle
CN109884916A (en) * 2019-02-26 2019-06-14 初速度（苏州）科技有限公司 A kind of automatic Pilot Simulation Evaluation method and device
CN109992884A (en) * 2019-04-01 2019-07-09 清华大学 Automatic driving vehicle Function Appraising method based on worst scene search
CN110160804A (en) * 2019-05-31 2019-08-23 中国科学院深圳先进技术研究院 A kind of test method of automatic driving vehicle, apparatus and system

Also Published As

Publication number Publication date
CN111290370A (en) 2020-06-16

Similar Documents

Publication Publication Date Title
CN111290370B (en) Automatic driving performance detection method and device
CN102076531B (en) Vehicle clear path detection
GB2573738A (en) Driving monitoring
CN113155173B (en) Perception performance evaluation method and device, electronic device and storage medium
CN111507370A (en) Method and device for obtaining sample image of inspection label in automatic labeling image
CN113343461A (en) Simulation method and device for automatic driving vehicle, electronic equipment and storage medium
CN111274886B (en) Deep learning-based pedestrian red light running illegal behavior analysis method and system
CN112699793A (en) Fatigue driving detection optimization identification method based on random forest
CN117173913B (en) Traffic control method and system based on traffic flow analysis at different time periods
CN114566052B (en) Method for judging rotation of highway traffic flow monitoring equipment based on traffic flow direction
CN110796360A (en) Fixed traffic detection source multi-scale data fusion method
CN114358401A (en) Road icing identification method fusing image and meteorological environment data
CN113406955A (en) Complex network-based automatic driving automobile complex environment model, cognitive system and cognitive method
CN115376101A (en) Incremental learning method and system for automatic driving environment perception
CN115470118A (en) Test analysis method, device, medium and equipment based on environment simulation scene
CN110909656B (en) Pedestrian detection method and system integrating radar and camera
CN115797403A (en) Traffic accident prediction method and device, storage medium and electronic device
CN111666848A (en) Method, device and equipment for detecting arrival of transport vehicle and storage medium
CN114863170A (en) Deep learning-based new energy vehicle battery spontaneous combustion early warning method and device
CN116872961B (en) Control system for intelligent driving vehicle
Fanthony et al. Yolo algorithm-based surrounding object identification on autonomous electric vehicle
CN116881783A (en) Road damage detection method, device, computer equipment and storage medium
CN113899948B (en) System and method for quickly extracting electromagnetic spectrum target characteristic data
CN113204246A (en) Unmanned aerial vehicle running state detection method
US20210248464A1 (en) Computer-implemented method and device for machine learning

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
REG Reference to a national code

Ref country code: HK

Ref legal event code: DE

Ref document number: 40024876

Country of ref document: HK

SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant