CN111858927A - Data testing method and device, electronic equipment and storage medium - Google Patents
Data testing method and device, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN111858927A CN111858927A CN202010518935.4A CN202010518935A CN111858927A CN 111858927 A CN111858927 A CN 111858927A CN 202010518935 A CN202010518935 A CN 202010518935A CN 111858927 A CN111858927 A CN 111858927A
- Authority
- CN
- China
- Prior art keywords
- data
- scene
- scene classification
- piece
- value
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000012360 testing method Methods 0.000 title claims abstract description 95
- 238000011156 evaluation Methods 0.000 claims abstract description 43
- 238000000034 method Methods 0.000 claims abstract description 41
- 238000012545 processing Methods 0.000 claims abstract description 23
- 238000013507 mapping Methods 0.000 claims abstract description 22
- 238000010586 diagram Methods 0.000 claims description 42
- 230000015654 memory Effects 0.000 claims description 19
- 238000007781 pre-processing Methods 0.000 claims description 9
- 239000000126 substance Substances 0.000 claims description 2
- 239000000463 material Substances 0.000 abstract description 4
- 230000008859 change Effects 0.000 description 7
- 230000008569 process Effects 0.000 description 7
- 238000004891 communication Methods 0.000 description 4
- 238000004590 computer program Methods 0.000 description 4
- 230000006870 function Effects 0.000 description 4
- 238000012986 modification Methods 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- 230000000007 visual effect Effects 0.000 description 3
- 230000008901 benefit Effects 0.000 description 2
- 238000013100 final test Methods 0.000 description 2
- 230000003993 interaction Effects 0.000 description 2
- 239000004973 liquid crystal related substance Substances 0.000 description 2
- 238000005065 mining Methods 0.000 description 2
- 238000003909 pattern recognition Methods 0.000 description 2
- 230000006399 behavior Effects 0.000 description 1
- 230000001413 cellular effect Effects 0.000 description 1
- 238000004883 computer application Methods 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000002474 experimental method Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000010295 mobile communication Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000001953 sensory effect Effects 0.000 description 1
- 238000004088 simulation Methods 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 238000000638 solvent extraction Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/35—Clustering; Classification
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/36—Creation of semantic tools, e.g. ontology or thesauri
- G06F16/367—Ontology
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Life Sciences & Earth Sciences (AREA)
- Databases & Information Systems (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Evolutionary Computation (AREA)
- Evolutionary Biology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Bioinformatics & Computational Biology (AREA)
- Artificial Intelligence (AREA)
- Animal Behavior & Ethology (AREA)
- Computational Linguistics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The application discloses a data testing method, a data testing device, electronic equipment and a storage medium, and relates to the field of automatic driving, wherein the method comprises the following steps: acquiring a first data set, operating each piece of data in the first data set, and respectively judging each piece of data according to an operation result to obtain a judgment result of whether each piece of data passes or not; respectively determining the scene classification of each piece of data in the first data set; and aiming at each scene classification, mapping each piece of data in the scene classification to a pre-generated statistical graph corresponding to the scene classification, and generating a test evaluation index corresponding to the scene classification according to a mapping result and a judgment result of each piece of data in the scene classification. By the scheme, manpower and material resources can be saved, and the processing efficiency is improved.
Description
Technical Field
The present application relates to computer application technologies, and in particular, to a data testing method and apparatus, an electronic device, and a storage medium in the field of automatic driving.
Background
In the unmanned vehicle test, except for a common road test, a virtualization test based on a simulation platform is required, so that the performance of the unmanned vehicle can be effectively verified with lower cost, higher efficiency and more comprehensive coverage.
At present, a playback type random test mode based on mass data is usually adopted, namely, a data playback mode is adopted, experiments are performed randomly, and a test result is obtained through manual analysis and the like.
Disclosure of Invention
The application provides a data testing method and device, electronic equipment and a storage medium.
A method of data testing, comprising:
acquiring a first data set, operating each piece of data in the first data set, and respectively judging each piece of data according to an operation result to obtain a judgment result of whether each piece of data passes or not;
respectively determining the scene classification of each piece of data in the first data set;
and aiming at each scene classification, respectively mapping each piece of data in the scene classification to a pre-generated statistical graph corresponding to the scene classification, and generating a test evaluation index corresponding to the scene classification according to a mapping result and a judgment result of each piece of data in the scene classification.
A data testing device, comprising: the device comprises a first test module and a second test module;
the first test module is used for acquiring a first data set, operating each piece of data in the first data set, and respectively judging each piece of data according to an operation result to obtain a judgment result of whether each piece of data passes or not;
The second testing module is configured to determine a scene classification to which each piece of data in the first data set belongs, map each piece of data in the scene classification to a pre-generated statistical graph corresponding to the scene classification, and generate a testing evaluation index corresponding to the scene classification according to a mapping result and a determination result of each piece of data in the scene classification.
An electronic device, comprising:
at least one processor; and
a memory communicatively coupled to the at least one processor; wherein the content of the first and second substances,
the memory stores instructions executable by the at least one processor to enable the at least one processor to perform a method as described above.
A non-transitory computer readable storage medium having stored thereon computer instructions for causing the computer to perform the method as described above.
One embodiment in the above application has the following advantages or benefits: the rapid automatic evaluation of the data test result can be realized without manual analysis and other processing, and the quantifiable test evaluation index is provided, so that the test result can be quantitatively described through the specific test evaluation index, the manpower and material resources are saved, the processing efficiency is improved, and the like. It should be understood that the statements in this section do not necessarily identify key or critical features of the embodiments of the present disclosure, nor do they limit the scope of the present disclosure. Other features of the present disclosure will become apparent from the following description.
Drawings
The drawings are included to provide a better understanding of the present solution and are not intended to limit the present application. Wherein:
FIG. 1 is a flow chart of an embodiment of a data testing method described herein;
FIG. 2 is a schematic diagram illustrating an overall implementation process of the data testing method according to the present application;
FIG. 3 is a schematic diagram of a component structure of an embodiment 30 of the data testing apparatus of the present application;
fig. 4 is a block diagram of an electronic device according to the method of an embodiment of the present application.
Detailed Description
The following description of the exemplary embodiments of the present application, taken in conjunction with the accompanying drawings, includes various details of the embodiments of the application for the understanding of the same, which are to be considered exemplary only. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the present application. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.
In addition, it should be understood that the term "and/or" herein is merely one type of association relationship that describes an associated object, meaning that three relationships may exist, e.g., a and/or B may mean: a exists alone, A and B exist simultaneously, and B exists alone. In addition, the character "/" herein generally indicates that the former and latter related objects are in an "or" relationship.
The application provides a data testing method, aiming at a first data set to be processed, various data can be operated, and various data can be judged according to operation results, so that judgment results of whether various data pass or not can be obtained, scene classification of various data can be determined, and further, rapid automatic evaluation and the like of data testing results can be realized according to the obtained judgment results and the scene classification results based on statistical graphs corresponding to different pre-generated scene classifications.
First, a method of generating a statistical map corresponding to different scene classifications will be described below.
And acquiring a second data set, wherein the second data set can comprise mass data accumulated in the ways of road testing, field acquisition, artificial creation and the like.
The scene classification to which each piece of data in the second data set belongs can be determined separately. For example, the data in the second data set may be divided into the corresponding scene classifications by using methods such as scene pattern recognition and category mining. The specific types of the scenes can be determined according to actual needs, such as left-turn scenes, lane-changing scenes, and the like. For a piece of data, it may belong to only one scene classification, or may belong to multiple scene classifications, and has no influence on subsequent processing. Through scene classification, effective splitting of data is achieved, and therefore processing efficiency of mass data is improved.
The specific form of the statistical map may be determined according to actual needs, such as a thermodynamic diagram or a scatter diagram with a certain transparency and capable of being accumulated, and the thermodynamic diagram is described as an example below.
And aiming at each scene classification, generating a thermodynamic diagram corresponding to the scene classification according to each piece of data in the scene classification. Preferably, for any scene classification, a first scene parameter and a second scene parameter of interest may be respectively selected from scene parameters corresponding to the scene classification, a value interval of the first scene parameter is averagely divided into M continuous sub-intervals, a value interval of the second scene parameter is averagely divided into N continuous sub-intervals, M and N are positive integers greater than one, and an interval combination to which each piece of data belongs is respectively determined according to a value of the first scene parameter and a value of the second scene parameter of each piece of data in the scene classification, where the interval combination is composed of one sub-interval corresponding to the first scene parameter and one sub-interval corresponding to the second scene parameter, and any two interval combinations are different, and further, a thermodynamic diagram corresponding to the scene classification may be generated based on the determination result.
The scene parameters for different scene categories may also be different. For any scene classification, a first scene parameter and a second scene parameter of interest may be selected from the scene parameters corresponding to the scene classification, respectively, that is, all the scene parameters may not be used, but some key/typical scene parameters, usually two, may be selected, corresponding to the horizontal axis and the vertical axis of the thermodynamic diagram, respectively. For example, for a lane change scene, two scene parameters, namely a lane change time and a lane change speed, may be selected, assuming that the duration of a piece of data is 2 minutes, where a lane change is performed from 60 seconds to 1 minute and 15 seconds, and then the lane change duration is 15 seconds, and the lane change speed may refer to a speed average value in the lane change process, or the like.
For the selected first scene parameter and the second scene parameter, the value interval of the first scene parameter can be averagely divided into M continuous subintervals, and the value interval of the second scene parameter can be averagely divided into N continuous subintervals, wherein M and N are positive integers greater than one, and the specific value can be determined according to actual needs.
By combining the subintervals, multiple combinations of intervals can be obtained. For example, the values of M and N are both 100, that is, 100 subintervals corresponding to the first scene parameter, respectively, subintervals 1 to 100, and 100 subintervals corresponding to the second scene parameter, respectively, subintervals 101 to 200, may be obtained, so that subinterval 1 and subinterval 101 may be combined to obtain an interval combination, subinterval 1 and subinterval 102 may be combined to obtain an interval combination, subinterval 2 and subinterval 101 may also be combined to obtain an interval combination, and the like. For any scene classification, the interval combination to which each piece of data belongs can be respectively determined according to the value of the first scene parameter and the value of the second scene parameter of each piece of data in the scene classification, and then the thermodynamic diagram corresponding to the scene classification can be generated based on the determination result. Preferably, for each interval combination, a ratio of the number of data pieces belonging to the interval combination to the number of data pieces in the scene classification may be determined, the first scene parameter and the second scene parameter are taken as a horizontal axis and a vertical axis, respectively, a thermodynamic diagram is drawn according to the ratio corresponding to each interval combination, and how to draw the thermodynamic diagram is the prior art.
Through the thermodynamic diagram mode, data under different scene classifications can be visually displayed, and visual understanding, analysis and the like are facilitated.
After generating the thermodynamic diagram, the division of the thermodynamic zones can also be performed, that is, for any scene classification, the following processes can be respectively performed for at least one set thermodynamic quantile value: and dividing the thermodynamic diagrams corresponding to the scene classification according to the thermodynamic quantile values to obtain a core thermodynamic area and a non-core thermodynamic area under the thermodynamic quantile values.
For example, the thermal place-dividing value is 90 places, the combinations of the intervals may be sorted in the order from the largest number of the data pieces belonging to the combinations of the intervals to the smallest number of the data pieces belonging to the combinations of the intervals, then the ratio of the combination of the interval in the first place after sorting to the combination of the interval in the second place after sorting (i.e. the ratio of the number of the data pieces belonging to the combination of the interval to the number of the data pieces in the scene classification) may be added, and the added value is compared with 90%, if the added value is smaller than 90%, the ratio of the combination of the interval in the next place after sorting may be added to the value, and the added value may be compared with 90% again, and so on, if the added value is equal to 90%, the processing may be stopped, the area corresponding to the combination of the interval corresponding to the newest added ratio and the combination of the interval before the newest added value may be used as the core thermal area, if the added value is larger than 90%, then the area corresponding to the interval combination before the interval combination corresponding to the latest added ratio can be used as the core thermal area, and the area outside the core thermal area is the non-core thermal area. In the same way, the core thermal area and the non-core thermal area under the thermal quantiles of 60 quantiles, 70 quantiles, 80 quantiles and the like can be obtained respectively.
The number and value of the thermodynamic quantile values can be determined according to actual needs. And purposeful detailed evaluation and the like of the test result can be realized through flexible setting of the thermal grading value.
After the above processing is completed, data testing for mass data can be performed. The mass data required to be subjected to the data test is included in the first data set.
FIG. 1 is a flow chart of an embodiment of a data testing method according to the present application. As shown in fig. 1, the following detailed implementation is included.
In 101, a first data set is obtained, each piece of data in the first data set is operated, and each piece of data is judged according to an operation result, so that a judgment result of whether each piece of data passes or not is obtained.
At 102, a scene classification to which each piece of data in the first data set belongs is determined.
In 103, for each scene classification, each piece of data in the scene classification is mapped to a pre-generated statistical map corresponding to the scene classification, and a test evaluation index corresponding to the scene classification is generated according to the mapping result and the judgment result of each piece of data in the scene classification.
For the first data set, each piece of data in the first data set can be run firstly, preferably in a distributed cloud computing platform, the speed can be increased through a parallelization mode, and how to run the data is the prior art.
In addition, each piece of data can be judged according to the operation result, so that a judgment result of whether each piece of data passes is obtained, for example, whether the behavior of the unmanned vehicle in each piece of data is reasonable and legal or meets the expectation can be judged according to the set rule, and a judgment result of whether the unmanned vehicle passes is given according to the set rule.
The scene classification to which each piece of data in the first data set belongs can also be determined separately. In practical application, the scene classification of each piece of data can be completed in advance, namely, the pre-classification is performed, or the scene classification can be performed when the data needs to be used, and the scene classification mode is the same as the scene classification mode when the thermodynamic diagram is generated.
For each scene classification, each piece of data in the scene classification can be mapped into the thermodynamic diagram corresponding to the scene classification. Preferably, for each piece of data in the scene classification, the data may be mapped to a corresponding position in the thermodynamic diagram corresponding to the scene classification according to a value of a first scene parameter and a value of a second scene parameter of the data, respectively.
And then, generating a test evaluation index corresponding to the scene classification according to the mapping result and the judgment result of each piece of data in the scene classification. Preferably, for any scene classification, the following processing can be respectively carried out for each thermal quantile value in the set at least one thermal quantile value: counting the number of data pieces in the core thermal area under the thermal quantile value in each piece of data in the scene classification after mapping, generating a test evaluation index corresponding to the scene classification under the thermal quantile value according to the statistical result and the judgment result of each piece of data in the scene classification, preferably, calculating the ratio of the number of data pieces passing through the core thermal area and the number of data pieces in the core thermal area according to the judgment result, obtaining the passing rate of the core thermal area under the thermal quantile value, and taking the passing rate of the core thermal area as the test evaluation index corresponding to the scene classification under the thermal quantile value.
For example, if the number of data pieces passing through the core thermal area is 480 and the number of data pieces passing through the core thermal area is 500, the passing rate of the core thermal area is 480/500.
Further, for any scene classification, for each thermal quantile value in the set at least one thermal quantile value, the following processing can be further performed respectively: counting the number of data pieces in the non-core thermal area under the thermal quantile value in each piece of data in the scene classification after mapping, calculating the ratio of the number of data pieces which are in the non-core thermal area and pass the judgment result to the number of data pieces in the non-core thermal area to obtain the passing rate of the non-core thermal area under the thermal quantile value, and taking the passing rate of the non-core thermal area as a test evaluation index corresponding to the scene classification under the thermal quantile value, namely taking both the passing rate of the core thermal area and the passing rate of the non-core thermal area as the test evaluation index corresponding to the scene classification under the thermal quantile value, thereby further enriching the test evaluation indexes, providing more valuable evaluation information for users and the like.
If the set thermal quantile value is one and 90 quantiles, test evaluation indexes corresponding to different scene classifications under the thermal quantile value can be obtained. Assuming that the set thermal quantiles are three, namely 70 quantiles, 80 quantiles and 90 quantiles, test evaluation indexes corresponding to different scene classifications under the thermal quantile of 70 quantiles, test evaluation indexes corresponding to different scene classifications under the thermal quantile of 80 quantiles and test evaluation indexes corresponding to different scene classifications under the thermal quantile of 90 quantiles can be obtained respectively.
In addition, test evaluation indexes corresponding to different scene classifications can be integrated, so that a final test result is obtained.
Based on the above description, it can be seen that, in the method of the embodiment, manual analysis and other processing are not required, rapid and automatic evaluation of data test results can be achieved, quantifiable test evaluation indexes are provided, and the test results can be quantitatively described through specific test evaluation indexes, so that manpower and material resources are saved, the processing efficiency is improved, and the passing rates among different unmanned vehicle systems and the like have quantifiable and comparable results and the like under the same thermodynamic diagram and mass data test scenes.
Fig. 2 is a schematic diagram of an overall implementation process of the data testing method according to the present application. As shown in fig. 2, the method of the present application mainly includes two links of thermodynamic diagram generation and mass data testing, and for concrete implementation of each link, reference is made to the foregoing related description, which is not repeated. In addition, a related thermodynamic diagram including the judgment result can be generated, namely after each piece of data is mapped on the thermodynamic diagram, the corresponding judgment result can be marked on the thermodynamic diagram, so that the related thermodynamic diagram is formed, and the manual understanding is facilitated.
It should be noted that the foregoing method embodiments are described as a series of acts or combinations for simplicity in explanation, but it should be understood by those skilled in the art that the present application is not limited by the order of acts described, as some steps may occur in other orders or concurrently depending on the application. Further, those skilled in the art should also appreciate that the embodiments described in the specification are preferred embodiments and that the acts and modules referred to are not necessarily required in this application.
The above is a description of method embodiments, and the embodiments of the present application are further described below by way of apparatus embodiments.
Fig. 3 is a schematic structural diagram of a data testing apparatus 30 according to an embodiment of the present disclosure. As shown in fig. 3, includes: a first test module 301 and a second test module 302.
The first testing module 301 is configured to obtain a first data set, run each piece of data in the first data set, and respectively determine each piece of data according to a running result to obtain a determination result of whether each piece of data passes through.
The second testing module 302 is configured to determine a scene classification to which each piece of data in the first data set belongs, map each piece of data in the scene classification to a pre-generated statistical graph corresponding to the scene classification, and generate a testing evaluation index corresponding to the scene classification according to a mapping result and a determination result of each piece of data in the scene classification.
It can be seen that, in order to implement the scheme described in this embodiment, statistical graphs corresponding to different scene classifications need to be generated in advance. To this end, the apparatus shown in fig. 3 may further include: the preprocessing module 300 is configured to obtain a second data set, determine a scene classification to which each piece of data in the second data set belongs, and generate a statistical graph corresponding to each scene classification according to each piece of data in the scene classification.
The second data set may include mass data accumulated by road testing, field collection, artificial creation, and the like. The data can be divided into the corresponding scene classification by adopting the modes of scene pattern recognition, category mining and the like.
For each scene classification, the preprocessing module 300 may generate a statistical graph, such as a thermodynamic graph, corresponding to the scene classification according to each piece of data in the scene classification. Preferably, for any scene classification, the preprocessing module 300 may respectively select a first scene parameter and a second scene parameter of interest from scene parameters corresponding to the scene classification, averagely divide a value interval of the first scene parameter into M continuous sub-intervals, averagely divide a value interval of the second scene parameter into N continuous sub-intervals, where M and N are positive integers greater than one, and respectively determine a section combination to which each piece of data belongs according to a value of the first scene parameter and a value of the second scene parameter of each piece of data in the scene classification, where the section combination is composed of one sub-interval corresponding to the first scene parameter and one sub-interval corresponding to the second scene parameter, and any two section combinations are different from each other, and further may generate heat corresponding to the scene classification based on the determination result. Preferably, the preprocessing module 300 may determine, for each interval combination, a ratio of the number of data pieces belonging to the interval combination to the number of data pieces in the scene classification, and draw the thermodynamic diagram according to the ratio corresponding to each interval combination by taking the first scene parameter and the second scene parameter as a horizontal axis and a vertical axis, respectively.
After generating the thermodynamic diagram, the preprocessing module 300 may further perform partitioning of the thermodynamic regions, that is, for any scene classification, the following processing may be performed for at least one set thermodynamic quantile value: and dividing the thermodynamic diagrams corresponding to the scene classification according to the thermodynamic quantile values to obtain a core thermodynamic area and a non-core thermodynamic area under the thermodynamic quantile values.
Accordingly, for any scene classification, the second testing module 302 may map, for each piece of data in the scene classification, the data to a corresponding position in the thermodynamic diagram corresponding to the scene classification according to a value of a first scene parameter and a value of a second scene parameter of the data, respectively.
In addition, for any of the scene classifications, the second testing module 302 may further perform the following processes for at least one thermal quantile value, respectively: counting the number of data in the core thermal area under the thermal quantile value in each piece of data in the scene classification after mapping, and generating a test evaluation index corresponding to the scene classification under the thermal quantile value according to the statistical result and the judgment result of each piece of data in the scene classification. Preferably, the second testing module 302 may calculate a ratio between the number of data pieces located in the core thermal area and passing the determination result and the number of data pieces located in the core thermal area, obtain a passing rate of the core thermal area under the thermal quantile value, and use the passing rate of the core thermal area as a testing evaluation index corresponding to the scene classification under the thermal quantile value.
Further, for any of the scene classifications, the second testing module 302 may further perform the following processes for at least one thermal quantile value, respectively: counting the number of data pieces in the non-core thermal area under the thermal quantile value in each piece of data in the scene classification after mapping, calculating the ratio of the number of data pieces which are in the non-core thermal area and pass the judgment result to the number of data pieces in the non-core thermal area to obtain the passing rate of the non-core thermal area under the thermal quantile value, and taking the passing rate of the non-core thermal area as a test evaluation index corresponding to the scene classification under the thermal quantile value.
Subsequently, the second testing module 302 may further integrate the testing evaluation indexes corresponding to different scene classifications to obtain a final testing result.
For a specific work flow of the apparatus embodiment shown in fig. 3, reference is made to the related description in the foregoing method embodiment, and details are not repeated.
In a word, by adopting the scheme of the embodiment of the device, manual analysis and other processing are not needed, rapid automatic evaluation of the data test result can be realized, a quantifiable test evaluation index is provided, and the test result can be quantitatively described through a specific test evaluation index, so that manpower and material resources are saved, the processing efficiency is improved, and the passing rate and the like among different unmanned vehicle systems have quantifiable and comparable results and the like under the same thermodynamic diagram and mass data test scenes; moreover, through scene classification, effective splitting of data is realized, so that the processing efficiency of mass data is further improved; in addition, data under different scene classifications can be visually displayed in modes of thermodynamic diagrams and the like, so that visual understanding, analysis and the like are facilitated; in addition, purposeful detailed evaluation and the like of the test result can be realized through flexible setting of the thermal grading value.
According to an embodiment of the present application, an electronic device and a readable storage medium are also provided.
Fig. 4 is a block diagram of an electronic device according to the method of the embodiment of the present application. Electronic devices are intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be examples only, and are not meant to limit implementations of the present application that are described and/or claimed herein.
As shown in fig. 4, the electronic apparatus includes: one or more processors Y01, a memory Y02, and interfaces for connecting the various components, including a high speed interface and a low speed interface. The various components are interconnected using different buses and may be mounted on a common motherboard or in other manners as desired. The processor may process instructions for execution within the electronic device, including instructions stored in or on the memory to display graphical information for a graphical user interface on an external input/output device (such as a display device coupled to the interface). In other embodiments, multiple processors and/or multiple buses may be used, along with multiple memories and multiple memories, as desired. Also, multiple electronic devices may be connected, with each device providing portions of the necessary operations (e.g., as a server array, a group of blade servers, or a multi-processor system). In fig. 4, one processor Y01 is taken as an example.
Memory Y02 is a non-transitory computer readable storage medium as provided herein. Wherein the memory stores instructions executable by at least one processor to cause the at least one processor to perform the methods provided herein. The non-transitory computer readable storage medium of the present application stores computer instructions for causing a computer to perform the methods provided herein.
Memory Y02 is provided as a non-transitory computer readable storage medium that can be used to store non-transitory software programs, non-transitory computer executable programs, and modules, such as program instructions/modules corresponding to the methods of the embodiments of the present application. The processor Y01 executes various functional applications of the server and data processing, i.e., implements the method in the above-described method embodiments, by executing non-transitory software programs, instructions, and modules stored in the memory Y02.
The memory Y02 may include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program required for at least one function; the storage data area may store data created according to use of the electronic device, and the like. Additionally, the memory Y02 may include high speed random access memory, and may also include non-transitory memory, such as at least one magnetic disk storage device, flash memory device, or other non-transitory solid state storage device. In some embodiments, memory Y02 may optionally include memory located remotely from processor Y01, which may be connected to the electronic device via a network. Examples of such networks include, but are not limited to, the internet, intranets, blockchain networks, local area networks, mobile communication networks, and combinations thereof.
The electronic device may further include: an input device Y03 and an output device Y04. The processor Y01, the memory Y02, the input device Y03 and the output device Y04 may be connected by a bus or other means, and the bus connection is exemplified in fig. 4.
The input device Y03 may receive input numeric or character information and generate key signal inputs related to user settings and function control of the electronic device, such as a touch screen, keypad, mouse, track pad, touch pad, pointer, one or more mouse buttons, track ball, joystick, or other input device. The output device Y04 may include a display device, an auxiliary lighting device, a tactile feedback device (e.g., a vibration motor), and the like. The display device may include, but is not limited to, a liquid crystal display, a light emitting diode display, and a plasma display. In some implementations, the display device can be a touch screen.
Various implementations of the systems and techniques described here can be realized in digital electronic circuitry, integrated circuitry, application specific integrated circuits, computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs that are executable and/or interpretable on a programmable system including at least one programmable processor, which may be special or general purpose, receiving data and instructions from, and transmitting data and instructions to, a storage system, at least one input device, and at least one output device.
These computer programs (also known as programs, software applications, or code) include machine instructions for a programmable processor, and may be implemented using high-level procedural and/or object-oriented programming languages, and/or assembly/machine languages. As used herein, the terms "machine-readable medium" and "computer-readable medium" refer to any computer program product, apparatus, and/or device (e.g., magnetic discs, optical disks, memory, programmable logic devices) used to provide machine instructions and/or data to a programmable processor, including a machine-readable medium that receives machine instructions as a machine-readable signal. The term "machine-readable signal" refers to any signal used to provide machine instructions and/or data to a programmable processor.
To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having: a display device (e.g., a cathode ray tube or a liquid crystal display monitor) for displaying information to a user; and a keyboard and a pointing device (e.g., a mouse or a trackball) by which a user can provide input to the computer. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including acoustic, speech, or tactile input.
The systems and techniques described here can be implemented in a computing system that includes a back-end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back-end, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local area networks, wide area networks, blockchain networks, and the internet.
The computer system may include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other.
It should be understood that various forms of the flows shown above may be used, with steps reordered, added, or deleted. For example, the steps described in the present application may be executed in parallel, sequentially, or in different orders, as long as the desired results of the technical solutions disclosed in the present application can be achieved, and the present invention is not limited herein.
The above-described embodiments should not be construed as limiting the scope of the present application. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions may be made in accordance with design requirements and other factors. Any modification, equivalent replacement, and improvement made within the spirit and principle of the present application shall be included in the protection scope of the present application.
Claims (16)
1. A method for testing data, comprising:
acquiring a first data set, operating each piece of data in the first data set, and respectively judging each piece of data according to an operation result to obtain a judgment result of whether each piece of data passes or not;
respectively determining the scene classification of each piece of data in the first data set;
and aiming at each scene classification, respectively mapping each piece of data in the scene classification to a pre-generated statistical graph corresponding to the scene classification, and generating a test evaluation index corresponding to the scene classification according to a mapping result and a judgment result of each piece of data in the scene classification.
2. The method of claim 1,
the method further comprises the following steps: acquiring a second data set, and respectively determining the scene classification of each piece of data in the second data set; and aiming at each scene classification, respectively generating a statistical graph corresponding to the scene classification according to each piece of data in the scene classification.
3. The method of claim 2,
the generating a statistical graph corresponding to the scene classification according to each piece of data in the scene classification includes:
respectively selecting a first scene parameter and a second scene parameter which are interested from scene parameters corresponding to the scene classification, averagely dividing a value interval of the first scene parameter into M continuous subintervals, and averagely dividing a value interval of the second scene parameter into N continuous subintervals, wherein M and N are positive integers which are more than one;
respectively determining the interval combination to which each piece of data belongs according to the value of the first scene parameter and the value of the second scene parameter of each piece of data in the scene classification; the interval combination is composed of a subinterval corresponding to the first scene parameter and a subinterval corresponding to the second scene parameter, and any two interval combinations are different;
generating a thermodynamic diagram corresponding to the scene classification based on the determination result.
4. The method of claim 3,
the generating a thermodynamic diagram corresponding to the scene classification based on the determination result comprises: respectively determining the ratio of the number of data belonging to the interval combination to the number of data in the scene classification aiming at each interval combination; respectively taking the first scene parameter and the second scene parameter as a horizontal axis and a vertical axis, and drawing the thermodynamic diagram according to the ratio corresponding to each interval combination;
The mapping of each piece of data in the scene classification to the pre-generated statistical graph corresponding to the scene classification includes: and for each piece of data in the scene classification, mapping the data to a corresponding position in a thermodynamic diagram corresponding to the scene classification according to the value of the first scene parameter and the value of the second scene parameter of the data.
5. The method of claim 4,
the method further comprises the following steps: for any scene classification, aiming at least one set thermodynamic quantile value, respectively carrying out the following processing: dividing a thermodynamic diagram corresponding to the scene classification according to the thermodynamic quantile values to obtain a core thermodynamic area and a non-core thermodynamic area under the thermodynamic quantile values;
the generating of the test evaluation index corresponding to the scene classification according to the mapping result and the judgment result of each piece of data in the scene classification includes: for any scene classification, respectively performing the following processing aiming at the at least one thermodynamic quantile value: counting the number of data in the core thermal area under the thermal quantile value in each piece of data in the scene classification after mapping, and generating a test evaluation index corresponding to the scene classification under the thermal quantile value according to the statistical result and the judgment result of each piece of data in the scene classification.
6. The method of claim 5,
the generating of the test evaluation index corresponding to the scene classification under the thermal place value according to the statistical result and the judgment result of each piece of data in the scene classification comprises:
and calculating the ratio of the number of data pieces which are positioned in the core thermal area and the judgment result of which is the number of passed data pieces to the number of data pieces which are positioned in the core thermal area to obtain the passing rate of the core thermal area under the thermal quantile value, and taking the passing rate of the core thermal area as the test evaluation index corresponding to the scene classification under the thermal quantile value.
7. The method of claim 6,
the method further comprises the following steps: for any scene classification, respectively performing the following processing aiming at the at least one thermodynamic quantile value: counting the number of data pieces in a non-core thermal area under the thermal grading value in each piece of data in the scene classification after mapping, calculating the ratio of the number of data pieces which are in the non-core thermal area and pass the judgment result to the number of data pieces in the non-core thermal area, obtaining the passing rate of the non-core thermal area under the thermal grading value, and taking the passing rate of the non-core thermal area as a test evaluation index corresponding to the scene classification under the thermal grading value.
8. A data testing apparatus, comprising: the device comprises a first test module and a second test module;
the first test module is used for acquiring a first data set, operating each piece of data in the first data set, and respectively judging each piece of data according to an operation result to obtain a judgment result of whether each piece of data passes or not;
the second testing module is configured to determine a scene classification to which each piece of data in the first data set belongs, map each piece of data in the scene classification to a pre-generated statistical graph corresponding to the scene classification, and generate a testing evaluation index corresponding to the scene classification according to a mapping result and a determination result of each piece of data in the scene classification.
9. The apparatus of claim 8,
the device further comprises: the preprocessing module is used for acquiring a second data set, respectively determining scene classifications to which each piece of data in the second data set belongs, and respectively generating a statistical graph corresponding to each scene classification according to each piece of data in the scene classifications aiming at each scene classification.
10. The apparatus of claim 9,
The preprocessing module selects a first scene parameter and a second scene parameter which are interested from scene parameters corresponding to any scene classification respectively aiming at the scene classification, averagely divides a value section of the first scene parameter into M continuous subintervals, averagely divides a value section of the second scene parameter into N continuous subintervals, wherein M and N are positive integers which are more than one, respectively determining the interval combination of each piece of data according to the value of the first scene parameter and the value of the second scene parameter of each piece of data in the scene classification, and generating a thermodynamic diagram corresponding to the scene classification based on a determination result, wherein the interval combination is composed of a subinterval corresponding to the first scene parameter and a subinterval corresponding to the second scene parameter, and any two interval combinations are different.
11. The apparatus of claim 10,
the preprocessing module respectively determines the ratio of the number of data belonging to the interval combination to the number of data in the scene classification aiming at each interval combination, and draws the thermodynamic diagram according to the ratio corresponding to each interval combination by taking the first scene parameter and the second scene parameter as a horizontal axis and a vertical axis respectively;
And the second testing module maps each piece of data in the scene classification to a corresponding position in the thermodynamic diagram corresponding to the scene classification according to the value of the first scene parameter and the value of the second scene parameter of the data.
12. The apparatus of claim 11,
the preprocessing module is further used for, for any scene classification, respectively performing the following processing for at least one set thermodynamic quantile value: dividing a thermodynamic diagram corresponding to the scene classification according to the thermodynamic quantile values to obtain a core thermodynamic area and a non-core thermodynamic area under the thermodynamic quantile values;
the second testing module classifies any scene, and respectively performs the following processing aiming at the at least one thermal quantile value: counting the number of data in the core thermal area under the thermal quantile value in each piece of data in the scene classification after mapping, and generating a test evaluation index corresponding to the scene classification under the thermal quantile value according to the statistical result and the judgment result of each piece of data in the scene classification.
13. The apparatus of claim 12,
The second testing module calculates the ratio of the number of data pieces which are located in the core thermal area and the judgment result of which is the passing number to the number of data pieces which are located in the core thermal area, obtains the passing rate of the core thermal area under the thermal quantile value, and takes the passing rate of the core thermal area as the testing evaluation index corresponding to the scene classification under the thermal quantile value.
14. The apparatus of claim 13,
the second testing module is further configured to, for any of the scene classifications, perform the following processing for the at least one thermodynamic quantile value, respectively: counting the number of data pieces in a non-core thermal area under the thermal grading value in each piece of data in the scene classification after mapping, calculating the ratio of the number of data pieces which are in the non-core thermal area and pass the judgment result to the number of data pieces in the non-core thermal area, obtaining the passing rate of the non-core thermal area under the thermal grading value, and taking the passing rate of the non-core thermal area as a test evaluation index corresponding to the scene classification under the thermal grading value.
15. An electronic device, comprising:
At least one processor; and
a memory communicatively coupled to the at least one processor; wherein the content of the first and second substances,
the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-7.
16. A non-transitory computer readable storage medium having stored thereon computer instructions for causing the computer to perform the method of any one of claims 1-7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010518935.4A CN111858927B (en) | 2020-06-09 | 2020-06-09 | Data testing method and device, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010518935.4A CN111858927B (en) | 2020-06-09 | 2020-06-09 | Data testing method and device, electronic equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111858927A true CN111858927A (en) | 2020-10-30 |
| CN111858927B CN111858927B (en) | 2023-11-24 |
Family
ID=72987348
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010518935.4A Active CN111858927B (en) | 2020-06-09 | 2020-06-09 | Data testing method and device, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111858927B (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115509909A (en) * | 2022-09-26 | 2022-12-23 | 北京百度网讯科技有限公司 | Test method, test device, electronic equipment and storage medium |
| CN115981179A (en) * | 2022-12-30 | 2023-04-18 | 西安深信科创信息技术有限公司 | Method and device for generating test indexes of automatic driving simulation test scene |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105954040A (en) * | 2016-04-22 | 2016-09-21 | 百度在线网络技术（北京）有限公司 | Testing method and device for driverless automobiles |
| CN106528398A (en) * | 2015-09-15 | 2017-03-22 | 网易（杭州）网络有限公司 | Game software performance visual analysis method |
| US20180354512A1 (en) * | 2017-06-09 | 2018-12-13 | Baidu Online Network Technology (Beijing) Co., Ltd. | Driverless Vehicle Testing Method and Apparatus, Device and Storage Medium |
| CN110059146A (en) * | 2019-04-16 | 2019-07-26 | 珠海金山网络游戏科技有限公司 | A kind of collecting method, calculates equipment and storage medium at server |
| CN110300370A (en) * | 2019-07-02 | 2019-10-01 | 广州纳斯威尔信息技术有限公司 | A kind of reconstruction wifi fingerprint map indoor orientation method |
| US20190378035A1 (en) * | 2018-06-11 | 2019-12-12 | Baidu Online Network Technology (Beijing) Co., Ltd. | Method, apparatus, device and medium for classifying driving scenario data |
| US20200089820A1 (en) * | 2018-09-14 | 2020-03-19 | Sino IC Technology Co.,Ltd. | Ic test information management system based on industrial internet |
| CN111122175A (en) * | 2020-01-02 | 2020-05-08 | 北京百度网讯科技有限公司 | Method and device for testing automatic driving system |
-
2020
- 2020-06-09 CN CN202010518935.4A patent/CN111858927B/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106528398A (en) * | 2015-09-15 | 2017-03-22 | 网易（杭州）网络有限公司 | Game software performance visual analysis method |
| CN105954040A (en) * | 2016-04-22 | 2016-09-21 | 百度在线网络技术（北京）有限公司 | Testing method and device for driverless automobiles |
| US20180354512A1 (en) * | 2017-06-09 | 2018-12-13 | Baidu Online Network Technology (Beijing) Co., Ltd. | Driverless Vehicle Testing Method and Apparatus, Device and Storage Medium |
| US20190378035A1 (en) * | 2018-06-11 | 2019-12-12 | Baidu Online Network Technology (Beijing) Co., Ltd. | Method, apparatus, device and medium for classifying driving scenario data |
| US20200089820A1 (en) * | 2018-09-14 | 2020-03-19 | Sino IC Technology Co.,Ltd. | Ic test information management system based on industrial internet |
| CN110059146A (en) * | 2019-04-16 | 2019-07-26 | 珠海金山网络游戏科技有限公司 | A kind of collecting method, calculates equipment and storage medium at server |
| CN110300370A (en) * | 2019-07-02 | 2019-10-01 | 广州纳斯威尔信息技术有限公司 | A kind of reconstruction wifi fingerprint map indoor orientation method |
| CN111122175A (en) * | 2020-01-02 | 2020-05-08 | 北京百度网讯科技有限公司 | Method and device for testing automatic driving system |
Non-Patent Citations (1)
| Title |
|---|
| 张静;董楠;: "基于运动行为统计图的异常行驶车辆检测", 机电产品开发与创新, no. 03 * |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115509909A (en) * | 2022-09-26 | 2022-12-23 | 北京百度网讯科技有限公司 | Test method, test device, electronic equipment and storage medium |
| CN115509909B (en) * | 2022-09-26 | 2023-11-07 | 北京百度网讯科技有限公司 | Test method, test device, electronic equipment and storage medium |
| CN115981179A (en) * | 2022-12-30 | 2023-04-18 | 西安深信科创信息技术有限公司 | Method and device for generating test indexes of automatic driving simulation test scene |
| CN115981179B (en) * | 2022-12-30 | 2023-11-21 | 安徽深信科创信息技术有限公司 | Automatic driving simulation test scene test index generation method and device |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111858927B (en) | 2023-11-24 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Avila et al. | SUSSING MERGER TREES: the influence of the halo finder | |
| CN111523677B (en) | Method and device for realizing interpretation of prediction result of machine learning model | |
| CN111294233A (en) | Network alarm statistical analysis method, system and computer readable storage medium | |
| CN110704509A (en) | Data classification method, device, equipment and storage medium | |
| CN112035549B (en) | Data mining method, device, computer equipment and storage medium | |
| CN110675644A (en) | Method and device for identifying road traffic lights, electronic equipment and storage medium | |
| CN111858927A (en) | Data testing method and device, electronic equipment and storage medium | |
| CN111539347B (en) | Method and device for detecting target | |
| CN108171617A (en) | A kind of power grid big data analysis method and device | |
| CN111460384B (en) | Policy evaluation method, device and equipment | |
| CN111652153A (en) | Scene automatic identification method and device, unmanned vehicle and storage medium | |
| CN111506803A (en) | Content recommendation method and device, electronic equipment and storage medium | |
| CN113408561A (en) | Model generation method, target detection method, device, equipment and storage medium | |
| CN111291082B (en) | Data aggregation processing method, device, equipment and storage medium | |
| CN111767477B (en) | Retrieval method, retrieval device, electronic equipment and storage medium | |
| CN114064834A (en) | Target location determination method and device, storage medium and electronic equipment | |
| CN111461306B (en) | Feature evaluation method and device | |
| CN106293650A (en) | A kind of folder attribute method to set up and device | |
| CN111597700B (en) | Signal control algorithm evaluation method and device, electronic equipment and readable storage medium | |
| Xiong et al. | Time irreversibility and intrinsics revealing of series with complex network approach | |
| CN114881521A (en) | Service evaluation method, device, electronic equipment and storage medium | |
| CN114218504A (en) | Blocked road segment identification method and device, electronic equipment and storage medium | |
| CN109743203B (en) | Distributed service security combination system and method based on quantitative information flow | |
| CN113987260A (en) | Video pushing method and device, electronic equipment and storage medium | |
| CN112382090A (en) | Method, apparatus, device and storage medium for outputting information |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20211018 Address after: 105 / F, building 1, No. 10, Shangdi 10th Street, Haidian District, Beijing 100085 Applicant after: Apollo Intelligent Technology (Beijing) Co.,Ltd. Address before: 2 / F, baidu building, 10 Shangdi 10th Street, Haidian District, Beijing 100085 Applicant before: BEIJING BAIDU NETCOM SCIENCE AND TECHNOLOGY Co.,Ltd. |
|
| TA01 | Transfer of patent application right | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |