CN111858927B - Data testing method and device, electronic equipment and storage medium - Google Patents
Data testing method and device, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN111858927B CN111858927B CN202010518935.4A CN202010518935A CN111858927B CN 111858927 B CN111858927 B CN 111858927B CN 202010518935 A CN202010518935 A CN 202010518935A CN 111858927 B CN111858927 B CN 111858927B
- Authority
- CN
- China
- Prior art keywords
- data
- scene
- scene classification
- piece
- classification
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000012360 testing method Methods 0.000 title claims abstract description 95
- 238000010586 diagram Methods 0.000 claims abstract description 47
- 238000011156 evaluation Methods 0.000 claims abstract description 44
- 238000000034 method Methods 0.000 claims abstract description 39
- 238000013507 mapping Methods 0.000 claims abstract description 21
- 238000011282 treatment Methods 0.000 claims abstract description 5
- 230000015654 memory Effects 0.000 claims description 21
- 238000012545 processing Methods 0.000 claims description 17
- 238000007781 pre-processing Methods 0.000 claims description 10
- 230000008569 process Effects 0.000 claims description 8
- 239000000758 substrate Substances 0.000 claims 3
- 239000000523 sample Substances 0.000 claims 1
- 238000010998 test method Methods 0.000 claims 1
- 239000000463 material Substances 0.000 abstract description 5
- 230000008859 change Effects 0.000 description 7
- 238000004590 computer program Methods 0.000 description 6
- 238000004891 communication Methods 0.000 description 4
- 230000006870 function Effects 0.000 description 4
- 238000012986 modification Methods 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- 230000000007 visual effect Effects 0.000 description 3
- 230000008901 benefit Effects 0.000 description 2
- 238000013100 final test Methods 0.000 description 2
- 230000003993 interaction Effects 0.000 description 2
- 239000004973 liquid crystal related substance Substances 0.000 description 2
- 238000005065 mining Methods 0.000 description 2
- 230000006399 behavior Effects 0.000 description 1
- 230000001413 cellular effect Effects 0.000 description 1
- 238000004883 computer application Methods 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000002474 experimental method Methods 0.000 description 1
- 238000010295 mobile communication Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000001953 sensory effect Effects 0.000 description 1
- 238000004088 simulation Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/35—Clustering; Classification
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/36—Creation of semantic tools, e.g. ontology or thesauri
- G06F16/367—Ontology
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Databases & Information Systems (AREA)
- Life Sciences & Earth Sciences (AREA)
- Animal Behavior & Ethology (AREA)
- Computational Linguistics (AREA)
- Artificial Intelligence (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The application discloses a data testing method, a data testing device, electronic equipment and a storage medium, and relates to the field of automatic driving, wherein the method can comprise the following steps: acquiring a first data set, operating each piece of data in the first data set, and respectively judging each piece of data according to an operation result to obtain a judgment result of whether each piece of data passes or not; respectively determining scene classification of each piece of data in the first data set; for each scene classification, mapping each piece of data in the scene classification into a statistics diagram corresponding to the scene classification, which is generated in advance, and generating a test evaluation index corresponding to the scene classification according to the mapping result and the judgment result of each piece of data in the scene classification. By applying the scheme of the application, manpower and material resources can be saved, the treatment efficiency can be improved, and the like.
Description
Technical Field
The present application relates to computer application technologies, and in particular, to a data testing method and apparatus, an electronic device, and a storage medium in the field of autopilot.
Background
In the unmanned vehicle test, besides the common road test, the virtualization test based on the simulation platform is required to be carried out, so that the performance of the unmanned vehicle can be effectively verified with lower cost, higher efficiency and more comprehensive coverage.
At present, a mode of playback random test based on mass data is generally adopted, namely, experiments are carried out randomly through a data playback mode, and test results are obtained through manual analysis and the like, but the mode needs to consume large manpower and material resources and has low efficiency.
Disclosure of Invention
The application provides a data testing method, a data testing device, electronic equipment and a storage medium.
A data testing method, comprising:
acquiring a first data set, operating each piece of data in the first data set, and respectively judging each piece of data according to an operation result to obtain a judgment result of whether each piece of data passes or not;
determining scene classification to which each piece of data in the first data set belongs respectively;
and mapping each piece of data in the scene classification into a pre-generated statistical graph corresponding to the scene classification, and generating a test evaluation index corresponding to the scene classification according to the mapping result and the judgment result of each piece of data in the scene classification.
A data testing apparatus, comprising: the first test module and the second test module;
the first test module is used for acquiring a first data set, running each piece of data in the first data set, and judging each piece of data according to the running result to obtain a judging result of whether each piece of data passes or not;
the second test module is configured to determine a scene classification to which each piece of data in the first data set belongs, map each piece of data in the scene classification to a statistics chart corresponding to the scene classification generated in advance for each scene classification, and generate a test evaluation index corresponding to the scene classification according to a mapping result and a judgment result of each piece of data in the scene classification.
An electronic device, comprising:
at least one processor; and
a memory communicatively coupled to the at least one processor; wherein,
the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method as described above.
A non-transitory computer readable storage medium storing computer instructions for causing the computer to perform the method as described above.
A computer program product comprising a computer program which, when executed by a processor, implements a method as described above.
One embodiment of the above application has the following advantages or benefits: the method can realize rapid automatic evaluation of the data test result without manual analysis and other processing, and provide quantifiable test evaluation indexes, namely, the test result can be quantitatively described through specific test evaluation indexes, so that manpower and material resources are saved, and the processing efficiency is improved. It should be understood that the description in this section is not intended to identify key or critical features of the embodiments of the disclosure, nor is it intended to be used to limit the scope of the disclosure. Other features of the present disclosure will become apparent from the following specification.
Drawings
The drawings are included to provide a better understanding of the present application and are not to be construed as limiting the application. Wherein:
FIG. 1 is a flow chart of an embodiment of a data testing method according to the present application;
FIG. 2 is a schematic diagram of the overall implementation process of the data testing method according to the present application;
FIG. 3 is a schematic diagram of the structure of a data testing device 30 according to an embodiment of the present application;
fig. 4 is a block diagram of an electronic device according to a method according to an embodiment of the application.
Detailed Description
Exemplary embodiments of the present application will now be described with reference to the accompanying drawings, in which various details of the embodiments of the present application are included to facilitate understanding, and are to be considered merely exemplary. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the application. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.
In addition, it should be understood that the term "and/or" herein is merely one association relationship describing the associated object, and means that three relationships may exist, for example, a and/or B may mean: a exists alone, A and B exist together, and B exists alone. In addition, the character "/" herein generally indicates that the front and rear associated objects are an "or" relationship.
The application provides a data testing method, which aims at a first data set to be processed, can run each piece of data in the first data set, can respectively judge each piece of data according to the running result so as to obtain a judging result of whether each piece of data passes or not, can respectively determine scene classification to which each piece of data belongs, and can further realize rapid automatic evaluation and the like on the data testing result according to the judging result and the scene classification result which are obtained based on statistics diagrams corresponding to different scene classifications which are generated in advance.
In the following, a method of generating a statistical map corresponding to different scene categories will be described first.
A second data set is acquired, wherein the second data set can comprise mass data accumulated by road testing, field acquisition, manual creation and the like.
The scene classification to which each piece of data in the second data set belongs can be determined separately. For example, the data in the second data set may be respectively divided into the scene categories by using modes such as scene mode recognition and category mining. The specific scene classification can be determined according to actual needs, such as left turn scenes, lane changing scenes, and the like. For a piece of data, it may belong to only one scene category or may belong to multiple scene categories, and no influence is exerted on subsequent processing. Through scene classification, effective splitting of data is achieved, and therefore processing efficiency of mass data is improved.
The specific form of the statistical chart may be a thermodynamic chart or a scattergram having a certain transparency and being capable of accumulating, and the thermodynamic chart will be described as an example.
For each scene classification, a thermodynamic diagram corresponding to the scene classification can be generated according to each piece of data in the scene classification. Preferably, for any scene classification, a first scene parameter and a second scene parameter of interest may be selected from the scene parameters corresponding to the scene classification, a value interval of the first scene parameter is divided into M continuous subintervals, a value interval of the second scene parameter is divided into N continuous subintervals, M and N are positive integers greater than one, and according to the value of the first scene parameter and the value of the second scene parameter of each piece of data in the scene classification, a combination of intervals to which each piece of data belongs is determined, where the combination of intervals is composed of one subinterval corresponding to the first scene parameter and one subinterval corresponding to the second scene parameter, any two combinations of intervals are different, and a thermodynamic diagram corresponding to the scene classification may be generated based on a determination result.
Scene parameters corresponding to different scene classifications may also be different. For any scene classification, the first scene parameter and the second scene parameter of interest may be selected from the scene parameters corresponding to the scene classification, so that all the scene parameters are not needed, but some key/typical scene parameters, usually two, may be selected, and the key/typical scene parameters correspond to the horizontal axis and the vertical axis of the thermodynamic diagram respectively. For example, for a lane change scene, two scene parameters of lane change time and lane change speed can be selected, and assuming that the duration of one piece of data is 2 minutes, and the lane change is performed from 60 seconds to 1 minute and 15 seconds, then the lane change duration is 15 seconds, and the lane change speed can be the average speed in the lane change process, etc.
For the selected first scene parameter and second scene parameter, the value interval of the first scene parameter can be divided into M continuous subintervals on average, the value interval of the second scene parameter can be divided into N continuous subintervals on average, M and N are positive integers which are larger than one, and specific values can be determined according to actual needs.
By combining the sub-sections, a plurality of section combinations can be obtained. For example, the values of M and N are all 100, that is, 100 subintervals corresponding to the first scene parameter can be obtained respectively, namely subinterval 1 to subinterval 100, and 100 subintervals corresponding to the second scene parameter can be obtained respectively, namely subinterval 101 to subinterval 200, so that subinterval 1 and subinterval 101 can be combined to obtain one interval combination, subinterval 1 and subinterval 102 can be combined to obtain one interval combination, subinterval 2 and subinterval 101 can also be combined to obtain one interval combination, and the like. For any scene classification, the section combination of each piece of data can be determined according to the value of the first scene parameter and the value of the second scene parameter of each piece of data in the scene classification, and then the thermodynamic diagram corresponding to the scene classification can be generated based on the determination result. Preferably, for each section combination, a ratio of the number of data bars belonging to the section combination to the number of data bars in the scene classification may be determined, the first scene parameter and the second scene parameter are respectively taken as a horizontal axis and a vertical axis, a thermodynamic diagram is drawn according to the ratio corresponding to each section combination, and how to draw the thermodynamic diagram is the prior art.
Through the thermodynamic diagram mode, the data under different scene classifications can be visually displayed, and visual understanding and analysis and the like are facilitated.
After the thermodynamic diagram is generated, the thermal area may be further divided, that is, for any scene classification, the following processes may be performed for at least one set thermal index value, respectively: and dividing the thermodynamic diagram corresponding to the scene classification according to the thermodynamic dividing value to obtain a core thermodynamic region and a non-core thermodynamic region under the thermodynamic dividing value.
For example, the thermal dividing value is 90 dividing bits, the interval combinations may be ordered according to the order of the number of data stripes belonging to each interval combination from large to small, then, the ratio corresponding to the interval combination in the first position after the ordering and the interval combination in the second position after the ordering (that is, the ratio of the number of data stripes belonging to the interval combination to the number of data stripes in the scene classification) may be added, the added value is compared with 90%, if the added value is less than 90%, the ratio corresponding to the interval combination in the next position after the ordering may be continuously added with the value, the added value may be compared with 90% again, and so on, if the added value is equal to 90%, the processing may be stopped, the area corresponding to the interval combination corresponding to the latest added ratio and the interval combination before the latest added ratio may be used as the core thermal area, and if the added value is greater than 90%, the area corresponding to the interval combination before the latest added ratio may be used as the core thermal area, and the area other than the core thermal area may be used as the non-core thermal area. In the same way, a core thermal region and a non-core thermal region under thermal split values of 60, 70, 80 and the like can be obtained respectively.
The number and the value of the thermodynamic dividing values can be determined according to actual needs. Through flexible setting of thermal index values, purposeful refinement evaluation and the like of test results can be realized.
After the processing is completed, data testing for mass data can be performed. And the first data set comprises mass data which needs to be subjected to data testing.
FIG. 1 is a flowchart of an embodiment of a data testing method according to the present application. As shown in fig. 1, the following detailed implementation is included.
In 101, a first data set is acquired, each piece of data in the first data set is operated, and each piece of data is respectively judged according to an operation result, so that a judgment result of whether each piece of data passes or not is obtained.
At 102, a scene classification to which each piece of data in the first dataset belongs is determined.
In 103, each piece of data in the scene classification is mapped to a statistics map corresponding to the scene classification generated in advance, and a test evaluation index corresponding to the scene classification is generated according to the mapping result and the judgment result of each piece of data in the scene classification.
For the first data set, each piece of data in the first data set can be firstly executed, preferably, the first data set can be executed in a distributed cloud computing platform, and speed up can be realized in a parallelization mode, so that how to execute the data is the prior art.
In addition, each piece of data can be respectively judged according to the operation result, so that a judgment result of whether each piece of data passes or not is obtained, if so, whether the behavior of the unmanned vehicle in each piece of data is reasonable and legal or accords with the expectation can be judged according to the set rule, and the judgment result of whether the unmanned vehicle passes or not is given according to the set rule.
The scene classification to which each piece of data in the first data set belongs can also be determined respectively. In practical application, the scene classification of each piece of data can be finished in advance, namely, the pre-classification is carried out, and the scene classification can be carried out when the data are needed to be used, wherein the scene classification mode is the same as the scene classification mode when the thermodynamic diagram is generated.
For each scene classification, each piece of data in the scene classification can be mapped into a thermodynamic diagram corresponding to the scene classification. Preferably, for each piece of data in the scene classification, the data can be mapped to a corresponding position in a thermodynamic diagram corresponding to the scene classification according to the value of the first scene parameter and the value of the second scene parameter of the data.
And then, according to the mapping result and the judging result of each piece of data in the scene classification, generating a test evaluation index corresponding to the scene classification. Preferably, for any scene classification, the following processing may be performed separately for each thermal index value of the set at least one thermal index value: and after the statistics mapping, the number of data in the core thermal area under the thermal index value in each piece of data in the scene classification generates a test evaluation index corresponding to the scene classification under the thermal index value according to the statistics result and the judgment result of each piece of data in the scene classification, preferably, the ratio of the number of data in the core thermal area and the number of data in the core thermal area, which are passed by the judgment result, can be calculated to obtain the passing rate of the core thermal area under the thermal index value, and the passing rate of the core thermal area is used as the test evaluation index corresponding to the scene classification under the thermal index value.
For example, if the number of data bars in the core thermal area is 480 and the number of data bars in the core thermal area is 500, the passing rate of the core thermal area is 480/500.
Further, for any scene classification, for each thermal index value of the set at least one thermal index value, the following processing may be performed separately: and counting the number of data in the non-core thermal area under the thermal index value in each piece of data in the scene classification after mapping, calculating the ratio of the number of data which are positioned in the non-core thermal area and pass by the judgment result to the number of data which are positioned in the non-core thermal area, obtaining the passing rate of the non-core thermal area under the thermal index value, taking the passing rate of the non-core thermal area as a test evaluation index corresponding to the scene classification under the thermal index value, and taking the passing rate of the core thermal area and the passing rate of the non-core thermal area as test evaluation indexes corresponding to the scene classification under the thermal index value, thereby further enriching the test evaluation indexes and providing more valuable evaluation information and the like for users.
Assuming that the set thermal index value is one and 90 index, the test evaluation index corresponding to different scene classifications under the thermal index value can be obtained. Assuming that the set thermal index values are three and respectively 70 index, 80 index and 90 index, the test evaluation index corresponding to different scene classifications under the thermal index value of 70 index, the test evaluation index corresponding to different scene classifications under the thermal index value of 80 index and the test evaluation index corresponding to different scene classifications under the thermal index value of 90 index can be obtained respectively.
In addition, the test evaluation indexes corresponding to different scene classifications can be integrated, so that a final test result is obtained.
Based on the above description, it can be seen that the method in this embodiment does not need to perform manual analysis and other processes, can implement rapid and automatic evaluation on the data test result, and provides quantifiable test evaluation indexes, i.e. can quantitatively describe the test result through specific test evaluation indexes, thereby saving manpower and material resources, improving processing efficiency, and enabling the passing rate and the like between different unmanned vehicle systems to have quantifiable and comparable results and the like under the same thermodynamic diagram and mass data test scenarios.
Fig. 2 is a schematic diagram of an overall implementation process of the data testing method according to the present application. As shown in fig. 2, the method of the present application mainly includes two links, namely thermodynamic diagram generation and mass data testing, and specific implementation of each link is referred to the foregoing related description and will not be repeated. In addition, an associated thermodynamic diagram including the judgment result can be generated, namely, after each piece of data is mapped onto the thermodynamic diagram, the corresponding judgment result can be marked on the thermodynamic diagram, so that the associated thermodynamic diagram is formed, and manual understanding and the like are facilitated.
It should be noted that, for the sake of simplicity of description, the foregoing method embodiments are expressed as a series of combinations of actions, but it should be understood by those skilled in the art that the present application is not limited by the order of actions described, as some steps may be performed in other order or simultaneously in accordance with the present application. Further, those skilled in the art will also appreciate that the embodiments described in the specification are all preferred embodiments, and that the acts and modules referred to are not necessarily required for the present application.
The above description of the method embodiments further describes the solution of the present application by means of device embodiments.
Fig. 3 is a schematic diagram of the composition structure of an embodiment 30 of the data testing device according to the present application. As shown in fig. 3, includes: a first test module 301 and a second test module 302.
The first test module 301 is configured to obtain a first data set, run each piece of data in the first data set, and respectively determine each piece of data according to the running result, so as to obtain a determination result of whether each piece of data passes or not.
The second test module 302 is configured to determine a scenario classification to which each piece of data in the first data set belongs, map each piece of data in the scenario classification to a statistics chart corresponding to the scenario classification generated in advance for each scenario classification, and generate a test evaluation index corresponding to the scenario classification according to a mapping result and a judgment result of each piece of data in the scenario classification.
It can be seen that, to implement the scheme described in this embodiment, statistics maps corresponding to different scene classifications need to be generated in advance. To this end, the apparatus shown in fig. 3 may further include: the preprocessing module 300 is configured to obtain the second data set, determine a scene classification to which each piece of data in the second data set belongs, and generate, for each scene classification, a statistical graph corresponding to the scene classification according to each piece of data in the scene classification.
The second data set may include massive data accumulated by way of road testing, field collection, manual creation, etc. The data can be respectively divided into the belonging scene categories by adopting modes such as scene mode identification, category mining and the like.
For each scene classification, the preprocessing module 300 may generate a statistical map, such as a thermodynamic diagram, corresponding to the scene classification according to each piece of data in the scene classification. Preferably, for any scene classification, the preprocessing module 300 may select a first scene parameter and a second scene parameter of interest from the scene parameters corresponding to the scene classification, divide the value interval of the first scene parameter into M continuous subintervals, divide the value interval of the second scene parameter into N continuous subintervals, where M and N are positive integers greater than one, and determine, according to the value of the first scene parameter and the value of the second scene parameter of each piece of data in the scene classification, an interval combination to which each piece of data belongs, where the interval combination is composed of one subinterval corresponding to the first scene parameter and one subinterval corresponding to the second scene parameter, and any two interval combinations are different, so that a thermodynamic diagram corresponding to the scene classification may be generated based on a determination result. Preferably, the preprocessing module 300 may determine, for each section combination, a ratio of the number of data stripes belonging to the section combination to the number of data stripes in the scene classification, and draw a thermodynamic diagram according to the ratio corresponding to each section combination by taking the first scene parameter and the second scene parameter as the horizontal axis and the vertical axis, respectively.
After generating the thermodynamic diagram, the preprocessing module 300 may further perform the division of the thermal area, that is, for any scene classification, the following processes may be performed for at least one set thermal split value, respectively: and dividing the thermodynamic diagram corresponding to the scene classification according to the thermodynamic dividing value to obtain a core thermodynamic region and a non-core thermodynamic region under the thermodynamic dividing value.
Accordingly, for any scene classification, the second test module 302 may map each piece of data in the scene classification to a corresponding position in the thermodynamic diagram corresponding to the scene classification according to the value of the first scene parameter and the value of the second scene parameter of the data.
In addition, for any scene classification, the second test module 302 may further perform the following processing for at least one thermal index value: and counting the number of data in the core thermal area under the thermal index value in each piece of data in the scene classification after mapping, and generating a test evaluation index corresponding to the scene classification under the thermal index value according to the counting result and the judgment result of each piece of data in the scene classification. Preferably, the second test module 302 may calculate a ratio of the number of data stripes that are located in the core thermal area and pass through the determination result to the number of data stripes located in the core thermal area, to obtain a passing rate of the core thermal area under the thermal index value, and use the passing rate of the core thermal area as a test evaluation index corresponding to the scene classification under the thermal index value.
Further, for any scene classification, the second test module 302 may further perform the following processing for at least one thermal index value: and counting the number of data in the non-core thermal area under the thermal index value in each piece of data in the scene classification after mapping, calculating the ratio of the number of data passing through the non-core thermal area and the number of data passing through the non-core thermal area as a judgment result to obtain the passing rate of the non-core thermal area under the thermal index value, and taking the passing rate of the non-core thermal area as a test evaluation index corresponding to the scene classification under the thermal index value.
Subsequently, the second test module 302 may further integrate test evaluation indexes corresponding to different scene classifications to obtain a final test result.
The specific workflow of the embodiment of the apparatus shown in fig. 3 is referred to the related description in the foregoing method embodiment, and will not be repeated.
In a word, by adopting the scheme of the embodiment of the device, the rapid automatic evaluation of the data test result can be realized without the treatment of manual analysis and the like, and the quantifiable test evaluation index is provided, namely, the test result can be quantitatively described through the specific test evaluation index, so that manpower and material resources are saved, the processing efficiency is improved, and the passing rate and the like among different unmanned vehicle systems can be quantified and compared under the same thermodynamic diagram and mass data test scene; moreover, by scene classification, the effective splitting of the data is realized, so that the processing efficiency of mass data and the like are further improved; in addition, the data under different scene classifications can be visually displayed in a thermodynamic diagram mode and the like, so that visual understanding, analysis and the like are facilitated; further, by flexibly setting the thermal index value, it is possible to realize a purposeful refinement evaluation of the test result, and the like.
According to an embodiment of the present application, the present application also provides an electronic device and a readable storage medium.
As shown in fig. 4, is a block diagram of an electronic device according to a method according to an embodiment of the application. Electronic devices are intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular telephones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be exemplary only, and are not meant to limit implementations of the applications described and/or claimed herein.
As shown in fig. 4, the electronic device includes: one or more processors Y01, memory Y02, and interfaces for connecting the components, including high-speed interfaces and low-speed interfaces. The various components are interconnected using different buses and may be mounted on a common motherboard or in other manners as desired. The processor may process instructions executing within the electronic device, including instructions stored in or on memory to display graphical information of a graphical user interface on an external input/output device, such as a display device coupled to the interface. In other embodiments, multiple processors and/or multiple buses may be used, if desired, along with multiple memories and multiple memories. Also, multiple electronic devices may be connected, each providing a portion of the necessary operations (e.g., as a server array, a set of blade servers, or a multiprocessor system). In fig. 4, a processor Y01 is taken as an example.
The memory Y02 is a non-transitory computer readable storage medium provided by the present application. Wherein the memory stores instructions executable by the at least one processor to cause the at least one processor to perform the methods provided by the present application. The non-transitory computer readable storage medium of the present application stores computer instructions for causing a computer to perform the method provided by the present application.
The memory Y02 serves as a non-transitory computer readable storage medium storing a non-transitory software program, a non-transitory computer executable program, and modules, such as program instructions/modules corresponding to the methods in the embodiments of the present application. The processor Y01 executes various functional applications of the server and data processing, i.e., implements the methods in the above-described method embodiments, by running non-transitory software programs, instructions, and modules stored in the memory Y02.
The memory Y02 may include a memory program area that may store an operating system, at least one application program required for functions, and a memory data area; the storage data area may store data created according to the use of the electronic device, etc. In addition, memory Y02 may include high-speed random access memory, and may also include non-transitory memory, such as at least one magnetic disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory Y02 may optionally include memory located remotely from processor Y01, which may be connected to the electronic device via a network. Examples of such networks include, but are not limited to, the internet, intranets, blockchain networks, local area networks, mobile communication networks, and combinations thereof.
The electronic device may further include: an input device Y03 and an output device Y04. The processor Y01, memory Y02, input device Y03, and output device Y04 may be connected by a bus or otherwise, for example in fig. 4.
The input device Y03 may receive input numeric or character information and generate key signal inputs related to user settings and function control of the electronic device, such as a touch screen, keypad, mouse, trackpad, touchpad, pointer stick, one or more mouse buttons, trackball, joystick, and like input devices. The output means Y04 may include a display device, an auxiliary lighting means, a tactile feedback means (e.g., a vibration motor), and the like. The display device may include, but is not limited to, a liquid crystal display, a light emitting diode display, and a plasma display. In some implementations, the display device may be a touch screen.
Various implementations of the systems and techniques described here can be realized in digital electronic circuitry, integrated circuitry, application specific integrated circuitry, computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs, the one or more computer programs may be executed and/or interpreted on a programmable system including at least one programmable processor, which may be a special purpose or general-purpose programmable processor, that may receive data and instructions from, and transmit data and instructions to, a storage system, at least one input device, and at least one output device.
These computing programs (also referred to as programs, software applications, or code) include machine instructions for a programmable processor, and may be implemented in a high-level procedural and/or object-oriented programming language, and/or in assembly/machine language. The terms "machine-readable medium" and "computer-readable medium" as used herein refer to any computer program product, apparatus, and/or device (e.g., magnetic discs, optical disks, memory, programmable logic devices) for providing machine instructions and/or data to a programmable processor, including a machine-readable medium that receives machine instructions as a machine-readable signal. The term "machine-readable signal" refers to any signal used to provide machine instructions and/or data to a programmable processor.
To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having: a display device (e.g., a cathode ray tube or a liquid crystal display monitor) for displaying information to a user; and a keyboard and pointing device (e.g., a mouse or trackball) by which a user can provide input to the computer. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including acoustic input, speech input, or tactile input.
The systems and techniques described here can be implemented in a computing system that includes a background component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such background, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local area networks, wide area networks, blockchain networks, and the internet.
The computer system may include a client and a server. The client and server are typically remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other.
It should be appreciated that various forms of the flows shown above may be used to reorder, add, or delete steps. For example, the steps described in the present application may be performed in parallel, sequentially, or in a different order, so long as the desired results of the technical solution disclosed in the present application can be achieved, and are not limited herein.
The above embodiments do not limit the scope of the present application. It will be apparent to those skilled in the art that various modifications, combinations, sub-combinations and alternatives are possible, depending on design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present application should be included in the scope of the present application.
Claims (14)
1. A method of testing data, comprising:
acquiring a first data set, operating each piece of data in the first data set, and respectively judging each piece of data according to an operation result to obtain a judgment result of whether each piece of data passes or not;
determining scene classification to which each piece of data in the first data set belongs respectively;
for each scene classification, mapping each piece of data in the scene classification into a statistics graph corresponding to the scene classification, which is generated in advance, and generating a test evaluation index corresponding to the scene classification according to a mapping result and a judgment result of each piece of data in the scene classification, wherein the test evaluation index comprises the following steps: for at least one thermal index value, the following treatments are respectively carried out: counting the number of data in a core thermal area under the thermal index value in each piece of data in the scene classification after mapping, and generating a test evaluation index corresponding to the scene classification under the thermal index value according to a counting result and a judging result of each piece of data in the scene classification; for any scene classification, the following treatments are respectively carried out for at least one set thermal index value: dividing the thermodynamic diagram corresponding to the scene classification according to the thermodynamic dividing value to obtain a core thermodynamic region and a non-core thermodynamic region under the thermodynamic dividing value, wherein the thermodynamic diagram is the statistical diagram.
2. The method of claim 1, wherein the step of determining the position of the substrate comprises,
the method further comprises the steps of: acquiring a second data set, and respectively determining scene classification of each piece of data in the second data set; and generating a statistical graph corresponding to each scene classification according to each piece of data in the scene classification for each scene classification.
3. The method of claim 2, wherein the step of determining the position of the substrate comprises,
the generating the statistical graph corresponding to the scene classification according to each piece of data in the scene classification comprises:
selecting a first scene parameter and a second scene parameter which are interested from scene parameters corresponding to the scene classification, dividing the value interval of the first scene parameter into M continuous subintervals on average, dividing the value interval of the second scene parameter into N continuous subintervals on average, wherein M and N are positive integers which are larger than one;
respectively determining interval combinations of each piece of data according to the values of the first scene parameters and the values of the second scene parameters of each piece of data in the scene classification; the interval combination consists of a subinterval corresponding to the first scene parameter and a subinterval corresponding to the second scene parameter, and any two interval combinations are different;
and generating a thermodynamic diagram corresponding to the scene classification based on the determination result.
4. The method of claim 3, wherein the step of,
the generating a thermodynamic diagram corresponding to the scene classification based on the determination result comprises: for each interval combination, determining the ratio of the number of data belonging to the interval combination to the number of data in the scene classification; the first scene parameter and the second scene parameter are respectively used as a horizontal axis and a vertical axis, and the thermodynamic diagram is drawn according to the ratio corresponding to each interval combination;
the mapping each piece of data in the scene classification to a pre-generated statistical chart corresponding to the scene classification comprises the following steps: and mapping each piece of data in the scene classification to a corresponding position in a thermodynamic diagram corresponding to the scene classification according to the value of the first scene parameter and the value of the second scene parameter of the data.
5. The method of claim 1, wherein the step of determining the position of the substrate comprises,
the generating the test evaluation index corresponding to the scene classification under the thermal index value according to the statistical result and the judgment result of each piece of data in the scene classification comprises:
calculating the ratio of the number of data stripes passing through the core thermal area and the number of data stripes passing through the core thermal area according to the judgment result to obtain the passing rate of the core thermal area under the thermal index value, and taking the passing rate of the core thermal area as a test evaluation index corresponding to the scene classification under the thermal index value.
6. The method of claim 5, wherein the step of determining the position of the probe is performed,
the method further comprises the steps of: for any scene classification, for the at least one thermal index value, the following processes are performed: and counting the number of data in the non-core thermal area under the thermal index value in each piece of data in the scene classification after mapping, calculating the ratio of the number of data passing through the non-core thermal area to obtain the passing rate of the non-core thermal area under the thermal index value, and taking the passing rate of the non-core thermal area as a test evaluation index corresponding to the scene classification under the thermal index value.
7. A data testing apparatus, comprising: the device comprises a preprocessing module, a first testing module and a second testing module;
the first test module is used for acquiring a first data set, running each piece of data in the first data set, and judging each piece of data according to the running result to obtain a judging result of whether each piece of data passes or not;
the second test module is configured to determine a scene classification to which each piece of data in the first data set belongs, map each piece of data in the scene classification to a statistics chart corresponding to the scene classification generated in advance for each scene classification, and generate a test evaluation index corresponding to the scene classification according to a mapping result and a judgment result of each piece of data in the scene classification, where the test evaluation index comprises: for at least one thermal index value, the following treatments are respectively carried out: counting the number of data in a core thermal area under the thermal index value in each piece of data in the scene classification after mapping, and generating a test evaluation index corresponding to the scene classification under the thermal index value according to a counting result and a judging result of each piece of data in the scene classification;
the preprocessing module is used for classifying any scene and respectively carrying out the following processing on at least one set thermodynamic quantile value: dividing the thermodynamic diagram corresponding to the scene classification according to the thermodynamic dividing value to obtain a core thermodynamic region and a non-core thermodynamic region under the thermodynamic dividing value, wherein the thermodynamic diagram is the statistical diagram.
8. The apparatus of claim 7, wherein the device comprises a plurality of sensors,
the preprocessing module is further used for acquiring a second data set, determining scene classification to which each piece of data in the second data set belongs, and generating a statistical graph corresponding to each scene classification according to each piece of data in the scene classification according to each scene classification.
9. The apparatus of claim 8, wherein the device comprises a plurality of sensors,
the preprocessing module respectively selects a first scene parameter and a second scene parameter which are interested from scene parameters corresponding to the scene classification, the value interval of the first scene parameter is divided into M continuous subintervals in average, the value interval of the second scene parameter is divided into N continuous subintervals in average, M and N are positive integers which are larger than one, and interval combinations which each piece of data belong to are respectively determined according to the value of the first scene parameter and the value of the second scene parameter of each piece of data in the scene classification, wherein the interval combinations consist of one subinterval which corresponds to the first scene parameter and one subinterval which corresponds to the second scene parameter, any two interval combinations are different, and a thermodynamic diagram which corresponds to the scene classification is generated based on a determination result.
10. The apparatus of claim 9, wherein the device comprises a plurality of sensors,
the preprocessing module respectively determines the ratio of the number of data belonging to the interval combination to the number of data in the scene classification according to each interval combination, takes the first scene parameter and the second scene parameter as a horizontal axis and a vertical axis respectively, and draws the thermodynamic diagram according to the ratio corresponding to each interval combination;
the second test module maps each piece of data in the scene classification to a corresponding position in a thermodynamic diagram corresponding to the scene classification according to the value of the first scene parameter and the value of the second scene parameter of the data.
11. The apparatus of claim 7, wherein the device comprises a plurality of sensors,
and the second test module calculates the ratio of the number of data strips which are positioned in the core thermal area and pass through the judgment result to the number of data strips positioned in the core thermal area to obtain the passing rate of the core thermal area under the thermal index value, and takes the passing rate of the core thermal area as a test evaluation index corresponding to the scene classification under the thermal index value.
12. The apparatus of claim 11, wherein the device comprises a plurality of sensors,
the second test module is further configured to, for any scene classification, perform the following processing for the at least one thermal index value: and counting the number of data in the non-core thermal area under the thermal index value in each piece of data in the scene classification after mapping, calculating the ratio of the number of data passing through the non-core thermal area to obtain the passing rate of the non-core thermal area under the thermal index value, and taking the passing rate of the non-core thermal area as a test evaluation index corresponding to the scene classification under the thermal index value.
13. An electronic device, comprising:
at least one processor; and
a memory communicatively coupled to the at least one processor; wherein,
the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-6.
14. A non-transitory computer readable storage medium storing computer instructions for causing the computer to perform the method of any one of claims 1-6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010518935.4A CN111858927B (en) | 2020-06-09 | 2020-06-09 | Data testing method and device, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010518935.4A CN111858927B (en) | 2020-06-09 | 2020-06-09 | Data testing method and device, electronic equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111858927A CN111858927A (en) | 2020-10-30 |
| CN111858927B true CN111858927B (en) | 2023-11-24 |
Family
ID=72987348
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010518935.4A Active CN111858927B (en) | 2020-06-09 | 2020-06-09 | Data testing method and device, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111858927B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115509909B (en) * | 2022-09-26 | 2023-11-07 | 北京百度网讯科技有限公司 | Test method, test device, electronic equipment and storage medium |
| CN115981179B (en) * | 2022-12-30 | 2023-11-21 | 安徽深信科创信息技术有限公司 | Automatic driving simulation test scene test index generation method and device |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105954040A (en) * | 2016-04-22 | 2016-09-21 | 百度在线网络技术（北京）有限公司 | Testing method and device for driverless automobiles |
| CN106528398A (en) * | 2015-09-15 | 2017-03-22 | 网易（杭州）网络有限公司 | Game software performance visual analysis method |
| CN110059146A (en) * | 2019-04-16 | 2019-07-26 | 珠海金山网络游戏科技有限公司 | A kind of collecting method, calculates equipment and storage medium at server |
| CN110300370A (en) * | 2019-07-02 | 2019-10-01 | 广州纳斯威尔信息技术有限公司 | A kind of reconstruction wifi fingerprint map indoor orientation method |
| CN111122175A (en) * | 2020-01-02 | 2020-05-08 | 北京百度网讯科技有限公司 | Method and device for testing automatic driving system |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109032102B (en) * | 2017-06-09 | 2020-12-18 | 百度在线网络技术（北京）有限公司 | Unmanned vehicle testing method, device, equipment and storage medium |
| CN113642633B (en) * | 2018-06-11 | 2023-06-20 | 百度在线网络技术（北京）有限公司 | Method, device, equipment and medium for classifying driving scene data |
| GB2581861B (en) * | 2018-09-14 | 2022-10-05 | Sino Ic Tech Co Ltd | IC Test Information Management System Based on Industrial Internet |
-
2020
- 2020-06-09 CN CN202010518935.4A patent/CN111858927B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106528398A (en) * | 2015-09-15 | 2017-03-22 | 网易（杭州）网络有限公司 | Game software performance visual analysis method |
| CN105954040A (en) * | 2016-04-22 | 2016-09-21 | 百度在线网络技术（北京）有限公司 | Testing method and device for driverless automobiles |
| CN110059146A (en) * | 2019-04-16 | 2019-07-26 | 珠海金山网络游戏科技有限公司 | A kind of collecting method, calculates equipment and storage medium at server |
| CN110300370A (en) * | 2019-07-02 | 2019-10-01 | 广州纳斯威尔信息技术有限公司 | A kind of reconstruction wifi fingerprint map indoor orientation method |
| CN111122175A (en) * | 2020-01-02 | 2020-05-08 | 北京百度网讯科技有限公司 | Method and device for testing automatic driving system |
Non-Patent Citations (1)
| Title |
|---|
| 张静 ; 董楠 ; .基于运动行为统计图的异常行驶车辆检测.机电产品开发与创新.2012,(03),全文. * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111858927A (en) | 2020-10-30 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107844837B (en) | Method and system for adjusting and optimizing algorithm parameters aiming at machine learning algorithm | |
| CN112764916B (en) | Data acquisition method and device | |
| CN108090032B (en) | Visual interpretation method and device of logistic regression model | |
| CN111858927B (en) | Data testing method and device, electronic equipment and storage medium | |
| CN110929489A (en) | Form generation and form data processing method and system | |
| CN111460384B (en) | Policy evaluation method, device and equipment | |
| CN113360711B (en) | Model training and executing method, device, equipment and medium for video understanding task | |
| CN108171617A (en) | A kind of power grid big data analysis method and device | |
| CN111539347B (en) | Method and device for detecting target | |
| CN112559371B (en) | Automatic driving test method and device and electronic equipment | |
| CN112765452B (en) | Search recommendation method and device and electronic equipment | |
| CN111444438B (en) | Method, device, equipment and storage medium for determining quasi-recall rate of recall strategy | |
| CN111291082B (en) | Data aggregation processing method, device, equipment and storage medium | |
| CN106909454B (en) | Rule processing method and equipment | |
| CN112288483A (en) | Method and device for training model and method and device for generating information | |
| CN108898229B (en) | Method and system for constructing machine learning modeling process | |
| CN116050159A (en) | Simulation scene set generation method, device, equipment and medium | |
| CN111563202B (en) | Resource data processing method, device, electronic equipment and medium | |
| CN111597700B (en) | Signal control algorithm evaluation method and device, electronic equipment and readable storage medium | |
| Xiong et al. | Time irreversibility and intrinsics revealing of series with complex network approach | |
| CN111860572B (en) | Data set distillation method, device, electronic equipment and storage medium | |
| CN113990068A (en) | Traffic data processing method, device, equipment and storage medium | |
| CN113987260A (en) | Video pushing method and device, electronic equipment and storage medium | |
| CN112560987A (en) | Image sample processing method, device, equipment, storage medium and program product | |
| CN112070162A (en) | Multi-class processing task training sample construction method, device and medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| TA01 | Transfer of patent application right | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20211018 Address after: 105 / F, building 1, No. 10, Shangdi 10th Street, Haidian District, Beijing 100085 Applicant after: Apollo Intelligent Technology (Beijing) Co.,Ltd. Address before: 2 / F, baidu building, 10 Shangdi 10th Street, Haidian District, Beijing 100085 Applicant before: BEIJING BAIDU NETCOM SCIENCE AND TECHNOLOGY Co.,Ltd. |
|
| GR01 | Patent grant | ||
| GR01 | Patent grant |