CN116467168A - Data set comparison method, device, equipment and storage medium - Google Patents
Data set comparison method, device, equipment and storage medium Download PDFInfo
- Publication number
- CN116467168A CN116467168A CN202310222046.7A CN202310222046A CN116467168A CN 116467168 A CN116467168 A CN 116467168A CN 202310222046 A CN202310222046 A CN 202310222046A CN 116467168 A CN116467168 A CN 116467168A
- Authority
- CN
- China
- Prior art keywords
- scene
- version
- scene set
- probability value
- preset
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 54
- 238000001514 detection method Methods 0.000 claims description 37
- 230000006870 function Effects 0.000 claims description 14
- 238000004364 calculation method Methods 0.000 claims description 11
- 238000012360 testing method Methods 0.000 description 8
- 238000010586 diagram Methods 0.000 description 6
- 230000001186 cumulative effect Effects 0.000 description 5
- 238000005315 distribution function Methods 0.000 description 5
- 230000035945 sensitivity Effects 0.000 description 5
- 238000012549 training Methods 0.000 description 4
- 238000012545 processing Methods 0.000 description 2
- 238000012795 verification Methods 0.000 description 2
- 238000005056 compaction Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000014509 gene expression Effects 0.000 description 1
- 230000005055 memory storage Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000002085 persistent effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/36—Preventing errors by testing or debugging software
- G06F11/3668—Software testing
- G06F11/3672—Test management
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
- Y02T10/00—Road transport of goods or passengers
- Y02T10/10—Internal combustion engine [ICE] based vehicles
- Y02T10/40—Engine management systems
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Quality & Reliability (AREA)
- Computer Hardware Design (AREA)
- Artificial Intelligence (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- Image Analysis (AREA)
Abstract
The application relates to the field of automatic driving, and discloses a data set comparison method, a device, equipment and a storage medium. The data set comparison method comprises the following steps: acquiring a first version scene set and a second version scene set of an automatic driving scene; detecting the grading index type of the automatic driving scene; calculating a hypothesis probability value of the first version scene set relative to the second version scene set according to the grading index type, wherein the hypothesis probability value is used for representing the probability that a zero hypothesis is established, and the zero hypothesis is that the second version scene is not significantly changed relative to the first version scene set; when the assumption probability value is smaller than a preset first threshold value, the second version scene set is determined to be significantly changed relative to the first version scene set. Compared with the prior art, when comparing two version scene sets with too few scenes, the method can obtain an accurate comparison result between the two version scene sets by calculating the hypothesis probability value and comparing the hypothesis probability value with the preset threshold value.
Description
Technical Field
The present disclosure relates to the field of autopilot, and in particular, to a data set comparison method, apparatus, device, and storage medium.
Background
Different scene sets are used for AB test in the process of automatic driving algorithm iteration, and in the AB test process, different results of good, bad or basically unchanged of single scenes in the scene set can be obtained. By dividing the scene sets of two versions, namely a first version scene set and a second version scene set, for the automatic driving scene in advance, whether the second version scene set is changed significantly relative to the first version scene set is judged by combining the change condition of single scenes in the scene sets, and when the change is determined to be significant, the second version scene set can be considered to be possibly covered to more different scene types, so that the generalization capability of the model can be better after the model is trained by using the new version scene set.
The current method mainly comprises the steps of calculating a scene passing rate difference value between two versions of scene sets and comparing the scene passing rate difference value with a preset threshold value to judge whether the change is obvious, but as the comparison result of the method is easily influenced by the number of the scenes in the scene sets, the detection sensitivity is too high in the automatic driving scene with too few scenes, so that the comparison result obtained by applying the method to the automatic driving scene with too few scenes is inaccurate.
Disclosure of Invention
The application provides a data set comparison method, which can obtain an accurate comparison result between two version scene sets based on comparison of an assumption probability value and a preset threshold value when comparing the two version scene sets with the too small number of scenes.
In a first aspect, the present application provides a data set comparison method, comprising: acquiring a first version scene set and a second version scene set of an automatic driving scene; detecting the grading index type of the automatic driving scene; calculating a hypothesis probability value of the first version scene set relative to the second version scene set according to the grading index type; the assumption probability value is used for representing the probability that a zero assumption is established, wherein the zero assumption is that the second version scene is not significantly changed relative to the first version scene set; when the assumption probability value is smaller than a preset first threshold value, the second version scene set is determined to be significantly changed relative to the first version scene set.
With reference to the first aspect, in certain possible implementation manners of the first aspect, the data set comparison method further includes: when the assumed probability value is not smaller than a preset first threshold value, acquiring the total number of scenes of the first version scene set; when the total scene number is larger than a preset second threshold value, calculating a scene passing rate difference value between the first version scene set and the second version scene set; and when the scene passing rate difference value is larger than a preset third threshold value, determining that the second version scene set is obviously changed relative to the first version scene set.
With reference to the first aspect and the foregoing implementation manners, in some possible implementation manners of the first aspect, when the total number of scenes is greater than a preset second threshold, calculating a scene passing difference value between the first version scene set and the second version scene set includes: when the total number of scenes is larger than a preset second threshold value, determining the scene passing number of the first version scene set and the scene passing number of the second version scene set respectively, wherein the scene passing number is used for counting the scenes detected by the preset scenes in the scene set; calculating a scene number difference between the scene passing number of the first version scene set and the scene passing number of the second version scene set; and carrying out quotient operation based on the scene number difference value and the scene total number to obtain a scene passing rate difference value between the first version scene set and the second version scene set.
With reference to the first aspect, in certain possible implementation manners of the first aspect, detecting a score indicator type of the autopilot scenario includes: acquiring scene detection rules of an automatic driving scene; and determining the grading index type of the automatic driving scene according to the scene detection rule.
With reference to the first aspect, in some possible implementations of the first aspect, calculating, according to the scoring indicator type, a hypothetical probability value of the first version scene set relative to the second version scene set includes: and when the scoring index type is Boolean type, calculating a first assumption probability value of the first version scene set relative to the second version scene set based on a preset first function.
With reference to the first aspect, in some possible implementations of the first aspect, calculating, according to the scoring indicator type, a hypothetical probability value of the first version scene set relative to the second version scene set includes: and when the scoring index type is continuous, calculating a second assumption probability value of the first version scene set relative to the second version scene set based on a preset second function.
With reference to the first aspect, in certain possible implementation manners of the first aspect, the data set comparison method further includes: when it is determined that the second version scene set varies significantly from the first version scene set, then an autopilot model is trained based on the second version scene set.
In a second aspect, an embodiment of the present application provides a data set comparing apparatus, including: the scene set acquisition module is used for acquiring a first version scene set and a second version scene set of the automatic driving scene; the index detection module is used for detecting the grading index type of the automatic driving scene; the hypothesis probability value calculation module is used for calculating a hypothesis probability value of the first version scene set relative to the second version scene set according to the grading index type; the assumption probability value is used for representing the probability that a zero assumption is established, wherein the zero assumption is that the second version scene is not significantly changed relative to the first version scene set; and the saliency judgment module is used for determining that the second version scene set changes remarkably relative to the first version scene set when the assumption probability value is smaller than a preset first threshold value.
With reference to the second aspect, in certain possible implementations of the second aspect, the data set comparing device further includes: the scene number acquisition module is used for acquiring the total number of scenes of the first version scene set when the assumed probability value is not smaller than a preset first threshold value; the difference value calculation module is used for calculating a scene passing rate difference value between the first version scene set and the second version scene set when the total number of scenes is larger than a preset second threshold value; and the auxiliary judging module is used for determining that the second version scene set is obviously changed relative to the first version scene set when the scene passing rate difference value is larger than a preset third threshold value.
With reference to the second aspect and the foregoing implementation manners, in some possible implementation manners of the second aspect, the difference calculating module specifically includes: the statistics unit is used for respectively determining the scene passing number of the first version scene set and the scene passing number of the second version scene set when the total number of scenes is larger than a preset second threshold value, wherein the scene passing number is used for counting the scenes detected by the preset scenes in the scene set; the difference operation unit is used for calculating a scene number difference value between the scene passing number of the first version scene set and the scene passing number of the second version scene set; and the quotient operation unit is used for carrying out quotient operation based on the scene number difference value and the total number of the scenes to obtain a scene passing rate difference value between the first version scene set and the second version scene set.
With reference to the second aspect, in some possible implementation manners of the second aspect, the indicator detection module specifically includes: the rule acquisition unit is used for acquiring scene detection rules of the automatic driving scene; and the type determining unit is used for determining the grading index type of the automatic driving scene according to the scene detection rule.
With reference to the second aspect, in some possible implementations of the second aspect, it is assumed that the probability value calculation module is specifically configured to: and when the scoring index type is Boolean type, calculating a first assumption probability value of the first version scene set relative to the second version scene set based on a preset first function.
With reference to the second aspect, in some possible implementations of the second aspect, it is assumed that the probability value calculation module is specifically configured to: and when the scoring index type is continuous, calculating a second assumption probability value of the first version scene set relative to the second version scene set based on a preset second function.
With reference to the second aspect, in some possible implementations of the second aspect, the data set comparing device further includes a model training module for training the automatic driving model based on the second version scene set when it is determined that the second version scene set changes significantly with respect to the first version scene set.
In a third aspect, the present application provides a data set comparison device comprising: a memory and at least one processor, the memory having instructions stored therein; the at least one processor invokes the instructions in the memory to cause the data set comparison device to perform the steps of the data set comparison method described above.
In a fourth aspect, the present application provides a computer readable storage medium having instructions stored therein which, when run on a computer, cause the computer to perform the steps of the data set comparison method described above.
In the application, a first version scene set and a second version scene set of the automatic driving scene are acquired; detecting the grading index type of the automatic driving scene; calculating a hypothesis probability value of the first version scene set relative to the second version scene set according to the grading index type; when the assumption probability value is smaller than a preset first threshold value, the second version scene set is determined to be significantly changed relative to the first version scene set. According to the invention, the hypothesis probability value of the first version scene set relative to the second version scene set is calculated according to the scoring index type corresponding to the current automatic driving scene and according to the calculation mode corresponding to the scoring index type, so that whether the second version scene set is obviously changed relative to the first version scene set or not is determined based on the comparison of the hypothesis probability value (namely, the probability value that the hypothesis that the second version scene set is not obviously changed relative to the first version scene set) in the hypothesis test is established with the threshold value which is preset and represents the significance level, and further, when two version scene sets with the too small number of scenes are compared, an accurate comparison result can be obtained.
Drawings
FIG. 1 is a flow chart of an embodiment of a first data set comparison method provided in an embodiment of the present application;
FIG. 2 is a flow chart of an embodiment of a second data set comparison method provided in an embodiment of the present application;
FIG. 3 is an embodiment flow chart of a third data set comparison method provided in an embodiment of the present application;
FIG. 4 is an embodiment flow chart of a fourth data set comparison method provided in an embodiment of the present application;
fig. 5 is a schematic structural diagram of a data set comparing device according to an embodiment of the present application;
FIG. 6 is a schematic diagram of another data set comparing device according to an embodiment of the present disclosure;
fig. 7 is a schematic structural diagram of a data set comparing device according to an embodiment of the present application.
Detailed Description
The embodiment of the application provides a data set comparison method, which can obtain an accurate comparison result between two version scene sets based on comparison of an assumed probability value and a preset threshold value when comparing the two version scene sets with too few scenes.
The technical solutions in the embodiments of the present application will be described below with reference to the drawings in the embodiments of the present application. Wherein the terms "first," "second," "third," "fourth," and the like in the description and in the claims of this application and in the above-described figures, if any, are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used may be interchanged where appropriate such that the embodiments described herein may be implemented in other sequences than those illustrated or otherwise described herein. Furthermore, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed or inherent to such process, method, article, or apparatus.
It is to be understood that the execution subject of the present application may be a data set comparing device, and may also be a terminal or a server, which is not limited herein.
For easy understanding, the embodiment of the present application uses a server as an execution body to describe the data set comparison method, and the following describes a specific flow of the embodiment of the present application, please refer to fig. 1, fig. 1 is an embodiment flowchart of a first data set comparison method provided in the embodiment of the present application, which includes:
101. acquiring a first version scene set and a second version scene set of an automatic driving scene;
it will be appreciated that a comparison between two or more versions of the scene data set is required during an AB test of an autopilot scene to determine an optimal version of the scene data set for use in optimizing an autopilot algorithm.
In the application, the first version scene set and the second version scene set are respectively two scene sets of different versions corresponding to the automatic driving scene, and each scene set contains scene images of the automatic driving scene. The automatic driving scene includes, but is not limited to, a cut-in scene, a lane changing scene, a jam scene, etc., which are not limited in the embodiment of the present application.
102. Detecting the grading index type of the automatic driving scene;
it should be appreciated that the server obtains the scene detection rules for the autopilot scene to determine a corresponding scoring indicator type based on the scene detection rules, including, but not limited to, boolean and continuous.
The automatic driving scene is a compaction line scene, the scene detection rule is whether the automatic driving vehicle is pressed to a solid line, if the solid line is pressed down, the automatic driving scene is determined to pass the scene detection, otherwise, the automatic driving scene is determined to not pass the scene detection, and the corresponding scoring index type is Boolean;
for another example, the automatic driving scene is an overtaking scene, the scene detection rule is whether the automatic driving vehicle successfully exceeds the target vehicle, if the overtaking is successful, the automatic driving scene is determined to pass the scene detection, otherwise, the automatic driving scene is determined to not pass the scene detection, and the corresponding scoring index type is also boolean.
The automatic driving scene is an occlusion scene, the scene detection rule is to calculate the similarity between the driving track of the automatic driving vehicle and the driving track of the human driver, so that the similarity is used as a scoring standard, whether the scene detection is passed or not is determined by comparing the similarity with a preset similarity threshold value, and the corresponding scoring index type is continuous;
for another example, the automatic driving scene is a lane change scene, the scene detection rule is to calculate a time difference between the lane change usage time of the automatic driving vehicle and the lane change usage time of the human driver, so that the time difference is used as a scoring standard, and whether the scene detection is passed or not is determined by comparing the time difference with a preset duration threshold, and the corresponding scoring index type is continuous.
It should be noted that the specific data form of the scene detection rule includes, but is not limited to, text and logical expressions.
103. Calculating a hypothesis probability value of the first version scene set relative to the second version scene set according to the grading index type, wherein the hypothesis probability value is used for representing the probability that a zero hypothesis is established, and the zero hypothesis is that the second version scene is not significantly changed relative to the first version scene set;
it should be understood that the hypothesis probability value (P-value) refers to a probability value corresponding to one statistic calculated in the hypothesis test. It represents the probability of occurrence of observed sample data or more extreme sample data. The smaller the hypothesis probability value, the more likely the observed data is not consistent with the original hypothesis, rejecting the original hypothesis. In this embodiment of the present application, the assumed probability value is a probability value that the assumption that the second version scene is not significantly changed with respect to the first version scene set is satisfied, and when the assumed probability value is smaller than a preset threshold, the server may directly negate that the assumption is satisfied, i.e. determine that the second version scene is significantly changed with respect to the first version scene set.
104. When the assumption probability value is smaller than a preset first threshold value, the second version scene set is determined to be significantly changed relative to the first version scene set.
It should be appreciated that there are different expectations of the server's sensitivity to detection of changes in the second version scene set relative to the first version scene set in different business requirements, and the first threshold is not specifically limited by the present application.
It will be appreciated that it is relatively easy to prove that a statistic is less than a maximum and that it is relatively not easy to prove that a statistic is less than a minimum.
In one possible implementation, the first threshold is a minimum probability value, for example 0.05, and the server may reject the assumption that the second version scene set is not significantly changed with respect to the first version scene set is satisfied only if the probability value is less than the minimum probability value, thereby determining that the second version scene set is significantly changed with respect to the first version scene set. In other words, the server cannot sensitively detect whether the second version scene set changes significantly with respect to the first version scene set.
In another possible implementation, the first threshold is a maximum probability value, for example, 0.7, and the server may reject the assumption that the second version scene set is not significantly changed with respect to the first version scene set is satisfied as long as the probability value is less than the maximum probability value, thereby determining that the second version scene set is significantly changed with respect to the first version scene set. In other words, the server may sensitively detect whether the second version scene set changes significantly relative to the first version scene set.
In an alternative embodiment, where only two versions of the autopilot scene set are used for AB testing, the server trains an autopilot model based on the second version scene set after determining that the second version scene set changes significantly relative to the first version scene set, optimizing the autopilot algorithm.
According to the method provided by the embodiment of the application, the assumption probability value of the second version scene set relative to the first version scene set is calculated according to the scoring index type corresponding to the current automatic driving scene and according to the calculation mode corresponding to the scoring index type, so that whether the second version scene set changes significantly relative to the first version scene set is determined based on comparison of the assumption probability value in the assumption test (namely, the probability value that the assumption that the second version scene set is not significantly changed relative to the first version scene set) and the threshold value which is preset to represent the significance level, and further an accurate comparison result can be obtained when two version scene sets with too few scenes are compared.
Referring to fig. 2, fig. 2 is a flowchart of an embodiment of a second data set comparison method provided in an embodiment of the present application, including:
201. acquiring a first version scene set and a second version scene set of an automatic driving scene;
202. detecting the grading index type of the automatic driving scene;
203. calculating a hypothesis probability value of the first version scene set relative to the second version scene set according to the grading index type, wherein the hypothesis probability value is used for representing the probability that a zero hypothesis is established, and the zero hypothesis is that the second version scene is not significantly changed relative to the first version scene set;
204. when the assumed probability value is smaller than a preset first threshold value, determining that the second version scene set is obviously changed relative to the first version scene set;
steps 201 to 204 are similar to the steps 101 to 104, and are not repeated here.
205. When the assumed probability value is not smaller than a preset first threshold value, acquiring the total number of scenes of the first version scene set;
206. when the total scene number is larger than a preset second threshold value, calculating a scene passing rate difference value between the first version scene set and the second version scene set;
it should be appreciated that for scene sets of orders of magnitude larger, the assumption that the probability value is not less than the preset first threshold value does not indicate that the second version scene set does not significantly change from the first version scene set, requiring further verification. Specifically, the server obtains the total number of scenes of the first version scene set (or the second version scene set, the number of scenes being identical) and compares it to a second threshold to determine whether further verification needs to be performed.
The calculating, by the server, the scene passing difference value between the first version scene set and the second version scene set specifically includes: respectively carrying out scene detection on the first version scene set and the second version scene set based on a preset scene detection rule, so as to determine the scene passing number of the first version scene set and the scene passing number of the second version scene set, wherein the scene passing number is used for counting the scenes passing the scene detection in the scene set; calculating a scene number difference between the scene passing number of the first version scene set and the scene passing number of the second version scene set; and carrying out quotient operation based on the scene number difference value and the total scene number, so as to obtain a scene passing rate difference value between the first version scene set and the second version scene set.
It will be appreciated that for scene sets of orders of magnitude larger, the first embodiment described above (based on hypothesis probability values) may be applied to determine whether a change between two version scene sets is significant, or the present embodiment (based on scene passing rate differences) may be applied to determine whether a change between two version scene sets is significant. For the scene set with smaller orders of magnitude, the comparison mode of calculating the scene passing rate difference between the two versions of the scene set and comparing the scene passing rate difference with the preset threshold value to judge whether the change is obvious or not is susceptible to the influence of the number of the scenes in the scene set, so that an inaccurate comparison result is obtained, for example, 60 scenes in the first version of the scene set and 61 scenes in the second version of the scene set, and even if the two version of the scene set and the first version of the scene set are different by only one passing scene, the passing rate difference between the two version of the scene set and the second version of the scene set is 1.67%, so that the change is judged to be obvious, and an error scene set comparison result is obtained.
207. And when the scene passing rate difference value is larger than a preset third threshold value, determining that the second version scene set is obviously changed relative to the first version scene set.
It should be appreciated that, in different service requirements, there are different expectations of the server regarding the detection sensitivity of the second version scene set to the first version scene set change, and therefore the third threshold is not specifically limited in this application. Preferably, the third threshold is 5% (i.e., the detection sensitivity is less sensitive) and 1% (i.e., the detection sensitivity is more sensitive).
Based on the method provided by the embodiment of the application, after the comparison result is not determined by assuming the probability value, the magnitude scale of the scene set can be verified, and when the magnitude of the scene set is smaller, the scene passing rate difference between the two scene sets is calculated to assist in judging whether the second version scene set is obviously changed relative to the first version scene set, so that the accuracy of the comparison result is improved.
Referring to fig. 3, fig. 3 is a flowchart of an embodiment of a third data set comparison method provided in an embodiment of the present application, including:
301. acquiring a first version scene set and a second version scene set of an automatic driving scene;
302. detecting the grading index type of the automatic driving scene;
steps 301 to 302 are similar to the steps 101 to 102, and are not repeated here.
303. When the grading index type of the automatic driving scene is Boolean type, calculating a first assumption probability value of a first version scene set relative to a second version scene set based on a preset first function, wherein the first assumption probability value is used for representing the probability that zero assumption is established, and the zero assumption is that the second version scene is not significantly changed relative to the first version scene set; the method comprises the steps of carrying out a first treatment on the surface of the
Specifically, the first preset function is p-value 1=1-norm cdf (|z|), p-value1 is the first hypothetical probability value, norm cdf (|z|) is a normal cumulative distribution function calculated on the absolute value of z (Cumulative Normal Distribution Function),n is the total number of scenes in the first version scene set or the second version scene set, p1 is the scene passing rate of the first version scene set, p2 is the scene passing rate of the second version scene set, and the scene passing rate is equal to the number of scenes detected by each scene passing scene set divided by the total number of scenes in the scene set N>The specific definition of the normal cumulative distribution function is common general knowledge, and thus will not be described in detail herein.
304. When the first hypothesis probability value is smaller than a preset first threshold value, the second version scene set is determined to be significantly changed relative to the first version scene set.
Step 304 is similar to the above-mentioned step 104, and is not repeated here.
According to the method provided by the embodiment of the application, the first assumption probability value of the first version scene set relative to the second version scene set is calculated through the first function, so that the probability value of the assumption that the first version scene set is not significant in change relative to the second version scene set under the automatic driving scene with the grading index type of Boolean is accurately calculated, and whether the first version scene set is significant in change relative to the second version scene set is accurately judged according to the comparison of the first probability value of the assumption to be true with the preset threshold value, and an accurate comparison result between the two version scene sets is obtained.
Referring to fig. 4, fig. 4 is a flowchart of an embodiment of a fourth data set comparison method provided in an embodiment of the present application, including:
401. acquiring a first version scene set and a second version scene set of an automatic driving scene;
402. detecting the grading index type of the automatic driving scene;
403. when the grading index type of the automatic driving scene is continuous, calculating a second assumption probability value of the first version scene set relative to the second version scene set based on a preset second function, wherein the second assumption probability value is used for representing the probability that zero assumption is established, and the zero assumption is that the second version scene is not significantly changed relative to the first version scene set;
specifically, the second preset function is p-value 2=2 (1-norm cdf (|s|), p-value2 is the second hypothetical probability value, norm cdf (|s|) is a normal cumulative distribution function calculated on the absolute value of s (Cumulative Normal Distribution Function),n is the total number of scenes in the first version scene set or the second version scene set, alpha is the average value of the scoring differences between the same sub-scenes in the first version scene set and the second version scene set, sigma is the standard deviation of the scoring differences between the same sub-scenes in the first version scene set and the second version scene set, by taking the above-mentioned stopover scenes as an example, the similarity between the driving track of each scene in the scene set and the driving track of the face driver is used as the scoring standard, and the scoring sequence of the first version scene set is [ ascore_1, ascore_2 … … ascore_N]The scoring sequence of the second version scene set is [ Bscore_1, bscore_2 … … Bscore_N]Correspondingly, the scoring difference sequences of the two are [ Ascore_1-Bscore_1, ascore_2-Bscore_2 … … Ascore_N-Bscore_N ]]Then
Wherein the digital sequence number i is used to distinguish between different sub-scenes in the stoppered scene.
404. When the second hypothesis probability value is smaller than a preset first threshold value, the second version scene set is determined to be significantly changed relative to the first version scene set.
Step 404 is similar to the above-mentioned step 104, and detailed description thereof is omitted herein.
Based on the method provided by the embodiment of the application, the second assumption probability value of the first version scene set relative to the second version scene set is calculated through the second function, so that the probability value of the assumption that the first version scene set is not significant in change relative to the second version scene set under the automatic driving scene with the Boolean type of the grading index is accurately calculated, and whether the first version scene set is significant in change relative to the second version scene set is accurately judged by comparing the probability value of the assumption with a preset threshold value according to the probability value of the assumption, and an accurate comparison result between the two version scene sets is obtained.
The data set comparing method in the embodiment of the present application is described above, and the data set comparing device in the embodiment of the present application is described below, referring to fig. 5, fig. 5 is a schematic structural diagram of the data set comparing device provided in the embodiment of the present application, including: a scene set acquisition module 501, configured to acquire a first version scene set and a second version scene set of an autopilot scene; the index detection module 502 is configured to detect a score index type of an autopilot scene; a hypothesis probability value calculation module 503, configured to calculate a hypothesis probability value of the first version scene set relative to the second version scene set according to the scoring indicator type, where the hypothesis probability value is used to characterize a probability that a null hypothesis is established, and the null hypothesis is that the second version scene is not significantly changed relative to the first version scene set; a saliency determination module 504, configured to determine that the second version scene set changes significantly with respect to the first version scene set when the probability value is assumed to be smaller than a preset first threshold.
According to the device provided by the embodiment of the application, the assumption probability value of the first version scene set relative to the second version scene set is calculated according to the scoring index type corresponding to the current automatic driving scene and according to the calculation mode corresponding to the scoring index type, so that whether the second version scene set changes significantly relative to the first version scene set is determined based on comparison of the assumption probability value in the assumption test (namely, the probability value that the assumption that the second version scene set is not significantly changed relative to the first version scene set) and the threshold value which is preset to represent the significance level, and further, when two version scene sets with too few scenes are compared, an accurate comparison result can be obtained.
Referring to fig. 6, fig. 6 is a schematic structural diagram of another data set comparing apparatus according to an embodiment of the present application, including: a scene set acquisition module 501, configured to acquire a first version scene set and a second version scene set of an autopilot scene; the index detection module 502 is configured to detect a score index type of an autopilot scene; a hypothesis probability value calculation module 503, configured to calculate a hypothesis probability value of the first version scene set relative to the second version scene set according to the scoring indicator type, where the hypothesis probability value is used to characterize a probability that a null hypothesis is established, and the null hypothesis is that the second version scene is not significantly changed relative to the first version scene set; a saliency determination module 504, configured to determine that the second version scene set changes significantly with respect to the first version scene set when the probability value is assumed to be smaller than a preset first threshold.
In a possible embodiment, the data set comparing device further comprises: a scene number obtaining module 505, configured to obtain a total number of scenes in the first version scene set when the probability value is not less than a preset first threshold; the difference calculating module 506 is configured to calculate a scene passing rate difference between the first version scene set and the second version scene set when the total number of scenes is greater than a preset second threshold; the auxiliary determination module 507 is configured to determine that the second version scene set changes significantly with respect to the first version scene set when the scene passing rate difference is greater than a preset third threshold.
In a possible embodiment, the data set comparison device further comprises a model training module 508 for training the automatic driving model based on the second version scene set when it is determined that the second version scene set changes significantly with respect to the first version scene set.
In one possible implementation, the index detection module 502 specifically includes: a rule acquisition unit 5021 for acquiring a scene detection rule of an automatic driving scene; the type determining unit 5022 is configured to determine a score indicator type of the automatic driving scene according to the scene detection rule.
In one possible implementation, it is assumed that the probability value calculation module 503 is specifically configured to: when the scoring index type is Boolean, calculating a first assumption probability value of the first version scene set relative to the second version scene set based on a preset first function; and when the scoring index type is continuous, calculating a second assumption probability value of the first version scene set relative to the second version scene set based on a preset second function.
In one possible implementation, the difference calculating module 506 specifically includes: a statistics unit 5061, configured to determine, when the total number of scenes is greater than a preset second threshold, a scene passing number of the first version scene set and a scene passing number of the second version scene set, where the scene passing number is used to count scenes in the scene set that pass through a preset scene detection; a difference operation unit 5062 for calculating a scene number difference between the scene passing number of the first version scene set and the scene passing number of the second version scene set; the quotient operation unit 5063 is configured to perform a quotient operation based on the scene number difference and the total number of scenes, so as to obtain a scene passing rate difference between the first version scene set and the second version scene set.
The technical effects of the device provided by the embodiments of the present application are similar to those of the above embodiments, and are not repeated here.
The data set comparing device in the embodiment of the present application is described in detail above in fig. 5 to 6 from the point of view of the modularized functional entity, and the data set comparing apparatus in the embodiment of the present application is described in detail below from the point of view of hardware processing.
Fig. 7 is a schematic diagram of a data set comparing device according to an embodiment of the present application, where the data set comparing device 700 may have a relatively large difference due to different configurations or performances, and may include one or more processors 710 (e.g., one or more processors) and a memory 720, and one or more storage media 730 storing application programs 733 or data 732. Wherein memory 720 and storage medium 730 may be transitory or persistent. The program stored in the storage medium 730 may include one or more modules (not shown), each of which may include a series of instruction operations in the data set comparison device 700. Still further, the processor 710 may be configured to communicate with the storage medium 730 and execute a series of instruction operations in the storage medium 730 on the data set comparison device 700.
The data set comparison device 700 may also include one or more power supplies 740, one or more wired or wireless network interfaces 750, one or more input/output interfaces 760, and/or one or more operating systems 731, such as Windows Serve, mac OS X, unix, linux, freeBSD, and the like. It will be appreciated by those skilled in the art that the data set comparison device structure shown in fig. 7 is not limiting of the data set comparison device and may include more or fewer components than shown, or may combine certain components, or a different arrangement of components.
The present application also provides a data set comparing device, where the computer device includes a memory and a processor, where the memory stores computer readable instructions that, when executed by the processor, cause the processor to perform the steps of the data set comparing method in the foregoing embodiments.
The present application also provides a computer readable storage medium, which may be a non-volatile computer readable storage medium, or may be a volatile computer readable storage medium, having stored therein instructions which, when executed on a computer, cause the computer to perform the steps of the dataset comparison method.
It will be clear to those skilled in the art that, for convenience and brevity of description, specific working procedures of the above-described systems, apparatuses and units may refer to corresponding procedures in the foregoing method embodiments, which are not repeated herein.
The integrated units, if implemented in the form of software functional units and sold or used as stand-alone products, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present application may be embodied in essence or a part contributing to the prior art or all or part of the technical solution in the form of a software product stored in a storage medium, including several instructions to cause a computer device (which may be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in the embodiments of the present application. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a read-only memory (ROM), a random access memory (random access memory, RAM), a magnetic disk, or an optical disk, or other various media capable of storing program codes.
The subject application is operational with numerous general purpose or special purpose computer system environments or configurations. For example: personal computers, server computers, hand-held or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments that include any of the above systems or devices, and the like. The application may be described in the general context of computer-executable instructions, such as program modules, being executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform particular tasks or implement particular abstract data types. The application may also be practiced in distributed computing environments where tasks are performed by remote processing devices that are linked through a communications network. In a distributed computing environment, program modules may be located in both local and remote computer storage media including memory storage devices.
The above embodiments are merely for illustrating the technical solution of the present application, and not for limiting the same; although the present application has been described in detail with reference to the foregoing embodiments, it should be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit and scope of the corresponding technical solutions.
Claims (10)
1. A data set comparison method, comprising:
acquiring a first version scene set and a second version scene set of an automatic driving scene;
detecting the grading index type of the automatic driving scene;
calculating a hypothesis probability value of the first version scene set relative to the second version scene set according to the scoring index type; wherein the hypothesis probability value is used to characterize a probability that a null hypothesis is established, the null hypothesis being that the second version scene does not change significantly relative to the first version scene set;
and when the hypothesis probability value is smaller than a preset first threshold value, determining that the second version scene set is significantly changed relative to the first version scene set.
2. The data set comparison method of claim 1, wherein the method further comprises:
when the assumption probability value is not smaller than a preset first threshold value, acquiring the total number of scenes of the first version scene set;
when the total number of scenes is larger than a preset second threshold value, calculating a scene passing rate difference value between the first version scene set and the second version scene set;
and when the scene passing rate difference value is larger than a preset third threshold value, determining that the second version scene set is obviously changed relative to the first version scene set.
3. The data set comparison method according to claim 2, wherein calculating a scene passing difference value between the first version scene set and the second version scene set when the total number of scenes is greater than a preset second threshold value comprises:
when the total number of scenes is larger than a preset second threshold, determining the scene passing number of the first version scene set and the scene passing number of the second version scene set respectively, wherein the scene passing number is used for counting the scenes passing through preset scene detection in the scene set;
calculating a scene number difference between the scene passing number of the first version scene set and the scene passing number of the second version scene set;
and carrying out quotient operation based on the scene number difference value and the scene total number to obtain a scene passing rate difference value between the first version scene set and the second version scene set.
4. The data set comparison method according to claim 1, wherein the detecting the score index type of the automated driving scene includes:
acquiring a scene detection rule of the automatic driving scene;
and determining the grading index type of the automatic driving scene according to the scene detection rule.
5. The data set comparison method of claim 1, wherein calculating a hypothetical probability value for the first version scene set relative to the second version scene set based on the scoring indicator type comprises:
and when the scoring index type is Boolean type, calculating a first assumption probability value of the first version scene set relative to the second version scene set based on a preset first function.
6. The data set comparison method of claim 1, wherein calculating a hypothetical probability value for the first version scene set relative to the second version scene set based on the scoring indicator type comprises:
and when the scoring index type is continuous, calculating a second assumption probability value of the first version scene set relative to the second version scene set based on a preset second function.
7. The data set comparison method according to any one of claims 1-6, wherein the method further comprises:
upon determining that the second version scene set varies significantly from the first version scene set, an autopilot model is trained based on the second version scene set.
8. A data set comparison apparatus, comprising:
the scene set acquisition module is used for acquiring a first version scene set and a second version scene set of the automatic driving scene;
the index detection module is used for detecting the grading index type of the automatic driving scene;
a hypothesis probability value calculation module, configured to calculate a hypothesis probability value of the first version scene set relative to the second version scene set according to the scoring indicator type; wherein the hypothesis probability value is used to characterize a probability that a null hypothesis is established, the null hypothesis being that the second version scene does not change significantly relative to the first version scene set;
and the significance judging module is used for determining that the second version scene set changes significantly relative to the first version scene set when the hypothesis probability value is smaller than a preset first threshold value.
9. A data set comparison device, the data set comparison device comprising: a memory and at least one processor, the memory having instructions stored therein;
the at least one processor invokes the instructions in the memory to cause the data set comparison device to perform the respective steps of the data set comparison method of any of claims 1-7.
10. A computer readable storage medium having instructions stored thereon, which when executed by a processor, perform the steps of the data set comparison method according to any of claims 1-7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310222046.7A CN116467168A (en) | 2023-03-08 | 2023-03-08 | Data set comparison method, device, equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310222046.7A CN116467168A (en) | 2023-03-08 | 2023-03-08 | Data set comparison method, device, equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116467168A true CN116467168A (en) | 2023-07-21 |
Family
ID=87184811
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310222046.7A Pending CN116467168A (en) | 2023-03-08 | 2023-03-08 | Data set comparison method, device, equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116467168A (en) |
-
2023
- 2023-03-08 CN CN202310222046.7A patent/CN116467168A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109934268B (en) | Abnormal transaction detection method and system | |
| US11461584B2 (en) | Discrimination device and machine learning method | |
| CN104216350A (en) | System and method for analyzing sensed data | |
| CN107203467A (en) | The reference test method and device of supervised learning algorithm under a kind of distributed environment | |
| CN109191133B (en) | Payment channel selection method and terminal equipment | |
| CN113535454B (en) | Log data anomaly detection method and device | |
| CN114764768A (en) | Defect detection and classification method and device, electronic equipment and storage medium | |
| US11037016B2 (en) | Feature determination apparatus and method adapted to multiple object sizes | |
| CN109766958B (en) | A kind of data preprocessing method and device for data fusion | |
| CN114595765A (en) | Data processing method and device, electronic equipment and storage medium | |
| CN117174316A (en) | Personnel state monitoring method and device, electronic equipment and storage medium | |
| CN1971619A (en) | Testing algorithm of image border based on cellular automata | |
| CN117786565A (en) | Method and device for determining semiconductor abnormal chip and electronic equipment | |
| CN116467168A (en) | Data set comparison method, device, equipment and storage medium | |
| CN109409091B (en) | Method, device and equipment for detecting Web page and computer storage medium | |
| CN111752734A (en) | Abnormal data classification method, abnormal data analysis method, abnormal data classification device and abnormal data analysis device, and storage medium | |
| CN108763092B (en) | Code defect detection method and device based on cross validation | |
| CN108985350B (en) | Method and device for recognizing blurred image based on gradient amplitude sparse characteristic information, computing equipment and storage medium | |
| CN116385007A (en) | Data detection method, device, computer equipment and computer readable storage medium | |
| CN112801987B (en) | Mobile phone part abnormity detection method and equipment | |
| CN113051127B (en) | Abnormal power consumption detection method and device, electronic equipment and storage medium | |
| CN111553383A (en) | Data risk detection method, device and equipment | |
| CN110162806B (en) | Simulation error evaluation method and system | |
| CN109583511A (en) | Speed fusion method and device | |
| CN109583590A (en) | Data processing method and data processing equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |