CN104850497B - The verification method and device of test result data - Google Patents
The verification method and device of test result data Download PDFInfo
- Publication number
- CN104850497B CN104850497B CN201510251237.1A CN201510251237A CN104850497B CN 104850497 B CN104850497 B CN 104850497B CN 201510251237 A CN201510251237 A CN 201510251237A CN 104850497 B CN104850497 B CN 104850497B
- Authority
- CN
- China
- Prior art keywords
- verification
- data items
- test cases
- probability
- data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000012795 verification Methods 0.000 title claims abstract description 253
- 238000012360 testing method Methods 0.000 title claims abstract description 218
- 238000000034 method Methods 0.000 title claims abstract description 51
- 230000003111 delayed effect Effects 0.000 claims description 30
- 238000004364 calculation method Methods 0.000 claims description 9
- 238000006467 substitution reaction Methods 0.000 description 3
- 238000010586 diagram Methods 0.000 description 2
- 238000013524 data verification Methods 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 238000003672 processing method Methods 0.000 description 1
Landscapes
- Storage Device Security (AREA)
Abstract
The present invention provides a kind of verification method and device of test result data.Methods described includes：Multiple test cases are obtained, the test case includes the multiple data item to be verified；The probability occurred according to the multiple data item in the multiple test case is respectively the data item setting authentication policy, during the test case is performed, to be verified according to the authentication policy of setting to the data item.The verification method and device for the test result data that the present invention is provided, corresponding authentication policy is automatically set for the data item according to the probability that multiple data item occur in multiple test cases for product test, so that the proving time is most short under this authentication policy establishing method, the time cost of the checking to test result data is reduced, verification efficiency is improved.
Description
Technical Field
The invention relates to the technical field of computers, in particular to a method and a device for verifying test result data.
Background
For a computer product, such as the API of an LBS platform, there are a large number of input parameters. When testing a product, in order to achieve the coverage rate of the test, a tester can combine a large number of input parameters by means of orthogonal interaction and the like to generate hundreds, thousands or even tens of thousands of test cases. Each generated test case includes a plurality of input parameters, an execution condition, and an expected value of an output result corresponding to each input parameter. For the same product, when the input parameters are different, the corresponding output results are different, that is, the output results of different test cases are different. For example: the input parameters (such as keys in key value pairs) of a certain test case are PA1, PB1 and PC1, and the corresponding output results are RA, RB1, RB11, RC1 and RC 11. The input parameters of another test case are PA2, PB2 and PC2, and the corresponding output results are RA, RB2, RB21, RC2 and RC 21.
In the prior art, a tester sets a verification policy for an expected value of each output result of each test case, and verifies the expected value of each output result according to the set verification policy to determine whether a test on a product passes or not.
The existing verification process for the test result data at least has the following problems: when many test cases exist, because a tester needs to set a verification policy for the expected value of each output result of each test case and verify the expected value of each output result according to the set verification policy, the input time cost is high, and the verification efficiency is low.
Disclosure of Invention
The invention aims to provide a method and a device for verifying test result data, which can reduce the time cost for verifying the test result data and improve the verification efficiency.
According to an aspect of the present invention, the present invention provides a method for verifying test result data, including: obtaining a plurality of test cases, wherein the test cases comprise a plurality of data items to be verified; and respectively setting verification strategies for the data items according to the probability of the plurality of data items appearing in the plurality of test cases, so that the data items are verified according to the set verification strategies in the process of executing the test cases.
Further, the verification policies include an active load verification policy and a delayed load verification policy.
Preferably, the processing of setting the verification policies for the data items according to the probabilities of the data items appearing in the test cases respectively includes: determining a benchmark probability which can make the verification time required by the data items in the test cases shortest according to the probability of the data items in the test cases; and respectively setting the verification strategies for the data items according to the determined reference probabilities.
Preferably, the process of setting the verification policies for the data items respectively according to the determined reference probabilities includes: if the probability of the data item appearing in the plurality of test cases is larger than the reference probability, setting the verification strategy corresponding to the data item as the active loading verification strategy; and if the probability of the data item appearing in the plurality of test cases is less than or equal to the reference probability, setting the verification strategy corresponding to the data item as the delayed loading verification strategy.
Further, the probability of the data item appearing in the test cases is the percentage of the number of the test cases in which the data item appears in the test cases to the total number of the test cases.
Optionally, the method further comprises: and performing deduplication processing on the plurality of data items in the plurality of test cases.
Preferably, the process of determining a reference probability that the required verification time of the data items in the test cases can be minimized according to the probability of the data items appearing in the test cases comprises: setting the number of the data items corresponding to the active loading verification strategy as K; determining that K data items with the highest probability of occurrence in the test cases in the plurality of data items are the data items corresponding to the active loading verification strategy according to the K, and the data items except the K data items in the plurality of data items are the data items corresponding to the delayed loading verification strategy; calculating first average verification time according to the K and the probability of the data items corresponding to the active loading verification strategy appearing in the test cases; calculating second average verification time according to the K and the probability of the data items corresponding to the delayed loading verification strategy appearing in the plurality of test cases; calculating a sum of the first average verification time and the second average verification time; and determining the reference probability according to the value of the K when the sum of the first average verification time and the second average verification time is minimum.
According to another aspect of the present invention, there is provided a verification apparatus for test result data, including: the device comprises a test case acquisition unit, a verification unit and a verification unit, wherein the test case acquisition unit is used for acquiring a plurality of test cases, and the test cases comprise a plurality of data items to be verified; and the verification strategy setting unit is used for respectively setting verification strategies for the data items according to the probability of the plurality of data items appearing in the plurality of test cases so as to verify the data items according to the set verification strategies in the process of executing the test cases.
Further, the verification policies include an active load verification policy and a delayed load verification policy.
Preferably, the verification policy setting unit includes: and the benchmark probability determining module is used for determining the benchmark probability which can make the verification time required by the data items in the test cases shortest according to the probability of the data items in the test cases. And the verification strategy setting module is used for respectively setting the verification strategies for the data items according to the determined reference probabilities.
Preferably, the verification policy setting module includes: the first setting submodule is used for setting the verification strategy corresponding to the data item as the active loading verification strategy if the probability of the data item appearing in the plurality of test cases is greater than the reference probability; and a second setting submodule, configured to set the verification policy corresponding to the data item as the delayed loading verification policy if the probability of the data item appearing in the plurality of test cases is less than or equal to the reference probability.
Further, the probability of the data item appearing in the test cases is the percentage of the number of the test cases in which the data item appears in the test cases to the total number of the test cases.
Optionally, the apparatus further comprises: and the deduplication unit is used for performing deduplication processing on the plurality of data items in the plurality of test cases.
Preferably, the reference probability determination module includes: the third setting submodule is used for setting the number of the data items corresponding to the active loading verification strategy to be K; a first determining submodule, configured to determine, according to the K, that K data items, which have a highest probability of appearing in the test cases, in the multiple data items are the data items corresponding to the active loading verification policy, and the data items, except the K data items, in the multiple data items are the data items corresponding to the delayed loading verification policy; the first calculation submodule is used for calculating first average verification time according to the K and the probability of the data items corresponding to the active loading verification strategy appearing in the test cases; the second calculation submodule is used for calculating second average verification time according to the K and the probability of the data items corresponding to the delayed loading verification strategy appearing in the test cases; a third calculation submodule for calculating a sum of the first average verification time and the second average verification time; a second determining submodule, configured to determine the reference probability according to the value of K when a sum of the first average verification time and the second average verification time is minimum.
According to the verification method and the verification device for the test result data, provided by the invention, the corresponding verification strategy is automatically set for the data items according to the occurrence probability of the data items in a plurality of test cases for product testing, so that the verification time under the verification strategy setting method is shortest, the time cost for verifying the test result data is reduced, and the verification efficiency is improved.
Drawings
FIG. 1 is a flowchart illustrating a verification method of test result data according to an exemplary embodiment of the present invention;
fig. 2 is a schematic diagram showing the structure of a verification apparatus for test result data according to an exemplary embodiment of the present invention.
Detailed Description
The basic concept of the invention is as follows: according to the probability of occurrence of a plurality of data items in a plurality of test cases for product testing, corresponding verification strategies are automatically set for the data items, so that the verification time under the verification strategy setting method is shortest, the time cost of verification of test result data is reduced, and the verification efficiency is improved.
The following describes in detail a verification method and apparatus for test result data according to an exemplary embodiment of the present invention with reference to the accompanying drawings.
Example one
Fig. 1 is a flowchart illustrating a verification method of test result data according to an exemplary embodiment of the present invention.
Referring to fig. 1, the method for verifying test result data of the present embodiment specifically includes:
in step S110, a plurality of test cases are obtained, where the test cases include a plurality of data items to be verified.
Specifically, a plurality of test cases of a product to be tested are obtained, each test case includes an input parameter, an execution condition, and an expected value of test result data, and the expected value of the test result data includes a data item to be verified and an expected value corresponding to the data item. For example: the expected values of the test result data of 3 test samples caseA, caseB and CaseC of the product to be tested are as follows:
expected value of test result data in test case CaseA:
expected value of test result data in test case caseB:
expected value of test result data in test case CaseC:
taking test case CaseA as an example, wherein Q1, Q2, Q3, a1, a2, a21, a22, and a23 are data items to be verified (i.e., keys in the key value pair), and Q1.value, Q2.value, Q3.value, a1.value, a21.value, a22.value, and a23.value are expected values corresponding to the data items (i.e., expected values of key values to keys).
In step S120, a verification policy is respectively set for the data items according to the probability of the plurality of data items appearing in the plurality of test cases, so that the data items are verified according to the set verification policy in the process of executing the test cases.
Specifically, all data items in the multiple test cases may be subjected to deduplication processing, and a data item set is constructed, where the data item set includes all data items appearing in the multiple test cases, a probability of each data item appearing in the multiple test cases, and a comparison policy corresponding to each data item. The probability of the data item appearing in the test cases, namely the percentage of the number of the test cases appearing in the data item in the test cases to the total number of the test cases. Comparison strategies include congruent, a certain precision range (e.g., both 1.00001 and 1.00002 may be considered verification passes), greater than, less than, equal to, regular matches, or verification of time formats, etc.
For example: constructing a data item set according to the 3 test cases CaseA, CaseB and CaseC of the product to be tested, wherein the constructed data item set is as follows:
with "a 1: 33.3%, exact match "is taken as an example, where" a1 "is a data item," 33.3% "is a probability that the data item a1 occurs in 3 test cases (the number of test cases in which the data item a1 occurs in the 3 test cases is 100%/the total number of test cases is 1% 100%/3 is 33.3%), and" exact match "is an alignment policy corresponding to the data item a1.
Further, the verification policy in this step may include an active loading verification policy and a delayed loading verification policy. Wherein,
the active loading verification strategy (comparison strategy-expected value-actual value) is that a comparison strategy corresponding to a data item is found in the data item set, the data item is searched in a plurality of test cases, the expected value corresponding to the data item in the plurality of test cases is loaded, the expected value corresponding to the data item in each test case and the actual value corresponding to the data item (actual value of test result data after the test case is executed) are compared according to the comparison strategy, if the comparison result meets the comparison strategy, the verification on the test item in the test case is passed, and if the comparison result does not meet the comparison strategy, the verification on the test item in the test case is not passed.
The delayed loading verification strategy (expected value-comparison strategy-actual value) is that according to data items sequentially appearing in a plurality of test cases, expected values corresponding to the data items in the plurality of test cases are loaded, a comparison strategy corresponding to the data items is searched in the data item set, the expected values corresponding to the data items and the actual values corresponding to the data items (actual values of test result data after the test cases are executed) are compared according to the comparison strategy, if the comparison result meets the comparison strategy, verification on the test items in the test cases is passed, and if the comparison result does not meet the comparison strategy, verification on the test items in the test cases is not passed.
Assuming that the number of test cases is M, the number of data items in the data item set is N, and the number of expected values corresponding to the data items in the test cases (or the number of data items in the test cases) is N (N)>N), the number of expected values corresponding to the data items in the jth (j is more than or equal to 1 and less than or equal to M) test cases (or the number of the data items in the test cases) is Nj(N≥nj) And is andthe verification time complexity required to employ either the active load verification strategy or the delayed load verification strategy alone is O (M × N). When N is much larger than NjIn this case, a large time loss is caused by matching between the comparison policy corresponding to the data item and the expected value corresponding to the data item.
In order to reduce the verification time and improve the verification efficiency, the active loading verification strategy can be set according to the probability of each data item appearing in a plurality of test cases, and the delayed loading verification strategy can be set for each data item. The specific setting process may include:
and determining the benchmark probability which can minimize the verification time required by the plurality of data items in the plurality of test cases according to the probability of the plurality of data items in the plurality of test cases. And respectively setting the adopted verification strategy for each data item according to the determined reference probability: and if the probability of the data items appearing in the plurality of test cases is greater than the reference probability, setting the verification strategy corresponding to the data items as an active loading verification strategy. And if the probability of the data item appearing in the plurality of test cases is less than or equal to the reference probability, setting the verification strategy corresponding to the data item as a delayed loading verification strategy.
Specifically, the data item S is seti(i ═ 1, 2, …, N) probability P (S) of occurrence in M test casesi)＝Count(Si) /M, wherein Count (S)i) For the occurrence of data item S in M test casesiThe reference probability is R, the number of data items adopting the active loading verification strategy is K, namely N data items Si(i-1, 2, …, N) satisfies P (S)i)>The number of data items of R is K. K data items (i.e. N data items S) using active load verification policyiProbability P (S) of occurrence in M test cases in (i ═ 1, 2, …, N)i) Maximum K data items) Si(i ═ 1, 2, …, K) required first average verification time T1(K) Comprises the following steps:
wherein,
N-K data items (namely N-K data items except the K data items in the N data items) S adopting delayed loading verification strategyi(i ═ K +1, K +2, …, N) required second average verification time T2(K) Comprises the following steps:
the average verification time T (K) required for N data items is the first average verification time T1(K) And a second average verification time T2(K) Sum T1(K)+T2(K) Respectively calculating the value of T (K) when the value of K traverses from 1 to N-1, and determining the minimum min (T) of T (K)1(K) +T2(K) ) value of K, and based on N data items Si(i-1, 2, …, N) satisfies P (S)i)>The number of data items of R is K to determine the value of the reference probability R, and N data items Si(i-1, 2, …, N) satisfies P (S)i)>K data items S of Ri(i ═ 1, 2, …, K) using an active load verification policy (i.e. N data items S)iProbability P (S) of occurrence in M test cases in (i ═ 1, 2, …, N)i) Maximum K data items Si(i-1, 2, …, K) using an active load verification policy), N-K data items S of the N data items other than the K data itemsi(i-K +1, K +2, …, N) employs a lazy-loading verification strategy.
Taking the product to be tested including the above 3 test cases, namely, CaseA, CaseB and CaseC as an example, the following describes in detail the setting process of which verification policy is adopted for the data item and the process of verifying the data item by adopting the corresponding verification policy.
The number M of test cases is 3, the number N of data items in the data item set is 15, the number N of expected values corresponding to the data items in the test cases is 26, and the data item S is seti(i ═ 1, 2, …, 15) probability P (S) of occurrence in 3 test casesi) With a base probability of R, usingIf the number of data items of the active loading verification strategy is K, then K data items (i.e. 15 data items S) of the active loading verification strategy are adoptediProbability P (S) of occurrence in 3 test cases in (i ═ 1, 2, …, 15)i) Maximum K data items) Si(i ═ 1, 2, …, K) required first average verification time T1(K) Comprises the following steps:
15-K data items (i.e. 15-K data items of the 15 data items other than the K data items) S adopting a delayed loading verification strategyi(i ═ K +1, K +2, …, 15) required second average verification time T2(K) Comprises the following steps:
the average verification time T (k) T required to calculate 15 data items1(K)+T2(K)：
When K is 1, adopting data item S of active loading verification strategyi(i-1) is a data item Q1 (or Q1, Q3), and the data item S adopts the delayed loading verification policyi(i-2, 3, …, 15) are 14 data items other than Q1 (or Q1, Q3), then the average verification time T (1) required for 15 data items when K-1:
p (S) in data item set1)＝P(Q1)＝100％、P(S2)＝P(Q2)＝100％、…、 P(S15) Substitution of P (C2) to 33.3%In the above equation, the average verification time T (1) required to obtain 15 data items when K is 1 is calculated as 61.83.
Similarly, the average verification time T (2) 58.67 required for 15 data items when K is 2, the average verification time T (3) 55.5 required for 15 data items when K is 3, the average verification time T (4) 56.28 required for 15 data items when K is 4, the average verification time T (5) 57.06 required for 15 data items when K is 5, the average verification time T (6) 57.83 required for 15 data items when K is 6, the average verification time T (7) 58.61 required for 15 data items when K is 7, the average verification time T (8) 59.39 required for 15 data items when K is 8, the average verification time T (9) 64.11 required for 15 data items when K is 9, the average verification time T (10) required for 15 data items when K is 10, and the average verification time T (29) 15 when K is 15, the average verification time T (11) required for 15 data items when K is 10, when K is 12, the average verification time T (12) required for 15 data items is 78.28, when K is 13, the average verification time T (13) required for 15 data items is 83, and when K is 14, the average verification time T (14) required for 15 data items is 87.72.
By comparing the average verification time T (K) with the average verification time T (K), T (3) is the smallest, where K is 3, i.e. 15 data items Si(i-1, 2, …, 15) satisfies P (S)i)>The number of the data items of R is 3, and the reference probability R meets 66.67 percent<R<100%, and 15 data items Si(i-1, 2, …, 15) satisfies P (S)i)>The 3 data items of R, Q1, Q2, Q3, employ an active load verification policy, and the remaining 12 data items B1, B2, B21, B22, B23, a1, a2, a21, a22, a23, C2, C2 employ a deferred load verification policy.
And (3) active loading verification process: finding the comparison strategy corresponding to the data item Q1 in the data item set as 'exact match', finding the data item Q1 in 3 test cases CaseA, CaseB and CaseC, loading expected values Q1 corresponding to the data item Q1 in the 3 test cases CaseA, CaseB and CaseC, comparing the expected values Q1 corresponding to the data item Q1 with actual values (the actual values of the data item Q1 in the test result data after the test cases are executed) corresponding to the data item Q1 according to the comparison strategy 'exact match', if 'exact match', the verification of the data item Q1 is passed, otherwise, the verification of the data item Q1 is not passed. Similarly, the verification process for the data items Q2, Q3 is completed in sequence in accordance with the above process.
And (3) delaying the loading verification process: according to the appearance sequence of the data items B1, B2, B21, B22, B23, A1, A2, A21, A22, A23, C2 and C2 in 3 test cases, CaseA, CaseB and CaseC, finding the data item A1 in 3 test cases, CaseA, CaseB and CaseC, loading the expected value a1 corresponding to the data item A1 in 3 test cases, CaseA, CaseB and CaseC, searching the comparison strategy corresponding to the data item A1 in the data item set as a "exact match", comparing the expected value a1 corresponding to the data item A1 with the actual value of the data item A1 (the actual value of the data item A1 in the test result data after the test cases are executed) according to the comparison strategy "exact match", and if the comparison result is "exact match", verifying the data item A1 is passed, otherwise, the verification result is not passed through the verification of the data item A1. Similarly, the verification processes for the data items a2, a21, a22, a23, B1, B2, B21, B22, B23, C2, and C2 are sequentially completed according to the above-described process.
According to the verification method of the test result data, the corresponding verification strategy is automatically set for the data items according to the probability of the occurrence of the data items in the test cases for product testing, so that the verification time under the verification strategy setting method is shortest, the time cost of verification of the test result data is reduced, and the verification efficiency is improved.
Example two
Fig. 2 is a schematic diagram showing the structure of a verification apparatus for test result data according to an exemplary embodiment of the present invention.
Referring to fig. 2, the verification apparatus for test result data of the present embodiment specifically includes: a test case acquisition unit 210 and a verification policy setting unit 220.
The test case obtaining unit 210 is configured to obtain a plurality of test cases, where each test case includes a plurality of data items to be verified.
The verification policy setting unit 220 is configured to set verification policies for the data items according to the probabilities of the data items appearing in the test cases, so as to verify the data items according to the set verification policies during the process of executing the test cases.
Further, the verification policies include an active load verification policy and a delayed load verification policy.
Preferably, the verification policy setting unit 220 includes: and the benchmark probability determining module is used for determining the benchmark probability which can make the verification time required by the data items in the test cases shortest according to the probability of the data items in the test cases. And the verification strategy setting module is used for respectively setting the verification strategies for the data items according to the determined reference probability.
Preferably, the verification policy setting module includes: the first setting submodule is used for setting the verification strategy corresponding to the data item as the active loading verification strategy if the probability of the data item appearing in the plurality of test cases is greater than the reference probability; and a second setting submodule, configured to set the verification policy corresponding to the data item as the delayed loading verification policy if the probability of the data item appearing in the plurality of test cases is less than or equal to the reference probability.
Further, the probability of the data item appearing in the test cases is the percentage of the number of the test cases in which the data item appears in the test cases to the total number of the test cases.
Optionally, the apparatus further comprises: and the deduplication unit is used for performing deduplication processing on the plurality of data items in the plurality of test cases.
Preferably, the reference probability determination module includes: the third setting submodule is used for setting the number of the data items corresponding to the active loading verification strategy to be K; a first determining submodule, configured to determine, according to the K, that K data items, which have a highest probability of appearing in the test cases, in the multiple data items are the data items corresponding to the active loading verification policy, and the data items, except the K data items, in the multiple data items are the data items corresponding to the delayed loading verification policy; the first calculation submodule is used for calculating first average verification time according to the K and the probability of the data items corresponding to the active loading verification strategy appearing in the test cases; the second calculation submodule is used for calculating second average verification time according to the K and the probability of the data items corresponding to the delayed loading verification strategy appearing in the test cases; a third calculation submodule for calculating a sum of the first average verification time and the second average verification time; a second determining submodule, configured to determine the reference probability according to the value of K when a sum of the first average verification time and the second average verification time is minimum.
According to the verification device of the test result data, disclosed by the embodiment of the invention, the corresponding verification strategy is automatically set for the data items according to the occurrence probability of the data items in the test cases for product testing, so that the verification time under the verification strategy setting method is shortest, the time cost for verifying the test result data is reduced, and the verification efficiency is improved.
It should be noted that, according to the implementation requirement, each step described in the present application can be divided into more steps, and two or more steps or partial operations of the steps can be combined into a new step to achieve the purpose of the present invention.
The above-described method according to the present invention can be implemented in hardware, firmware, or as software or computer code that can be stored in a recording medium such as a CD, ROM, RAM, floppy disk, hard disk, or magneto-optical disk, or as computer code originally stored in a remote recording medium or a non-transitory machine-readable medium and to be stored in a local recording medium downloaded through a network, so that the method described herein can be stored in such software processes on a recording medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware such as an ASIC or FPGA. It will be appreciated that the computer, processor, microprocessor controller or programmable hardware includes memory components (e.g., RAM, ROM, flash memory, etc.) that can store or receive software or computer code that, when accessed and executed by the computer, processor or hardware, implements the processing methods described herein. Further, when a general-purpose computer accesses code for implementing the processes shown herein, execution of the code transforms the general-purpose computer into a special-purpose computer for performing the processes shown herein.
The above description is only for the specific embodiments of the present invention, but the scope of the present invention is not limited thereto, and any person skilled in the art can easily conceive of the changes or substitutions within the technical scope of the present invention, and all the changes or substitutions should be covered within the scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the appended claims.
Claims (12)
1. A method for validating test result data, the method comprising:
obtaining a plurality of test cases, wherein the test cases comprise a plurality of data items to be verified;
respectively setting verification strategies for the data items according to the probability of the plurality of data items appearing in all the plurality of test cases, so that the data items are verified according to the set verification strategies in the process of executing the test cases;
the verification policies include an active load verification policy and a delayed load verification policy, wherein,
the verifying the data item according to the set verification policy comprises:
according to the determined active loading verification strategy, firstly searching a comparison strategy corresponding to the data item, then searching an expected value corresponding to the data item, and then comparing the expected value and an actual value of the data item according to the comparison strategy;
according to the determined delayed loading verification strategy, an expected value corresponding to the data item is searched first, then a comparison strategy corresponding to the data item is searched, and then the expected value and the actual value of the data item are compared according to the comparison strategy.
2. The method according to claim 1, wherein the process of setting the verification policies for the data items according to the probabilities of the data items appearing in all the test cases comprises:
determining a benchmark probability which can make the verification time required by the data items in the test cases shortest according to the probability of the data items in the test cases;
and respectively setting the verification strategies for the data items according to the determined reference probabilities.
3. The method according to claim 2, wherein the process of setting the verification policies for the data items respectively according to the determined reference probabilities includes:
if the probability of the data item appearing in the test cases is larger than the reference probability, setting the verification strategy corresponding to the data item as the active loading verification strategy;
and if the probability of the data item appearing in the plurality of test cases is less than or equal to the reference probability, setting the verification strategy corresponding to the data item as the delayed loading verification strategy.
4. The method of any one of claims 1-3, wherein the probability that the data item occurs in the plurality of test cases is a percentage of the number of the test cases in which the data item occurs in the plurality of test cases to the total number of the plurality of test cases.
5. The method according to any one of claims 1-3, further comprising:
and performing deduplication processing on the plurality of data items in the plurality of test cases.
6. The method of claim 3, wherein the determining a benchmark probability that minimizes the verification time required for the plurality of data items in the plurality of test cases according to the probability of the plurality of data items occurring in all of the plurality of test cases comprises:
setting the number of the data items corresponding to the active loading verification strategy as K;
determining that K data items with the highest probability of occurrence in the test cases in the plurality of data items are the data items corresponding to the active loading verification strategy according to the K, and the data items except the K data items in the plurality of data items are the data items corresponding to the delayed loading verification strategy;
calculating first average verification time according to the K and the probability of the data items corresponding to the active loading verification strategy appearing in the test cases;
calculating second average verification time according to the K and the probability of the data items corresponding to the delayed loading verification strategy appearing in the plurality of test cases;
calculating a sum of the first average verification time and the second average verification time;
and determining the reference probability according to the value of the K when the sum of the first average verification time and the second average verification time is minimum.
7. An apparatus for validating test result data, the apparatus comprising:
the device comprises a test case acquisition unit, a verification unit and a verification unit, wherein the test case acquisition unit is used for acquiring a plurality of test cases, and the test cases comprise a plurality of data items to be verified;
the verification strategy setting unit is used for respectively setting verification strategies for the data items according to the probability of the plurality of data items appearing in all the plurality of test cases so as to verify the data items according to the set verification strategies in the process of executing the test cases;
the verification policies include an active load verification policy and a delayed load verification policy, wherein,
the verifying the data item according to the set verification policy comprises:
according to the determined active loading verification strategy, firstly searching a comparison strategy corresponding to the data item, then searching an expected value corresponding to the data item, and then comparing the expected value and an actual value of the data item according to the comparison strategy;
according to the determined delayed loading verification strategy, an expected value corresponding to the data item is searched first, then a comparison strategy corresponding to the data item is searched, and then the expected value and the actual value of the data item are compared according to the comparison strategy.
8. The apparatus of claim 7, wherein the authentication policy setting unit comprises:
a benchmark probability determining module, configured to determine, according to probabilities of the plurality of data items appearing in the plurality of test cases, a benchmark probability that a verification time required for the plurality of data items in the plurality of test cases is shortest;
and the verification strategy setting module is used for respectively setting the verification strategies for the data items according to the determined reference probability.
9. The apparatus of claim 8, wherein the verification policy setting module comprises:
a first setting sub-module, configured to set the verification policy corresponding to the data item as the active loading verification policy if a probability that the data item appears in the multiple test cases is greater than the reference probability;
and a second setting submodule, configured to set the verification policy corresponding to the data item as the delayed loading verification policy if the probability that the data item appears in the multiple test cases is smaller than or equal to the reference probability.
10. The apparatus of any of claims 7-9, wherein a probability that the data item occurs in the plurality of test cases is a percentage of the number of the test cases in which the data item occurs in the plurality of test cases to a total number of the plurality of test cases.
11. The apparatus according to any one of claims 7-9, further comprising:
and the deduplication unit is used for performing deduplication processing on the plurality of data items in the plurality of test cases.
12. The apparatus of claim 9, wherein the benchmark probability determination module comprises:
the third setting submodule is used for setting the number of the data items corresponding to the active loading verification strategy to be K;
a first determining submodule, configured to determine, according to the K, that K data items, which have a highest probability of appearing in the test cases, in the multiple data items are the data items corresponding to the active loading verification policy, and the data items, except the K data items, in the multiple data items are the data items corresponding to the delayed loading verification policy;
the first calculation submodule is used for calculating first average verification time according to the K and the probability of the data items corresponding to the active loading verification strategy appearing in the test cases;
the second calculation submodule is used for calculating second average verification time according to the K and the probability of the data items corresponding to the delayed loading verification strategy appearing in the test cases;
a third calculation submodule for calculating a sum of the first average verification time and the second average verification time;
and the second determining submodule is used for determining the reference probability according to the value of the K when the sum of the first average verification time and the second average verification time is minimum.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510251237.1A CN104850497B (en) | 2015-05-15 | 2015-05-15 | The verification method and device of test result data |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510251237.1A CN104850497B (en) | 2015-05-15 | 2015-05-15 | The verification method and device of test result data |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN104850497A CN104850497A (en) | 2015-08-19 |
| CN104850497B true CN104850497B (en) | 2017-10-31 |
Family
ID=53850150
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201510251237.1A Active CN104850497B (en) | 2015-05-15 | 2015-05-15 | The verification method and device of test result data |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN104850497B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108874665A (en) * | 2018-05-29 | 2018-11-23 | 百度在线网络技术（北京）有限公司 | A kind of test result method of calibration, device, equipment and medium |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7299453B2 (en) * | 2001-12-20 | 2007-11-20 | International Business Machines Corporation | Testing measurements |
| CN102122265A (en) * | 2011-03-03 | 2011-07-13 | 中国工商银行股份有限公司 | System and method for verifying computer software test results |
| CN103020269A (en) * | 2012-12-26 | 2013-04-03 | 广州市西美信息科技有限公司 | Method and device for verifying data |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20070038894A1 (en) * | 2005-08-09 | 2007-02-15 | Microsoft Corporation | Test Data verification with different granularity levels |
-
2015
- 2015-05-15 CN CN201510251237.1A patent/CN104850497B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7299453B2 (en) * | 2001-12-20 | 2007-11-20 | International Business Machines Corporation | Testing measurements |
| CN102122265A (en) * | 2011-03-03 | 2011-07-13 | 中国工商银行股份有限公司 | System and method for verifying computer software test results |
| CN103020269A (en) * | 2012-12-26 | 2013-04-03 | 广州市西美信息科技有限公司 | Method and device for verifying data |
Also Published As
| Publication number | Publication date |
|---|---|
| CN104850497A (en) | 2015-08-19 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11501163B2 (en) | Abnormality detection device, abnormality detection method, and storage medium | |
| KR101997119B1 (en) | Method for estimating interleaving period and system thereof | |
| US10049113B2 (en) | File scanning method and apparatus | |
| US9619372B2 (en) | Method and system for hybrid testing | |
| WO2021244677A1 (en) | Method and device for optimization of tire model | |
| CN111475402A (en) | Program function testing method and related device | |
| CN110968895B (en) | Data processing method and device, electronic equipment and storage medium | |
| CN104850497B (en) | The verification method and device of test result data | |
| CN113486358B (en) | Vulnerability detection method and device | |
| CN109033818B (en) | Terminal, authentication method, and computer-readable storage medium | |
| CN108196975B (en) | Data verification method and device based on multiple checksums and storage medium | |
| CN108920601B (en) | Data matching method and device | |
| US9342686B2 (en) | Systems and methods for updating scanning rules | |
| CN108075876B (en) | Time calibration method, device and system | |
| CN110287118B (en) | Test data generation method and software test method of test case | |
| CN110263417B (en) | Time sequence characteristic acquisition method and device and electronic equipment | |
| US20170163678A1 (en) | Network security assessment system | |
| US20190087304A1 (en) | Data analysis | |
| CN106776264B (en) | Application program code testing method and device | |
| CN109471766B (en) | Sequential fault diagnosis method and device based on testability model | |
| CN112000958A (en) | Method and device for detecting logic bugs during application program login and storage medium | |
| CN112733433A (en) | Equipment testability strategy optimization method and device | |
| CN105634668B (en) | A kind of empty inspection screening method and device of DCI0 signal | |
| CN106685966B (en) | Method, device and system for detecting leakage information | |
| CN105786791A (en) | Data topic acquisition method and apparatus |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| EXSB | Decision made by sipo to initiate substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |