CN108804334B - Discrete software reliability increase testing and evaluating method based on self-adaptive sampling - Google Patents

Discrete software reliability increase testing and evaluating method based on self-adaptive sampling Download PDF

Info

Publication number
CN108804334B
CN108804334B CN201810616381.4A CN201810616381A CN108804334B CN 108804334 B CN108804334 B CN 108804334B CN 201810616381 A CN201810616381 A CN 201810616381A CN 108804334 B CN108804334 B CN 108804334B
Authority
CN
China
Prior art keywords
test
probability
software
test cases
round
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201810616381.4A
Other languages
Chinese (zh)
Other versions
CN108804334A (en
Inventor
李秋英
王瑾益
张超
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Tianhang Changying (Jiangsu) Technology Co.,Ltd.
Original Assignee
Beihang University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beihang University filed Critical Beihang University
Priority to CN201810616381.4A priority Critical patent/CN108804334B/en
Publication of CN108804334A publication Critical patent/CN108804334A/en
Application granted granted Critical
Publication of CN108804334B publication Critical patent/CN108804334B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/36—Preventing errors by testing or debugging software
    • G06F11/3668—Software testing
    • G06F11/3672—Test management
    • G06F11/3684—Test management for test design, e.g. generating new test cases
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/36—Preventing errors by testing or debugging software
    • G06F11/3668—Software testing
    • G06F11/3672—Test management
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/36—Preventing errors by testing or debugging software
    • G06F11/3668—Software testing
    • G06F11/3672—Test management
    • G06F11/3688—Test management for test execution, e.g. scheduling of test suites

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Hardware Design (AREA)
  • Quality & Reliability (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Debugging And Monitoring (AREA)

Abstract

The invention provides a discrete software reliability growth test and evaluation method based on self-adaptive sampling selection test cases, and belongs to the field of software reliability engineering. The method is based on the traditional software reliability increase test, the self-adaptive sampling method is applied to the Musa operation profile, the extraction of the test cases is controlled by adjusting the occurrence probability of the operation profile, the purpose is to increase the extraction quantity of the test cases in the areas with more defects, reduce the extraction quantity of the test cases in the areas with less defects, form a self-adaptive adjustment feedback mechanism of the reliability test, determine the test case quantity required by the next round of test by using the KLD sampling principle, change the traditional simple random sampling into the random group sampling method sampling, and ensure the unbiased property of the traditional software reliability increase test on the evaluation result. The invention has better test efficiency, and compared with the traditional test method, the invention has the advantages of less test dosage and accurate test result.

Description

Discrete software reliability increase testing and evaluating method based on self-adaptive sampling
Technical Field
The invention belongs to the field of software reliability engineering, particularly relates to the field of software reliability testing and evaluation, and particularly relates to a discrete software reliability growth testing and evaluation method based on self-adaptive sampling selection of test cases.
Background
The software reliability test has extremely important significance for the quality assurance of the software, on one hand, the software defect which has important influence on the software reliability is continuously eliminated through the software reliability increase test, and the continuous increase of the software reliability level is realized so as to meet the requirement of a user on the software reliability; on the other hand, whether the software reaches the required reliability level needs to be judged through a software reliability verification test, and whether the software can be received or issued is further determined.
However, at present, the application of software reliability testing is not ideal, a huge number of test cases under the guidance of the traditional reliability testing theory and method is an important reason that the software is not enough, and the problem is particularly prominent for high-reliability software. For analysis reasons, the conventional reliability test method only uses information at the end of the test, but does not use information during the test, so that the amount of information is not sufficiently used, and generally, a large number of test cases need to be used. Therefore, how to improve the efficiency of the traditional software reliability test, accelerate the progress of the software reliability test, and complete the specified reliability test target in as little time and cost as possible becomes a problem which the researchers in the current software reliability direction have to consider.
A software self-adaptive test method based on a control theory is provided by Chua's element and a team, a controlled Markov chain is taken as a basis, tested software is taken as a controlled object, a corresponding test strategy is taken as a controller, and according to a preset test target, the online estimation is carried out on related parameters by using online collected test data, so that the software test strategy is further optimized. The method is a software defect testing method, does not emphasize specific testing types and control strategies, is an abstract method at a frame level, and limits the application range and efficiency of the model due to the fact that the test model carries out special treatment on part of assumed conditions; tson Yueh Chen provides a self-adaptive random test method to improve the effectiveness of random test, a certain number of test cases are randomly generated each time to form a candidate test case set, and then a candidate test case which is farthest away from all the current test cases is selected from the candidate test case set to serve as the next test case. Therefore, each generated test case is far away from the tested test case, so that the probability that the generated test case falls in the failure area is increased, and all the generated test cases can be uniformly distributed in the whole input area. However, this method has two problems: firstly, the algorithm overhead is the time complexity required for generating the nth test case, and secondly, along with the increase of the dimension of the input domain, the effectiveness of the test method is rapidly reduced and even worse than the random test. Moreover, the random test in the method does not point to the reliability test, and therefore the reliability test target and the characteristic are not taken as the basic starting point.
Therefore, how to study the software reliability test of the adaptive sampling under the condition of taking the traditional operation profile as the basis, and comprehensively study and consider the software reliability growth test and the software reliability evaluation has important research significance and application value.
Disclosure of Invention
The invention provides a software reliability growth test and evaluation method based on self-adaptive sampling aiming at discrete software, aiming at solving the problem of low software reliability test efficiency of the traditional Musa operation profile.
The invention provides a software reliability growth test and evaluation method based on adaptive sampling, which comprises the following specific steps:
the method comprises the following steps that firstly, a traditional Musa operation section of tested software is constructed, and therefore each operation and the corresponding occurrence probability are obtained;
secondly, randomly sampling according to the constructed operation profile to generate n test cases (the value of n is usually determined by experience, namely the number of the test cases required by the required reliability test growth target is completed under the guidance of the traditional operation profile, and generally, n is a relatively large numerical value);
thirdly, executing the generated test cases in sequence until the first failure occurs, taking the test from the first executed test case to the first failed test case as a first test, mapping the test cases used in the first test to the corresponding input domains corresponding to the operation, and calculating the failure probability of each input domain to obtain the success rate of the whole software;
fourthly, calculating the updated occurrence probability of each operation by using a probability updating formula according to the failure probability of each region calculated in the third step;
fifthly, determining the number of test cases to be extracted in the next test;
sixthly, distributing the number of test cases to be extracted by each operation according to the updated occurrence probability of the operation, and extracting the corresponding number of test cases from each operation by utilizing a hierarchical sampling method;
seventhly, operating the extracted test case set, calculating to obtain failure probability corresponding to each region according to the number of failures, and further calculating the success rate of the whole software;
and step eight, judging whether the tested software meets the preset reliability requirement, if so, stopping the test, and if not, turning to the step four to execute.
Compared with the prior art, the invention has obvious advantages and beneficial effects:
1) the software reliability growth test and evaluation method based on the self-adaptive sampling changes the occurrence probability of the traditional operation profile, controls the extraction of the test cases by adjusting the occurrence probability of the operation according to the failure result, realizes the increase of the use of the test cases in the areas with more defects, reduces the use of the test cases in the areas with less defects, and effectively forms a self-adaptive adjustment feedback mechanism for the extraction of the test cases in the test process, thereby achieving the purpose of ensuring that the test cases and the test time are reduced under the same test effect by adjusting the test cases.
2) The method can utilize the original theory and result in the reliability test aspect, such as the structure of the traditional operation profile and the layered sampling method, so that the method provided by the invention is well continued with the existing research result, and the repeated research of the original research result is not completely abandoned.
3) Through example application, the invention has better test efficiency, and compared with the traditional test technology, the invention has the advantages of short test time and accurate test result, and is convenient for engineering personnel to understand and actually operate.
Drawings
FIG. 1 is a schematic flow chart of a software reliability growth test and evaluation method of the present invention;
FIG. 2 is a flow chart illustrating probability updating according to the present invention.
Detailed Description
Specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings and examples.
FIG. 1 is a flow chart of a software reliability growth test and evaluation method of the present invention. The method comprises the steps of firstly establishing an operation profile of tested software, generating a test case by using a random sampling method, determining a first-round test according to a first failure occurrence after the test is started, and calculating the failure probability of each operation corresponding to an input domain according to the result of the first-round test. FIG. 2 is a flow chart of probability updating of the present invention. According to the failure probability of each input domain, calculating the updated occurrence probability of operation by using a probability updating formula, determining the quantity of test cases required by the next round of test according to a distance formula, distributing the test cases according to the updated occurrence probability, extracting the cases and executing the test based on a random group selection method, calculating the success rate of software, judging whether the success rate reaches a reliability increasing target, if so, stopping the test, and if not, continuing the steps until the reliability increasing target is reached. The method comprises the following specific steps:
in the first step, an operational profile of the software is established. The operation profile is a quantitative description of how to use the software system, the traditional Musa operation profile is used in the method, and the method is simple and practical and is very effective for software which has no mutual influence between operations and has independent operations. The mathematical expression for the Musa operating profile is: { OP | OP ═<Oi,pi>,i＝1,2,...,m}，OiDenotes the ith operation, piThe occurrence probability of the ith operation is represented, and m represents the number of software operations. Thus, an operational profile can be viewed as a set of a series of software operations and the probabilities of those operations occurring, where a software operation corresponds to a run of a software reliability test, i.e., a complete software reliability test case.
And secondly, generating a large number of test cases according to the operation profile, and distributing the test cases according to the occurrence probability of each operation. The specific method comprises the following steps:
1) carrying out random sampling according to the operation profile constructed in the first step to generate a test case, and the specific process is as follows: firstly, the specific value of each input variable in the operation of the tested software is determined, and a test case is generated by sampling the two steps of extracting the operation and extracting the value of each input variable.
2) Generating N test cases and the occurrence probability p of each operationiMultiplying to obtain the number N of reserved test cases of each operationi＝N×pi。
And thirdly, sequentially executing the randomly generated reliability test cases until the first failure occurs, defining the test performed before as a first-round test, and calculating the failure probability of the input domain corresponding to each operation.
1) Failure is defined as, assuming that a test case t is selected,
Figure GDA0002764103160000049
is a test case applicationThe actual output obtained after the software is run,
Figure GDA00027641031600000410
is the result expected by the test case running software, using ZtRecording the results of the test when
Figure GDA00027641031600000411
When Z istWhen 1 is equal to
Figure GDA00027641031600000412
When Z ist0, namely:
Figure GDA0002764103160000041
2) let the failure probability of the ith zone in the kth test be recorded as
Figure GDA0002764103160000042
(wherein k.gtoreq.2), then
Figure GDA0002764103160000043
The calculation method comprises the following steps:
Figure GDA0002764103160000044
Figure GDA0002764103160000045
Figure GDA0002764103160000046
wherein,
Figure GDA0002764103160000047
the probability of failure in the ith zone at the time of the kth test,
Figure GDA0002764103160000048
is the probability of occurrence of the i-th region in the k-th test, Yi (k-1)Indicates the number of failures found in the i-th zone during the k-th test, Ni (k-1)Represents the total number of test cases extracted from the ith area during the kth test,
Figure GDA0002764103160000051
indicating the success rate of the entire software after the k-th test.
It should be noted that the calculation formula of the above values corresponding to the first round of test is as follows (when k is 1):
Figure GDA0002764103160000052
Figure GDA0002764103160000053
Figure GDA0002764103160000054
wherein,
Figure GDA0002764103160000055
the probability of occurrence of the ith region in the first test, namely the probability of occurrence of the ith region in the Musa operation profile,
Figure GDA0002764103160000056
is the failure probability in the ith area in the first round of test, Yi (0)The failure number in the ith area in the first round of test,
Figure GDA0002764103160000057
the number of test cases in the ith area during the first round of test is shown.
It should be noted that there is no adaptive adjustment process in the first round of testing, and therefore the purpose of the first round of testing is only to start the adaptive sampling process.
Fourthly, updating the occurrence probability of each region according to a formula, and carrying out normalization processing on the updated occurrence probability to obtain the updated occurrence probability corresponding to the operation;
1)πi (k)′indicates the occurrence probability of the ith area in the k +1 test round without normalization, and the failure probability of the area i in the k test round
Figure GDA0002764103160000058
And the probability of occurrence of the ith region in the kth round of testing as follows:
Figure GDA0002764103160000059
wherein γ is a smoothing factor, and takes a value of [0,1] for adjusting the weight value occupied by the previous test in the current test, and generally, γ may take 0.5.
2) Pi obtained by formula (8)i (k)′The value of (d) needs to be normalized to ensure that all values of (n) are equali (k)The sum is 1, and the calculation formula is as follows:
Figure GDA00027641031600000510
where m represents the number of software operations, i.e., the number of regions corresponding to the operations. Obtaining normalized pi by formula (9)i (k)It can be seen that, as long as the input domain corresponding to the operation i is failed in the test process, the occurrence probability of the operation in the corresponding domain is changed.
And fifthly, determining the quantity of the test cases required to be extracted in the next round of test.
Minimum sample size T of test case of next round of test(k)The calculation method comprises the following steps:
Figure GDA0002764103160000061
wherein ξ(k)The K-L distance representing the probability distribution between the K +1 th test and the K test,
Figure GDA0002764103160000062
to represent
Figure GDA0002764103160000063
p (x) represents
Figure GDA0002764103160000064
ξ(k)Representing two probability distributions
Figure GDA0002764103160000065
And p (x), i.e.:
Figure GDA0002764103160000066
the sample size of the test case required for the k +1 th round of test is:
Figure GDA0002764103160000067
wherein, T(k)Sample size of test case, pi, required for the k +1 th round of testi (k)Represents the occurrence probability of the ith area in the k +1 test round, pii (k-1)Representing the probability of occurrence of the ith region, ξ, during the kth test(k)Is pii (k)And pii (k-1)K-L distance between, 1-delta is the confidence level, z1-δIs a critical value for a standard normal distribution with confidence 1- δ. As can be seen from equation (12), the size of the sample size of the test case of the k +1 th round test is related to the number m of partitions and the distance between the two probability distributions.
The principle of determining the test case quantity by using the K-L distance is as follows: and determining the number of test cases required by the next round of test in real time on line according to the distribution condition of the test cases on the operation profile by using a KLD (Kullback-Leibler divergence) sampling principle, reducing the number of cases extracted in an area which does not contribute to failure finding, and only reserving the minimum number of test cases required for ensuring the reliability estimation precision.
KLD sampling is an algorithm that adaptively adjusts the sampling size using KLD, and has been successfully applied to the practice of particle filtering algorithms. The core idea is to control the particle sample size generated in the filtering process by introducing a certain error range, use more particles when the uncertainty of the particle distribution is higher, and use less particles in the particle filtering algorithm when the uncertainty of the particle distribution is lower, so as to effectively reduce the number of particles used in the filtering process on the premise of ensuring certain filtering precision.
In particular, the K-L distance represents two probability distributions p (x) and
Figure GDA0002764103160000071
the approximation difference between them can be expressed as:
Figure GDA0002764103160000072
the K-L distance is non-negative, if and only if p (x) and
Figure GDA0002764103160000073
when equal, it has a value of 0.
Assuming that the sample size n is sampled from k discrete independent subspaces, with the vector X ═ X (X)1,...,Xk) Represents the amount of samples collected from each subspace, and piRepresenting the probability of each subspace. The maximum likelihood estimate of the probability to be estimated is expressed as
Figure GDA0002764103160000074
Likelihood ratio estimator lambda examining pnComprises the following steps:
Figure GDA0002764103160000075
likelihood ratio lambdanConverge to χ → ∞ at n → ∞2The degree of freedom of the distribution is recorded as k-1. If n samples of a true unknown distribution are given, the cumulative sum of the rightmost (14) equations is equivalent to the K-L distance sum obtained using each sample. When in use
Figure GDA0002764103160000076
In time, there are:
Figure GDA0002764103160000077
χ2the quantile of the distribution is
Figure GDA0002764103160000078
Order to
Figure GDA0002764103160000079
It is possible to obtain:
Figure GDA00027641031600000710
according to the formula (15), can obtain
Figure GDA00027641031600000711
At this point, if the minimum condition is satisfied, an approximate estimate of the sample size is obtained, which means that the maximum likelihood estimate of the true distribution and the probability that the K-L distance of the true distribution is less than an arbitrary value epsilon are guaranteed to be not less than 1-delta. To calculate
Figure GDA00027641031600000712
The approximation calculation formula of n when the approximation error is guaranteed to be minimum can be obtained by adopting a Wilson-Hilferty transformation method as follows:
Figure GDA00027641031600000713
wherein z is1-δIs the upper bound of the 1-delta quantile of the standard normal distribution N (0, 1). When n takes the minimum value, we can approximate:
Figure GDA0002764103160000081
the meaning is as follows: the K-L distance between the probability density of the true unknown distribution and the probability density of the estimated distribution is guaranteed to be smaller than a threshold value epsilon, and the threshold value epsilon is guaranteed to be minimum, and the sample size n is the minimum value at the moment.
And sixthly, distributing the test case quantity obtained in the fifth step according to the updated occurrence probability of the operation, calculating the test case quantity to be distributed by each operation, and extracting the test cases with corresponding quantity from each operation. The specific process is as follows:
1) determining the number T of test cases extracted in the ith area by the (18) th round testi (k)：
Figure GDA0002764103160000082
2) The test case extraction adopts a random group decimation method, which comprises the following steps: if a sample with the sample size of n is to be extracted from the population U, the population is randomly divided into n pairwise disjoint sets Ui(i ═ 1,2, …, n), i.e. U1Is extracted from U by simple random sampling method without returning2Is to use a simple random sampling method without putting back to the slave U-U1Extract, and so on. Number of units N in each groupiCan be selected in advance and is preferably equal to m or m +1, where m is an integer part of N/N, where N is the total number of units
Figure GDA0002764103160000083
The principle of the sampling method is as follows:
for group g, let ZgIs a total measure of the relative size of the g-th group, then the second in that groupThe probability of i units being drawn is zi/ZgEach group only draws one sample unit, and the unit index value drawn by the g group is recorded as ygThe corresponding value of Z is ZgThen the estimate of the total sum is:
Figure GDA0002764103160000084
is an unbiased estimate of the total sum, where
Figure GDA0002764103160000085
Is the total value Y of the g-th groupgUnbiased estimation of (d). Estimator
Figure GDA0002764103160000086
The variance of (c) is:
Figure GDA0002764103160000087
and an unbiased estimate thereof:
Figure GDA0002764103160000091
and seventhly, running the test case, recording failure data, calculating the failure probability of each area, and calculating the success rate of the software by using the formula (4).
And eighth step, comparing with a preset reliability increase target, if the requirement is met, terminating the test, and if the requirement is not met, switching to the fourth step.
The embodiments described herein are merely intended to illustrate and explain the present invention and not to limit the present invention by applying the concrete method of the present invention to a certain software with reference to the accompanying drawings.
The embodiment of the invention takes an ATM simulation software as a representative, and explains the software reliability increase testing and evaluating method based on the self-adaptive sampling provided by the invention.
Table 1 is a representation of a conventional operating profile of the prior art established for the software S.
TABLE 1 conventional operating profiles for software S
Operation oi o1 o2 … oi … om
Input field Di D1 D2 … Di … Dm
Probability of occurrence pi p1 p2 … pi … pm
A conventional operating profile is first constructed using the Musa operating profile construction method, as shown in table 2:
TABLE 2 operational Profile of ATM simulation software
Figure GDA0002764103160000092
The growing targets for the reliability tests given beforehand are: success rate RobjThe values of the parameters required to be used are respectively as follows: delta 0.025, z1-δ＝1.96，γ＝0.5。
Secondly, generating 8000 test cases according to the operation section of the software, and then distributing the test cases according to the occurrence probability of each operation:
for example, the number of test cases allocated in operation 1 (mobile payment) is as follows
0.081×8000＝648
The number of test cases distributed by other operations is calculated in the same way, and the result is shown in table 3:
TABLE 3 test case quantities allocated for each operation
Figure GDA0002764103160000101
And thirdly, executing the test according to the traditional software reliability increase test method, and executing 8000 test cases in sequence from the first one. In the present embodiment, a failure occurs when the 25 th test case is executed. Therefore, the first test run from the start of the test to the 25 th case is defined as the first test of the current test, and thus it can be seen that the number of test cases of the first test run is 25. The 25 th test case was confirmed to be under operation 7 (loss of report operation). The first round of test cases and the failure number are counted, and the results are shown in table 4:
TABLE 4 test cases and failure counts for the first round of testing
Figure GDA0002764103160000102
Calculating the failure probability of the input domain corresponding to each operation as follows:
Figure GDA0002764103160000103
Figure GDA0002764103160000104
wherein,
Figure GDA0002764103160000105
indicating the probability of failure of the ith zone at the time of the first round of testing.
The fourth step, utilize
Figure GDA0002764103160000106
The value of (d) and γ are substituted into equation (8) to update the occurrence probability, and the following are obtained:
Figure GDA0002764103160000107
Figure GDA0002764103160000108
Figure GDA0002764103160000111
Figure GDA0002764103160000112
Figure GDA0002764103160000113
Figure GDA0002764103160000114
Figure GDA0002764103160000115
wherein, pii (1)′Is the probability of occurrence of the ith region in the second round of testing without normalization.
The above values are substituted into equation (9) to be normalized to obtain
Figure GDA0002764103160000116
The results of normalization for other occurrence probabilities were calculated in the same manner, see table 5:
TABLE 5 probability of occurrence of input fields corresponding to each operation in the second round of testing
Figure GDA0002764103160000117
As can be seen from table 5, except that the occurrence probability of the input field corresponding to the operation 7 that has failed is improved, the occurrence probability of the other operations that have not found failure is reduced, which embodies the intention that the software reliability increase test method provided by the present invention wants to control the extraction number of test cases by adjusting the occurrence probability of the operation profile, that is, the extraction number of test cases is increased in the area with many defects, whereas the extraction number of test cases is decreased in the area with few defects, thereby forming an adaptive adjustment feedback mechanism for reliability testing.
And fifthly, determining the number of test cases required by the second round of test.
Calculating according to the formula (11) to obtain xi(1)：
Figure GDA0002764103160000121
Substituting the values of the variables into equations to obtain xi(1)＝0.075。
And using the formula (12) where δ is 0.025, z1-δ1.96 and gamma 0.5, can obtain
Figure GDA0002764103160000122
Wherein, T(1)Is the amount of test cases required for the second round of testing.
Therefore, the number of test cases required for the second round of testing is 94.
Sixthly, distributing the test case quantity according to the occurrence probability of the updated operation to obtain the test case quantity corresponding to each operation, wherein the calculation method comprises the following steps:
the number of test cases for operation 1 is: 0.0775X 94. apprxeq.7.
The test case numbers of other operations are calculated in the same way, and are shown in table 6:
TABLE 6 test case number for each operation of the second round of testing
Figure GDA0002764103160000123
Then extracting a corresponding number of test cases from each operation, wherein the specific implementation method comprises the following steps:
when l test cases need to be extracted from the input domain corresponding to the operation i, the total test cases of the operation i are divided into l parts, and then one test case is randomly extracted from each part, so that l test cases are obtained.
For example, operation 1 needs to extract 7 test cases, so that the remaining test cases 648-3 (used in the previous round of test) of operation 1 (645) are randomly and averagely divided into 7 parts, and one is randomly extracted from each part. The extraction process of the test cases of the rest operations is also carried out as the same.
And seventhly, after the test cases are extracted, executing the corresponding test cases to obtain failure data of the test in the round, and calculating the success rate of the current software according to a formula. The collected failure data information is shown in table 7:
TABLE 7 test results of the second round of testing
Figure GDA0002764103160000124
Figure GDA0002764103160000131
Calculating the failure probability of each input domain according to formula (2), for example, the failure probability of the input domain corresponding to operation 1 is:
Figure GDA0002764103160000132
the failure probability of other input domains is calculated by the same method, and is shown in the table 8:
TABLE 8 results of the calculations after the second round of testing
Figure GDA0002764103160000133
According to the formula (4), the success rate of the software at the moment is calculated and obtained:
Figure GDA0002764103160000134
and judging whether the success rate at the moment meets the requirement or not according to the calculation result. Because R is(1)＝0.9368<0.965, so the success rate does not meet the reliability growth requirement, and the test continues.
Eighth step (refer to fourth step), using
Figure GDA0002764103160000135
The sum of (a) and (y) being 0.5 is substituted intoThe updated results of the occurrence probability are obtained by equations (9) and (10), see table 9:
TABLE 9 probability of occurrence of each operation in the third round of testing
Figure GDA0002764103160000136
And a ninth step (refer to the fifth step), determining the number of test cases required by the third test, and calculating the result as follows:
Figure GDA0002764103160000141
Figure GDA0002764103160000142
i.e. the number of test cases required for the third round of testing is 131.
A tenth step (refer to the sixth step), of distributing the number of test cases obtained by calculation according to the updated occurrence probability of the operation, to obtain the number of test cases of the input domain corresponding to each operation, as shown in table 10:
TABLE 10 test cases for the third round of testing each operation
Figure GDA0002764103160000143
And a tenth step (refer to the seventh step), after the test case extraction is finished, performing a test to obtain a third round of test results and calculation results of the failure probability of the corresponding input domain, as shown in table 11.
TABLE 11 third round test results
Figure GDA0002764103160000144
According to formula (4), the success rate of the software at this time is calculated:
Figure GDA0002764103160000145
because of R at this time(2)0.9693, greater than the set target value (R)obj0.965), so the test is stopped.
The test results are compared with those of the conventional reliability increase test method, as shown in table 12.
TABLE 12 comparison of test results of two test methods
Figure GDA0002764103160000151
Therefore, when the reliability increase target of the same level (or level) is reached, the software reliability increase test method provided by the invention has the test efficiency improved by 85 percent compared with the traditional software reliability test method, namely, the number of test cases can be reduced, and the test efficiency is greatly improved.
The invention provides a flow, an execution method and a software reliability evaluation method of a software reliability increase test method of self-adaptive sampling on the basis of a traditional Musa operation profile, the extraction quantity of test cases is controlled by adjusting the occurrence probability of operation, the extraction of the test cases is increased in areas with more defects, the extraction of the test cases is reduced in areas with less defects, and a self-adaptive adjustment feedback mechanism of the reliability increase test is formed; and the simple random sampling is changed into random group sampling, so that the unbiased performance of the reliability increase test of the traditional software on the evaluation result is ensured.
Finally, it should be noted that: the above embodiments are only used to illustrate the present invention and do not limit the technical solutions described in the present invention; thus, although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those skilled in the art that various changes may be made and equivalents may be substituted; all such modifications and variations are intended to be included herein within the scope of this disclosure and the present invention and protected by the following claims.

Claims (3)

1. A discrete software reliability growth test and evaluation method based on self-adaptive sampling is characterized in that the self-adaptive sampling method is applied to a Musa operation profile to form a new software reliability growth test method, the occurrence probability of the operation profile is changed according to the occurrence condition of failure, the extraction of test cases is increased in a region with many defects, the extraction of test cases is reduced in a region with few defects, and the method is realized through the following steps:
step one, constructing a traditional Musa operation profile of the tested software;
randomly generating a large number of test cases according to the constructed operation profile, and distributing the test cases according to the occurrence probability of each operation;
step three, executing the test cases in sequence according to a traditional test method until the first failure occurs, defining the test cases as a first round of test, and calculating the failure probability of each operation corresponding to the input domain;
updating the occurrence probability in the section according to the failure probability of each region;
step five, determining the quantity of test cases required to be extracted in the next round of test;
step six, distributing the determined test cases according to the updated occurrence probability of the operation, and extracting a corresponding number of test cases from each operation by using a random group decimation method;
and step seven, running the test case, recording failure data, calculating failure probability of each area and success rate of the whole software, comparing with a reliability increase test target, stopping testing if the requirements are met, and turning to step four if the requirements are not met.
2. The method for testing and evaluating the reliability increase of discrete software based on adaptive sampling according to claim 1, wherein in the fourth step, the formula for updating the occurrence probability is as follows:
Figure FDA0002764103150000011
wherein, pii (k)′Indicates the occurrence probability of the ith area in the k +1 test round without normalization treatment, pii (k-1)The occurrence probability of the ith area in the kth test is shown, gamma is a smoothing factor, and the value range is [0,1]]An interval for adjusting the size of the weight value occupied by the current test,
Figure FDA0002764103150000012
the failure probability of the area i in the k-th round of test is shown, and k is more than or equal to 1;
to pi obtained by the above formulai (k)′Then normalization is carried out to ensure all pii (k)The sum of 1 is specifically as follows:
Figure FDA0002764103150000013
wherein, pii (k)The occurrence probability of the ith area in the k +1 test is shown, and m represents the number of software operations.
3. The method for testing and evaluating the reliability of discrete software based on adaptive sampling according to claim 1, wherein in the fifth step, the method for determining the quantity of test cases to be extracted in the next test round is as follows:
Figure FDA0002764103150000014
Figure FDA0002764103150000021
wherein, T(k)The sample size of the test case required for the k +1 th round of test, and m represents the softwareNumber of operations, ni (k)Represents the occurrence probability of the ith area in the k +1 test round, pii (k-1)Representing the probability of occurrence of the ith region, ξ, during the kth test(k)Is pii (k)And pii (k-1)K-L distance between, 1-delta is the confidence level, z1-δIs a critical value for a standard normal distribution with confidence 1- δ.
CN201810616381.4A 2018-06-15 2018-06-15 Discrete software reliability increase testing and evaluating method based on self-adaptive sampling Active CN108804334B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201810616381.4A CN108804334B (en) 2018-06-15 2018-06-15 Discrete software reliability increase testing and evaluating method based on self-adaptive sampling

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201810616381.4A CN108804334B (en) 2018-06-15 2018-06-15 Discrete software reliability increase testing and evaluating method based on self-adaptive sampling

Publications (2)

Publication Number Publication Date
CN108804334A CN108804334A (en) 2018-11-13
CN108804334B true CN108804334B (en) 2021-01-12

Family

ID=64086372

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201810616381.4A Active CN108804334B (en) 2018-06-15 2018-06-15 Discrete software reliability increase testing and evaluating method based on self-adaptive sampling

Country Status (1)

Country Link
CN (1) CN108804334B (en)

Families Citing this family (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111061640B (en) * 2019-12-18 2023-02-17 电信科学技术第十研究所有限公司 Software reliability test case screening method and system
CN111783293B (en) * 2020-06-24 2022-04-19 西北工业大学 Method for analyzing post-buckling reliability of composite material stiffened wall panel based on self-adaptive important sampling
CN112306859B (en) * 2020-07-10 2022-07-08 北京航空航天大学 Improved software self-adaptive testing method
CN113189985B (en) * 2021-04-16 2022-09-23 南京大学 Partially observable driving planning method based on adaptive particle and belief filling
CN114779058B (en) * 2022-06-23 2022-09-23 联宝(合肥)电子科技有限公司 Mainboard detection method, device, equipment and medium for dynamically adjusting item measurement proportion
CN117575408A (en) * 2023-11-22 2024-02-20 上海正泰电源系统有限公司 Simulation method for product reliability growth

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9582410B2 (en) * 2010-10-27 2017-02-28 International Business Machines Corporation Testing software on a computer system
CN102360332B (en) * 2011-09-28 2013-11-13 北京航空航天大学 Software reliability accelerated test and evaluation method and computer-aided tool used in same
CN102324102B (en) * 2011-10-08 2014-04-16 北京航空航天大学 Method for automatically filling structure information and texture information of hole area of image scene
CN102629232B (en) * 2012-01-09 2014-03-26 北京航空航天大学 Software reliability verifying and testing method with test validity introduced

Also Published As

Publication number Publication date
CN108804334A (en) 2018-11-13

Similar Documents

Publication Publication Date Title
CN108804334B (en) Discrete software reliability increase testing and evaluating method based on self-adaptive sampling
CN108986470B (en) Travel time prediction method for optimizing LSTM neural network by particle swarm optimization
CN113837356B (en) Intelligent sewage treatment prediction method based on fused neural network
CN109214708B (en) Electric power system risk assessment method based on cross entropy theory optimization support vector machine
CN112115419B (en) System state estimation method and system state estimation device
CN110910004A (en) Reservoir dispatching rule extraction method and system with multiple uncertainties
CN112364560B (en) Intelligent prediction method for working hours of mine rock drilling equipment
CN110889085A (en) Intelligent wastewater monitoring method and system based on complex network multiple online regression
CN107798426A (en) Wind power interval Forecasting Methodology based on Atomic Decomposition and interactive fuzzy satisfying method
CN113505477A (en) Process industry soft measurement data supplementing method based on SVAE-WGAN
CN112434848A (en) Nonlinear weighted combination wind power prediction method based on deep belief network
CN111783242B (en) RVM-KF-based rolling bearing residual life prediction method and device
CN111414703A (en) Method and device for predicting residual life of rolling bearing
CN110163743A (en) A kind of credit-graded approach based on hyperparameter optimization
Ma et al. Optimum step‐stress accelerated life test plans for log‐location‐scale distributions
CN115982141A (en) Characteristic optimization method for time series data prediction
CN113344288A (en) Method and device for predicting water level of cascade hydropower station group and computer readable storage medium
CN118313265A (en) Oil reservoir history fitting method, system, equipment and medium based on error learning double-agent model
Wang et al. Composite quantile regression for GARCH models using high-frequency data
CN115062528A (en) Prediction method for industrial process time sequence data
CN115100233B (en) Radar target tracking method based on generation of anti-network resampling particle filtering
CN114360641B (en) Gene regulation network structure identification method based on variable decibel leaf
CN115759455A (en) Load probability density prediction method based on time sequence Gaussian mixture density network
CN112967154B (en) Assessment method and device for Well-rolling of power system
CN115345303A (en) Convolutional neural network weight tuning method, device, storage medium and electronic equipment

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant
TR01 Transfer of patent right

Effective date of registration: 20220127

Address after: 215488 No. 301, building 11, phase II, Taicang University Science Park, No. 27, Zigang Road, science and education new town, Taicang City, Suzhou City, Jiangsu Province

Patentee after: Tianhang Changying (Jiangsu) Technology Co.,Ltd.

Address before: 100191 No. 37, Haidian District, Beijing, Xueyuan Road

Patentee before: BEIHANG University

TR01 Transfer of patent right