CN103631714A - Method for generating minimum combination testing cases based on matrix multiplicity - Google Patents
Method for generating minimum combination testing cases based on matrix multiplicity Download PDFInfo
- Publication number
- CN103631714A CN103631714A CN201310556759.3A CN201310556759A CN103631714A CN 103631714 A CN103631714 A CN 103631714A CN 201310556759 A CN201310556759 A CN 201310556759A CN 103631714 A CN103631714 A CN 103631714A
- Authority
- CN
- China
- Prior art keywords
- factor
- matrix
- dematrix
- row
- combination
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Landscapes
- Mobile Radio Communication Systems (AREA)
Abstract
The invention discloses a method for generating minimum combination testing cases based on matrix multiplicity. The method includes the steps that firstly, an input matrix is generated according to the number of factors and the number of values of each factor; secondly, an initial solution matrix is generated through the multiplicity comparison method; thirdly, whether the generated solution matrix can cover all factor combinations or not is verified, and factor combinations which are not covered are recorded; fourthly, the factor combinations which are not covered are covered based on the conception of the greedy algorithm to obtain the final solution matrix, wherein each line in the solution matrix is one testing case. The method has the advantage that the number of the generated testing cases is small under the condition that all the factor combinations are covered; through comparison experiments between the method and existing normally-used combination testing tools like an ACTS and a PICT, the effectiveness of the method and the advantage that the number of the generated combination testing cases is small are verified.
Description
Technical field
The invention belongs to the combined test use-case generation technique field in software test, relate to a kind of minimum combination method for generating test case based on matrix multiplicity.
Background technology
Conventionally software systems are complicated flogic systems, and its normal operation is subject to multifactorial impact, and these factors comprise system configuration, internal event, outside input and running environment etc., can abstract representation the input factor that is software systems.The interaction of these factors can cause the inefficacy of software conventionally, in actual software test, can to software systems, test fully by testing all combinations of these systematic parameters.Have n input parameter (factor) for one, the value number of each factor is v
1, v
2..., v
nsoftware systems, according to these software systems needs of input parameter combination Complete test
individual test case, for general test macro, this combination is a very huge numeral.For use test use-case as few as possible carrys out effectively more early to detect software error, people have proposed the method for testing software covering based on combination, are called for short combined test (Combination Testing).At international esbablished corporations such as industry member ,IBM, Microsoft and Bell laboratories, all developed corresponding combined test instrument, these combined test instruments, aspect test case generating algorithm, generally adopt the greedy algorithm of speed.The thought of greedy algorithm is from empty matrix, and progressively or column by column extended matrix, until all t combination is all capped.According to the difference of extended mode, can be divided into One-Dimensional Extended and two-dimensional expansion two classes.The algorithm frame of One-Dimensional Extended is realized by business test case Core Generator AETG at first, the Greedy strategy of AETG is non-deterministic, so repeatedly move the test case the possibility of result difference that AETG obtains, thus be difficult to guarantee that the combined test set of uses case generating minimizes, and unstable result.The method of the similar AETG of method that the instrument PICT of Microsoft's exploitation adopts, but the Greedy strategy of PICT is deterministic, repeatedly moves coming to the same thing of PICT, but cannot guarantee that equally the combined test set of uses case of its generation minimizes.In the mode of two-dimensional expansion, the people such as Lei have proposed IPO (in-parameter-order) algorithm, this algorithm is mainly for pairwise testing, can only produce at first paired covering array, the people such as Lei expanded this algorithm afterwards, had obtained a kind of algorithm IPOG that can produce any t combined test use-case.Although IPOG algorithm is greedy algorithm, it has adopted the method for exponential complexity to select minimum row to carry out extends perpendicular, therefore still need to spend the plenty of time when processing extensive example.
In order further to reduce combined test use-case quantity, save software test cost.We change into matrix form the software input space, then by the multiplicity in the ranks of test case in judgment matrix, generate the minimum dematrix that covers, and finally generate minimum combined test set of uses case.
Summary of the invention
Although the greedy algorithm efficiency such as PICT recited above, IPOG are higher, the test case quantity generating is many.The present invention is directed to a fairly large number of shortcoming of test case that current method generates, a kind of minimum combination method for generating test case based on matrix multiplicity has been proposed, guaranteeing in the nondecreasing situation of coverage rate that combination covers, make the test case number of generation still less, thereby improve the efficiency of software test, reduce the cost of software test.And the result of the experimental result of this method and existing combined test instrument ACTS and PICT is compared, verified the validity of put forward the methods and the few advantage of test case number of generation.
Design of the present invention is as follows:
Step 1, generates input matrix A according to factor number and each factor value number;
The concrete steps of above-mentioned steps 1 are as follows:
Step 1.1, determines the columns of input matrix A and the maximal value of each factor according to factor number;
Step 1.2, is set to initial value by each factor, and this line is joined in input matrix A;
Step 1.3, the lowest order of factor adds 1, if be greater than this maximal value, forward carry;
Step 1.4, if most significant digit is less than or equal to the maximal value of most significant digit, joins a line just having generated in output matrix A, then forwards step 1.3 to, if most significant digit is greater than the maximal value of most significant digit, and the input matrix A that output obtains, algorithm finishes.
In above-mentioned steps 1.1-step 1.4, the initial value of each factor and span are made following agreement, the initial value of the 1st factor is 100, if the value number of first factor is 3, the span of the 1st factor is 100~102 integer, and maximal value is that the initial value of 102, the second factors is 200, the 3rd initial value is 300, by that analogy.
The concrete steps of above-mentioned steps 2 are as follows:
Step 2.1, initialization dematrix B, joins the first row of the input matrix A obtaining in step 1 in dematrix B;
Step 2.2, is made as the first row of input matrix A when pre-treatment row;
Step 2.3, compares the every a line in pre-treatment row and dematrix B one by one, if multiplicity all meets the demands, current line is joined in dematrix B, then from input matrix A, deletes current line;
Step 2.4, if also there is untreated row in input matrix A, is made as this row when pre-treatment row, forwards step 2.3 to, otherwise output dematrix B, algorithm finishes.
The concrete steps of above-mentioned steps 3 are as follows:
Step 3.1, chooses n factor according to factor coverage n, if the combination of this n factor was not selected, forwards step 3.2 to, otherwise forwards step 3.4 to;
Step 3.2, generates the value of this n factor, if the combination of the value of this n factor was not generated, forwards step 3.3 to, otherwise forwards step 3.1 to;
Step 3.3, whether the combination of the value of n the factor that judgement generates is covered by dematrix B, if not capped, this factor combination is joined in Matrix C, forwards step 3.2 to;
Step 3.4, output matrix C, algorithm finishes.
The concrete steps of above-mentioned steps 4 are as follows:
Step 4.1, the every a line in calculating input matrix A can cover the number of the row in Matrix C, and note, in array A, is marked under the row of covering in array B;
Step 4.2, if not capped line number is greater than 0 in Matrix C, forwards step 4.3 to, otherwise forwards step 4.5 to;
Step 4.3, chooses one maximum in array A, this line is added in dematrix B, then according to the record in array B, these row states in Matrix C are made as and are covered;
Step 4.4, the information of renewal array A and array B, forwards step 4.2 to;
Step 4.5, output dematrix B, algorithm finishes.
Accompanying drawing explanation
Fig. 1 is the minimum combination method for generating test case process flow diagram based on matrix multiplicity.
Fig. 2 is the process flow diagram that generates input matrix A.
Fig. 3 is the process flow diagram that generates preliminary dematrix B.
Fig. 4 is the process flow diagram of generator matrix C.
Fig. 5 is the process flow diagram that generates final dematrix B.
Fig. 6 is scale F=3
10time this method and the IPOG of ACTS instrument, the test case of the IPOG-D of ACTS instrument, PICT instrument generates quantity comparison diagram.
Fig. 7 is scale F=4
8time this method and the IPOG of ACTS instrument, the test case of the IPOG-D of ACTS instrument, PICT instrument generates quantity comparison diagram.
Fig. 8 is scale F=2
12time this method and PICT instrument test case generate quantity comparison diagram.
Fig. 9 is scale F=2
13time this method and PICT instrument test case generate quantity comparison diagram.
Embodiment
In order more clearly to understand the technology contents of the minimum combination method for generating test case that the present invention is based on matrix multiplicity, below in conjunction with accompanying drawing and a case study on implementation, the invention will be further described, it should be noted that, the case study on implementation providing of describing is intended to be convenient to the understanding of the present invention, and to it without any limiting requirement.
As shown in Figure 1, the first step is inputted factor number and each factor value number to the process flow diagram of the minimum combination method for generating test case based on matrix multiplicity that the present invention provides; Second step generates input matrix A, and the 3rd step adopts the method for heavier multiplicity to generate preliminary dematrix B; Whether the dematrix B that the 4th step card generates can cover all factor combinations, will not have chlamydate factor combination to be recorded in Matrix C; The 5th step adopts the thought of greedy algorithm to cover Matrix C, obtains final dematrix B.
First, provide the several concept definitions that arrive involved in the present invention as follows.
Definition 1(test use cases): the total n of input factor that supposes software to be tested is individual, forms input space S set={ f
1, f
2..., f
n, factor f wherein
icomprise a
iindividual value, this factor value set is V
i(1≤i≤n), claims a n tuple t=(v
1, v
2..., v
n) (v
1∈ V
1, v
2∈ V
2..., v
n∈ V
n) be a test case of software to be tested, a plurality of n tuple t are just a test use cases of software to be tested, this test use cases can change into matrix form.
The minimum N factor combination of definition 2(covers matrix): the input space of establishing software systems can be expressed as m * n matrix, note A=(a
ij)
m * n.The factor f of software to be tested is shown in its j list
j, this column element is taken from set V
j(j=1,2 ..., n), i.e. a
ij∈ V
j.If the N that appoints of A is listed as, i.e. i
1, i
2..., i
nrow all meet: V
i1, V
i2..., V
inall N dimension combinations of middle symbol are all listed as in the orderly group of formed N unit and at least occur once at this N, claim that A is that a N factor combination covers matrix.Every a line of A is exactly a test case, and m is the number of test case.If m is the minimum positive integer that guarantees that above-mentioned condition is set up, claim that A is that minimum N factor combination covers matrix.
Definition 3(dematrix): dematrix is m * n matrix, note B=(b
ij) m * n.Wherein m is test case number, only contains a line (individual) test case of first value composition of each factor in initial B.It is that minimum two factors combinations cover matrixes that two factors cover dematrixs, and it is that minimum three factors combinations cover matrixes that three factors cover dematrixs, and N factor to cover dematrix be that minimum N factor combination covers matrix.
Definition 4(multiplicity): establish t
1=(v
1, v
2..., v
n) and t
2=(s
1, s
2..., s
n) be two test cases of software to be tested.If t
1and t
2on h position, respective value is identical, and the multiplicity that claims two test cases is h.
Definition 5(matrix solution): establish t
i=(v
1, v
2..., v
n), and
if t
ithe multiplicity that covers arbitrary row test case in dematrix B with two factors is less than or equal to 1, t
iit is the solution that two factors cover dematrix B.If t
ithe multiplicity h that covers arbitrary row test case in dematrix B with three factors is less than or equal to 2, t
iit is the solution that three factors cover dematrix B.In like manner, if t
ithe multiplicity h that covers arbitrary row test case in dematrix B with N factor is less than or equal to N-1, t
ia solution for N factor covering dematrix B.
The software parameter factor number in the input of the first step of take is below 5, and each factor value number is 2 for case study on implementation, and the implementation process of the inventive method is described.
With reference to Fig. 2, the process that generates input matrix A is as follows:
With reference to Fig. 3, the process that generates preliminary dematrix B is as follows:
Step 301 initialization dematrix B, joins the first row of input matrix A in matrix B; Step 302 is made as current line by the first row in input matrix A; Step 303 is the heavier multiplicity one by one of the row in pre-treatment row and dematrix B, and multiplicity refers to the identical number of factor value in two row of comparison here, and for example 100,200,300,400,500 and 100,200,300,401,501 multiplicity is 3; Step 304 judges whether multiplicity all meets the demands, and meets and forwards step 305 to, does not meet and forwards step 306 to, the multiplicity here meets the demands and refers to: if ask the combination of n factor to cover, multiplicity must be less than n, take n=3 here as example, in matrix A 100,200,301,401,501 with B in unique row 100,200,300,400,500 multiplicities are less than 3, meet the demands; Step 305 joins current line in dematrix B, and in A, deletes current line; Also whether step 306 judge in A and have and forward step 307 to, otherwise algorithm to finish promising processing row; Step 307 is made as current line by the next one in A for processing row.
With reference to Fig. 4, the process of generator matrix C is as follows:
With reference to Fig. 5, the process that generates final dematrix B is as follows:
For the validity of checking this method, collected these two combined test instruments of ACTS and PICT, the IPOG-D algorithm of the IPOG algorithm of this method and ACTS, ACTS and PICT algorithm have been carried out to contrast experiment.First the scale of set of uses case has selected F=3
10and F=4
8, from 2 factor combinations, covering until 6 factor combinations cover respectively, as shown in Table 1 and Table 2, corresponding graphical result is as shown in Figure 6 and Figure 7 for experimental result.Result demonstration, the test case quantity that this method generates is compared obvious minimizing with other several methods, but has reached same combination level of coverage, has verified the few advantage of this method generating test use case quantity.Then the scale selection of set of uses case F=2
12and F=2
13carry out high factor covering, from 2 factor combinations, cover until total factor combination covers (being that 12 factor combinations cover and the combination of 13 factors covers) here respectively, because IPOG and IPOG-D do not support 6 factors to cover above, here compare with the good PICT of effect in IPOG, IPOG-D and tri-kinds of algorithms of PICT, as shown in Table 3 and Table 4, corresponding graphical result as shown in Figure 8 and Figure 9 for experimental result.Result demonstration, this method not only can support high factor to cover, and still keeps the advantage that generating test use case quantity is few.
The scale F=3 of table 1 set of uses case
10time each algorithm test case number comparison of generating
N factor | This method | ACTS(IPOG) | ACTS(IPOG-D) | |
2 | 15 | 15 | 21 | 18 |
3 | 60 | 66 | 71 | 65 |
4 | 208 | 233 | 278 | 231 |
5 | 660 | 761 | 1118 | 735 |
6 | 1966 | 2262 | 2834 | 2164 |
The scale F=4 of table 2 set of uses case
8time each algorithm test case number comparison of generating
N factor | This method | ACTS(IPOG) | ACTS(IPOG-D) | |
2 | 28 | 30 | 32 | 28 |
3 | 112 | 121 | 124 | 139 |
4 | 496 | 670 | 720 | 600 |
5 | 1792 | 2375 | 2816 | 2361 |
6 | 7330 | 9824 | 10848 | 8330 |
The scale F=2 of table 3 set of uses case
12time each algorithm test case number comparison of generating
N factor | This | PICT | |
2 | 9 | 9 | |
3 | 20 | 19 | |
4 | 47 | 48 | |
5 | 85 | 103 | |
6 | 203 | 222 | |
7 | 256 | 428 | |
8 | 742 | 786 | |
9 | 1280 | 1371 | |
10 | 2324 | 2176 | |
11 | 2048 | 2717 | |
12 | 4096 | 4096 |
[0065]the scale F=2 of table 4 set of uses case
13time each algorithm test case number comparison of generating
N factor | This | PICT | |
2 | 9 | 9 | |
3 | 22 | 21 | |
4 | 47 | 52 | |
5 | 108 | 113 | |
6 | 229 | 244 | |
7 | 256 | 485 | |
8 | 869 | 925 | |
9 | 1651 | 1670 | |
10 | 3045 | 2821 | |
11 | 4496 | 4477 | |
12 | 4096 | 6106 | |
13 | 8192 | 8192 |
Claims (5)
1. the minimum combination method for generating test case based on matrix multiplicity, is characterized in that, comprises the following steps:
Step 1, generates input matrix A according to parameter factor number and each factor value number;
Step 2, adopts the method for heavier multiplicity to generate preliminary dematrix B;
Step 3, whether the dematrix B that checking generates can cover all parameter factor combinations, will not have chlamydate factor combination to be recorded in Matrix C;
Step 4, adopts the thought of greedy algorithm to cover Matrix C, obtains final dematrix B.
2. method according to claim 1, is characterized in that: the concrete steps of described step 1 are as follows:
Step 1.1, determines the columns of input matrix A and the maximal value of each factor according to factor number;
Step 1.2, is set to initial value by each factor, and this line is joined in input matrix A;
Step 1.3, the lowest order of factor adds 1, if be greater than this maximal value, forward carry;
Step 1.4, if most significant digit is less than or equal to the maximal value of most significant digit, joins a line just having generated in output matrix A, then forwards step 1.3 to, if most significant digit is greater than the maximal value of most significant digit, and the input matrix A that output obtains, algorithm finishes.
3. method according to claim 1, is characterized in that: the concrete steps of described step 2 are as follows:
Step 2.1, initialization dematrix B, joins the first row of the input matrix A obtaining in step 1 in dematrix B;
Step 2.2, is made as the first row of input matrix A when pre-treatment row;
Step 2.3, compares the every a line in pre-treatment row and dematrix B one by one, if multiplicity all meets the demands, current line is joined in dematrix B, then from input matrix A, deletes current line;
Step 2.4, if also there is untreated row in input matrix A, is made as this row when pre-treatment row, forwards step 2.3 to, otherwise output dematrix B, algorithm finishes.
4. method according to claim 1, is characterized in that: the concrete steps of described step 3 are as follows:
Step 3.1, chooses n factor according to factor coverage n, if the combination of this n factor was not selected, forwards step 3.2 to, otherwise forwards step 3.4 to;
Step 3.2, generates the value of this n factor, if the combination of the value of this n factor was not generated, forwards step 3.3 to, otherwise forwards step 3.1 to;
Step 3.3, whether the combination of the value of n the factor that judgement generates is covered by dematrix B, if not capped, this factor combination is joined in Matrix C, forwards step 3.2 to;
Step 3.4, output matrix C, algorithm finishes.
5. method according to claim 1, is characterized in that: the concrete steps of described step 4 are as follows:
Step 4.1, the every a line in calculating input matrix A can cover the number of the row in Matrix C, and note, in array A, is marked under the row of covering in array B;
Step 4.2, if not capped line number is greater than 0 in Matrix C, forwards step 4.3 to, otherwise forwards step 4.5 to;
Step 4.3, chooses one maximum in array A, this line is added in dematrix B, then according to the record in array B, these row states in Matrix C are made as and are covered;
Step 4.4, the information of renewal array A and array B, forwards step 4.2 to;
Step 4.5, output dematrix B, algorithm finishes.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201310556759.3A CN103631714A (en) | 2013-11-11 | 2013-11-11 | Method for generating minimum combination testing cases based on matrix multiplicity |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201310556759.3A CN103631714A (en) | 2013-11-11 | 2013-11-11 | Method for generating minimum combination testing cases based on matrix multiplicity |
Publications (1)
Publication Number | Publication Date |
---|---|
CN103631714A true CN103631714A (en) | 2014-03-12 |
Family
ID=50212798
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN201310556759.3A Pending CN103631714A (en) | 2013-11-11 | 2013-11-11 | Method for generating minimum combination testing cases based on matrix multiplicity |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN103631714A (en) |
Cited By (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN106844193A (en) * | 2016-12-17 | 2017-06-13 | 福建瑞之付微电子有限公司 | A kind of Systematic Method of flush bonding module cross-beta |
CN107992414A (en) * | 2017-11-28 | 2018-05-04 | 曲明成 | A kind of method that dependence between scheduler module is obtained based on test case |
CN108009082A (en) * | 2017-11-22 | 2018-05-08 | 中国航空工业集团公司西安飞机设计研究所 | A kind of combined test case generation method based on flight management |
CN109815108A (en) * | 2018-12-10 | 2019-05-28 | 江苏大学 | A kind of combined test set of uses case priorization sort method and system based on weight |
CN110515845A (en) * | 2019-08-20 | 2019-11-29 | 义乌工商职业技术学院 | Optimize generation method based on the combined test use-case for improving IPO strategy |
-
2013
- 2013-11-11 CN CN201310556759.3A patent/CN103631714A/en active Pending
Non-Patent Citations (2)
Title |
---|
陈锦富: "基于错误注入的构件安全性测试理论与技术研究", 《中国博士学位论文全文数据库·信息科技辑》 * |
陈锦富等: "软件错误注入测试技术研究", 《软件学报》 * |
Cited By (8)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN106844193A (en) * | 2016-12-17 | 2017-06-13 | 福建瑞之付微电子有限公司 | A kind of Systematic Method of flush bonding module cross-beta |
CN108009082A (en) * | 2017-11-22 | 2018-05-08 | 中国航空工业集团公司西安飞机设计研究所 | A kind of combined test case generation method based on flight management |
CN107992414A (en) * | 2017-11-28 | 2018-05-04 | 曲明成 | A kind of method that dependence between scheduler module is obtained based on test case |
CN107992414B (en) * | 2017-11-28 | 2020-11-17 | 曲明成 | Method for acquiring dependency relationship between process modules based on test cases |
CN109815108A (en) * | 2018-12-10 | 2019-05-28 | 江苏大学 | A kind of combined test set of uses case priorization sort method and system based on weight |
CN109815108B (en) * | 2018-12-10 | 2021-12-21 | 江苏大学 | Weight-based combined test case set prioritization ordering method and system |
CN110515845A (en) * | 2019-08-20 | 2019-11-29 | 义乌工商职业技术学院 | Optimize generation method based on the combined test use-case for improving IPO strategy |
CN110515845B (en) * | 2019-08-20 | 2022-12-30 | 义乌工商职业技术学院 | Combined test case optimization generation method based on improved IPO strategy |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
Bauer et al. | Distributed computation of persistent homology | |
WO2019209628A1 (en) | Method and system for quantum computing | |
CN103631714A (en) | Method for generating minimum combination testing cases based on matrix multiplicity | |
CN101853190A (en) | Data integrity verification method suitable for embedded processor | |
Dieudonné et al. | Circle formation of weak mobile robots | |
CN106295670A (en) | Data processing method and data processing equipment | |
Chen et al. | A high-throughput FPGA accelerator for short-read mapping of the whole human genome | |
WO2017039684A1 (en) | Classifier | |
CN107291898B (en) | FPGA-based MySQ L authentication password recovery system and method thereof | |
Liang et al. | Fast search of the optimal contraction sequence in tensor networks | |
Acar | Effects of the correlation model, the trend model, and the number of training points on the accuracy of K riging metamodels | |
Avila-George et al. | New bounds for ternary covering arrays using a parallel simulated annealing | |
CN106469164A (en) | The out of order treating method and apparatus of data set | |
US20170124027A1 (en) | Method and system for generating an embedding pattern used for solving a quadratic binary optimization problem | |
CN109815108B (en) | Weight-based combined test case set prioritization ordering method and system | |
Dasari et al. | Maximal clique enumeration for large graphs on hadoop framework | |
Bhowmick et al. | An approach for improving complexity of longest common subsequence problems using queue and divide-and-conquer method | |
Zhang et al. | Mining approximate order preserving clusters in the presence of noise | |
Hong et al. | Deadlock prevention policy with behavioral optimality or suboptimality achieved by the redundancy identification of constraints and the rearrangement of monitors | |
CN108304467A (en) | For matched method between text | |
Singh et al. | Reliability optimization using hardware redundancy: a parallel BB-BC based approach | |
Pinar et al. | Combinatorial algorithms for computing column space bases that have sparse inverses | |
EP2439666B1 (en) | Parallel sifting algorithm | |
Kaveh et al. | Eigensolution of augmented graph products using shifted inverse iteration method | |
Meng et al. | A Flexible Global GCRO‐DR Method for Shifted Linear Systems and General Coupled Matrix Equations |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
C10 | Entry into substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
C02 | Deemed withdrawal of patent application after publication (patent law 2001) | ||
WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20140312 |