US20150234007A1 - Generation device, generation method, and program - Google Patents

Generation device, generation method, and program Download PDF

Info

Publication number
US20150234007A1
US20150234007A1 US14/627,409 US201514627409A US2015234007A1 US 20150234007 A1 US20150234007 A1 US 20150234007A1 US 201514627409 A US201514627409 A US 201514627409A US 2015234007 A1 US2015234007 A1 US 2015234007A1
Authority
US
United States
Prior art keywords
test
error
sequence
test vectors
partial sequences
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Abandoned
Application number
US14/627,409
Other languages
English (en)
Inventor
Shunichi Amano
Hisashi Miyashita
Hideki Tai
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
International Business Machines Corp
Original Assignee
International Business Machines Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by International Business Machines Corp filed Critical International Business Machines Corp
Assigned to INTERNATIONAL BUSINESS MACHINES CORPORATION reassignment INTERNATIONAL BUSINESS MACHINES CORPORATION ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: MIYASHITA, HISASHI, TAI, HIDEKI, AMANO, SHUNICHI
Publication of US20150234007A1 publication Critical patent/US20150234007A1/en
Priority to US15/415,331 priority Critical patent/US10394676B2/en
Abandoned legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01R—MEASURING ELECTRIC VARIABLES; MEASURING MAGNETIC VARIABLES
    • G01R31/00—Arrangements for testing electric properties; Arrangements for locating electric faults; Arrangements for electrical testing characterised by what is being tested not provided for elsewhere
    • G01R31/28—Testing of electronic circuits, e.g. by signal tracer
    • G01R31/317—Testing of digital circuits
    • G01R31/3177—Testing of logic operation, e.g. by logic analysers
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/22—Detection or location of defective computer hardware by testing during standby operation or during idle time, e.g. start-up testing
    • G06F11/26—Functional testing
    • G06F11/263—Generation of test inputs, e.g. test vectors, patterns or sequences ; with adaptation of the tested hardware for testability with external testers
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01R—MEASURING ELECTRIC VARIABLES; MEASURING MAGNETIC VARIABLES
    • G01R31/00—Arrangements for testing electric properties; Arrangements for locating electric faults; Arrangements for electrical testing characterised by what is being tested not provided for elsewhere
    • G01R31/28—Testing of electronic circuits, e.g. by signal tracer
    • G01R31/317—Testing of digital circuits
    • G01R31/3181—Functional testing
    • G01R31/3183—Generation of test inputs, e.g. test vectors, patterns or sequences
    • G01R31/318342—Generation of test inputs, e.g. test vectors, patterns or sequences by preliminary fault modelling, e.g. analysis, simulation
    • G01R31/31835—Analysis of test coverage or failure detectability
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01R—MEASURING ELECTRIC VARIABLES; MEASURING MAGNETIC VARIABLES
    • G01R31/00—Arrangements for testing electric properties; Arrangements for locating electric faults; Arrangements for electrical testing characterised by what is being tested not provided for elsewhere
    • G01R31/28—Testing of electronic circuits, e.g. by signal tracer
    • G01R31/317—Testing of digital circuits
    • G01R31/3181—Functional testing
    • G01R31/3183—Generation of test inputs, e.g. test vectors, patterns or sequences
    • G01R31/318371—Methodologies therefor, e.g. algorithms, procedures
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/22—Detection or location of defective computer hardware by testing during standby operation or during idle time, e.g. start-up testing
    • G06F11/26—Functional testing
    • G06F11/273—Tester hardware, i.e. output processing circuits

Definitions

  • the present disclosure generally relates to a generation device, and more particularly relates to a generation method, and a program for generating a test sequence.
  • Patent Literature 1 and Non-patent Literatures 1 and 2 a technique for performing a test that covers a large number of input conditions, using a combinational test has been known (for example, Patent Literature 1 and Non-patent Literatures 1 and 2). Unfortunately, if the combinational test is applied to a system for receiving an ordered sequence, the number of tests could be too large and/or the test could be too long.
  • Patent Literatures 2 and 3 a known method for testing a system for receiving an ordered sequence involves checking a test case by describing a state machine, a temporal logic, and the like (for example, Patent Literatures 2 and 3).
  • Patent Literatures 2 and 3 a known method for testing a system for receiving an ordered sequence.
  • an input space is complicated, checking of completeness is difficult, and detailed specifications about the operation sequence received by the system as a test target are required.
  • Patent Literature 2 has a problem that the test target needs to be reducible to an automaton
  • Patent Literature 3 has a problem that a specific algorithm for generating the test case is not disclosed.
  • the present invention provides a technique for ensuring a large coverage of a test case without defining an operation sequence in detail for a system for receiving an ordered sequence.
  • a first aspect of the present invention provides a device for generating a test sequence to be supplied to a test target, the generation device including: a test vector generation unit for selecting, for each of a plurality of parameters to be included in a test vector, one value from among possible values for the parameter to generate a plurality of test vectors whose combinations of values are different from each other; an extraction unit for extracting, as a plurality of partial sequences each including one or more test vectors, a plurality of portions of a series including the plurality of test vectors output by the test vector generation unit; and a test sequence generation unit for generating a test sequence based on the extracted plurality of partial sequences.
  • the first aspect of the present invention also provides a generation method using the generation device and a program used for the generation device.
  • FIG. 1 illustrates an outline of processing of the present embodiment
  • FIG. 2 illustrates a configuration of a generation device 10 of the present embodiment
  • FIG. 3 illustrates a processing flow by the generation device 10 of the present embodiment
  • FIG. 4 illustrates example parameters of test vectors according to the present embodiment
  • FIG. 5 illustrates an example series of the test vectors according to the present embodiment
  • FIG. 6 illustrates example processing in S 130 by an extraction unit 150 of the present embodiment
  • FIG. 7 illustrates example partial sequences according to the present embodiment
  • FIG. 8 illustrates example processing by the extraction unit 150 according to the present embodiment
  • FIG. 9 illustrates example processing by the extraction unit 150 according to the present embodiment
  • FIG. 10 illustrates example processing by the extraction unit 150 according to the present embodiment
  • FIG. 11 illustrates example processing by a test sequence generation unit 160 according to the present embodiment
  • FIG. 12 illustrates an example test sequence according to the present embodiment
  • FIG. 13 illustrates an example test sequence according to the present embodiment
  • FIG. 14 illustrates an example test sequence according to the present embodiment
  • FIG. 15 illustrates an example test sequence according to the present embodiment
  • FIG. 16 illustrates an example test sequence according to the present embodiment
  • FIG. 17 illustrates an example test sequence according to the present embodiment
  • FIG. 18 illustrates a proof of completeness of a state transition of a test target by the generation device 10 .
  • FIG. 19 illustrates an example hardware configuration of a computer 1900 .
  • FIG. 1 illustrates an outline of processing of the present embodiment.
  • a generation device 10 receives, from a user, a setting file in which specifications of an operation of a test target (SUT) and the length of a test sequence are set, and defines an operation in accordance with the setting file, to thereby generate test vectors according to a pairwise method or the like.
  • the generation device 10 generates a test sequence including a set of operations based on the test vectors.
  • a test driver causes the test target to execute the operations based on the test sequence, and acquires the execution result, to thereby obtain a test result corresponding to the test sequence.
  • the generation device 10 can ensure completeness of operations and states on the basis of the setting file, and can generate the test sequence whose length is suppressed within a realistically executable range.
  • FIG. 2 illustrates a configuration of the generation device 10 of the present embodiment.
  • the generation device 10 generates a test sequence to be supplied to a system as a test target for receiving an ordered sequence, the test sequence being a consecutive operation sequence.
  • the generation device 10 generates a test sequence obtained by coupling test vectors that are operations to be input to the test target.
  • the generation device 10 includes a first reception unit 100 , a test vector generation unit 120 , a second reception unit 130 , a third reception unit 140 , an extraction unit 150 , and a test sequence generation unit 160 .
  • the first reception unit 100 receives, from a user terminal 20 or the like, prohibition rule information for defining combinations of values that cannot be used, for parameters as factors included in a test vector.
  • the first reception unit 100 supplies the received prohibition rule information to the test vector generation unit 120 .
  • the test vector generation unit 120 selects a value of each parameter (that is, a level of each factor) of a test vector to generate test vectors whose combinations of values taken by the parameters are different from each other. For example, according to a pairwise method, the test vector generation unit 120 generates the test vectors that cover all possible patterns taken by the combinations of the values of the parameters.
  • test vector generation unit 120 may generate the test vectors that take combinations of values whose use is permitted, on the basis of the prohibition rule information.
  • the test vector generation unit 120 outputs a series of the generated test vectors to the extraction unit 150 .
  • the second reception unit 130 receives, from the user terminal 20 or the like, commutativity information indicating a condition of values of the parameters under which the order of two or more test vectors is changeable.
  • the second reception unit 130 supplies the commutativity information to the extraction unit 150 .
  • the third reception unit 140 receives, from the user terminal 20 or the like, error condition information indicating a condition of a series including one or more test vectors under which causes an error and makes it impossible to continue a test of the test target.
  • the third reception unit 140 supplies the error condition information to the extraction unit 150 .
  • the extraction unit 150 extracts, as partial sequences each including one or more test vectors, portions of the series including the test vectors output by the test vector generation unit 120 .
  • the extraction unit 150 extracts the mutually commutative partial sequences from the series including the test vectors, on the basis of the commutativity information.
  • the extraction unit 150 may exclude a portion that causes the error from the extracted partial sequences, on the basis of the error condition information, and may further extract partial sequences that do not cause the error.
  • the extraction unit 150 may further generate an error sequence used for a test in which the error is caused, on the basis of the portion that is excluded from the series including the test vectors and causes the error.
  • the extraction unit 150 supplies the partial sequences that do not cause the error and the error sequence to the test sequence generation unit 160 .
  • the test sequence generation unit 160 generates a test sequence on the basis of the extracted partial sequences that do not cause the error. For example, the test sequence generation unit 160 generates a test sequence including a cyclic group of the partial sequences, on the basis of the extracted partial sequences that do not cause the error. The test sequence generation unit 160 may add the error sequence to the test sequence.
  • the test vector generation unit 120 generates a series of test vectors in which pairwise completeness is ensured
  • the extraction unit 150 extracts the mutually commutative partial sequences from the series of the test vectors
  • the test sequence generation unit 160 generates a test sequence including a cyclic group of the partial sequences.
  • FIG. 3 illustrates a processing flow by the generation device 10 of the present embodiment.
  • the generation device 10 executes processing in S 110 to S 160 , to thereby generate the test sequence for the test target.
  • the first reception unit 100 receives, from the user terminal 20 or the like, prohibition rule information for defining combinations of values that cannot be used for the parameters included in a test vector.
  • ID an identification number of a record
  • OP an operation content
  • Serial a personal identification number such as an employee number
  • Name a name
  • ID is one of the numbers 1-3.
  • OP is one of INSERT, UPDATE and DELETE.
  • Serial is one of 100, 101, 102 and Null.
  • Name is one of John, Mike, Alice and Null.
  • the test vector generation unit 120 selects, for each of the parameters to be included in the test vector, one value from among possible values taken by each parameter to generate test vectors whose combinations of values taken by the parameters are different from each other. For example, according to a pairwise method, an orthogonal table, a naive method in which all patterns of combinations of parameter values are covered, or the like, the test vector generation unit 120 generates a list including the test vectors that cover all possible patterns taken by the combinations of the values of the parameters.
  • test vector generation unit 120 may select possible values taken by all the parameters included in the test vector. Alternatively, the test vector generation unit 120 may select possible values taken by only a predetermined number of parameters of all the parameters, and may set a predetermined value to each of the other parameters.
  • the test vector generation unit 120 generates the test vectors that take combinations of values whose use is permitted, on the basis of the prohibition rule information acquired from the first reception unit 100 .
  • the test vector generation unit 120 may set negative to a condition of a parameter whose error is indicated by the prohibition rule information, to generate the test vectors according to the pairwise method.
  • the test vector generation unit 120 generates a test vector in which ID is Delete and Name or Serial is Null, and avoids generating a test vector in which ID is Delete and Name or Serial is other than Null.
  • the test vector generation unit 120 generates eighteen test vectors according to the pairwise method, on the basis of the values of the parameters illustrated in FIG. 4 .
  • the eighteen test vectors illustrated in FIG. 5 include pairs of all values such as “INSERT & 102 ” and “UPDATE & Mike”.
  • the test vector generation unit 120 outputs a series including the generated test vectors to the extraction unit 150 .
  • the extraction unit 150 extracts partial sequences in which one or more test vectors are ordered and arranged, from the series including the test vectors.
  • the second reception unit 130 receives, from the user terminal 20 or the like, commutativity information indicating a condition of values of the parameters under which the order of two or more test vectors is changeable, and supplies the commutativity information to the extraction unit 150 . That is, for two test vectors a i and a j included in a series a of n test vectors (a ⁇ a 1 , a 2 , . . .
  • the second reception unit 130 supplies a condition of parameters of the test vectors a i and a j under which the result obtained by executing in order of a i ⁇ a j and a result obtained by executing in order of a j ⁇ a i are equal to each other, as the commutativity information to the extraction unit 150 .
  • the obtained result is changed by UPDATE (an update process) that is last executed. Because two test vectors having different values in ID (for example, the first test vector [2, INSERT, 102, Mike] and the second test vector [1, UPDATE, 101, Mike]) are processes on different records, even if the two test vectors are executed in the reverse order, the obtained result is not changed.
  • the second reception unit 130 receives, from the outside, commutativity information to the effect that “operations having different values in ID are commutative”, and supplies the commutativity information to the extraction unit 150 .
  • the commutativity information may be created in advance so as to suit specifications of the test target.
  • the extraction unit 150 extracts the mutually commutative partial sequences each including one or more test vectors from the series including the test vectors that are arranged in the listed order, on the basis of the commutativity information. For example, the extraction unit 150 extracts three partial sequences (G 1 , G 2 , and G 3 ) illustrated in FIG. 7 from the series of the test vectors illustrated in FIG. 5 .
  • the test vectors included in one partial sequence (for example, G 1 ) and the test vectors included in another partial sequence (for example, G 2 ) are different in ID from each other, and thus are mutually commutative.
  • the extraction unit 150 determines whether or not the partial sequence Gi that is the i th partial sequence is an empty set. If the partial sequence Gi is empty, the extraction unit 150 executes the processing in S 136 . If the partial sequence Gi is not empty, the extraction unit 150 executes the processing in S 134 .
  • the extraction unit 150 determines whether or not at least one test vector included in the partial sequence Gi and the test vector a m are mutually commutative. If the extraction unit 150 determines that the vectors are mutually commutative, the extraction unit 150 proceeds the processing to S 135 . If no, the extraction unit 150 proceeds the processing to S 136 .
  • the extraction unit 150 executes a process of adding 1 to i. After that, the extraction unit 150 returns the processing to S 133 .
  • the extraction unit 150 executes a process of adding a m to the partial sequence Gi. After that, the extraction unit 150 returns the processing to S 131 .
  • the extraction unit 150 allocates each partial sequence a m to any of the partial sequences Gi. Note that, in the case where a state machine of the test target is defined, the extraction unit 150 may make the upper limit of the number of partial sequences added in S 135 equivalent to the number of states of the test target, and may proceed the processing to S 136 without executing S 135 if the value of i exceeds the number of states.
  • the extraction unit 150 excludes a portion that causes an error from the partial sequences, and extracts partial sequences that do not cause the error.
  • the third reception unit 140 receives, from the user terminal 20 or the like, error condition information indicating a condition of a series including one or more test vectors under which an error that makes it impossible to continue a test of the test target is caused, and supplies the error condition information to the extraction unit 150 .
  • the extraction unit 150 excludes a test vector that causes the error from each of the partial sequences, on the basis of the error condition information, and generates partial sequences (herein, also referred to as non-error sequences) that do not cause the error and each include one or more test vectors.
  • the extraction unit 150 further generates an error sequence used for a test in which the error is caused, on the basis of the portion that is excluded from the series including the test vectors and causes the error. For example, the extraction unit 150 extracts, for each test vector that causes the error in each partial sequence, one or more test vectors that do not cause the error from the first test vector up to the test vector that causes the error, and adds the test vector that causes the error to the one or more test vectors that do not cause the error, and thereby generate an error sequence. The extraction unit 150 may generate error sequences for each test vector that causes the error.
  • FIG. 8 illustrates a non-error sequence G 1 -N, an error sequence G 1 - 1 , an error sequence G 1 - 2 , and an error sequence G 1 - 3 that are generated by the extraction unit 150 from the partial sequence G 1 .
  • the partial sequence G 1 includes six test vectors ( 1 ) to ( 6 ).
  • the test vectors ( 2 ) to ( 4 ) of the six test vectors ( 1 ) to ( 6 ) correspond to an INSERT or UPDATE operation containing Null in Serial or Name, and thus are test vectors that cause the error.
  • the extraction unit 150 excludes the test vectors ( 2 ) to ( 4 ) that cause the error from the test vectors ( 1 ) to ( 6 ) included in the partial sequence G 1 , to thereby generate the non-error sequence G 1 -N including the test vector ( 1 ), the test vector ( 5 ), and the test vector ( 6 ).
  • the extraction unit 150 couples: one or more test vectors that do not cause the error in a range of from the first test vector ( 1 ) to the test vector ( 2 ) that causes the error, that is, the test vector ( 1 ); to each of the test vectors ( 2 ) to ( 4 ) that cause the error.
  • the extraction unit 150 generates: the error sequence G 1 - 1 including the test vectors ( 1 ) and ( 2 ); the error sequence G 1 - 2 including the test vectors ( 1 ) and ( 3 ); and the error sequence G 1 - 3 including the test vectors ( 1 ) and ( 4 ).
  • FIG. 9 illustrates a non-error sequence G 2 -N, an error sequence G 2 - 1 , and an error sequence G 2 - 2 that are generated by the extraction unit 150 from the partial sequence G 2 .
  • the extraction unit 150 excludes test vectors ( 3 ) to ( 4 ) that cause the error from test vectors ( 1 ) to ( 5 ) included in the partial sequence G 2 , to thereby generate the non-error sequence G 2 -N including the test vector ( 1 ), the test vector ( 2 ), and the test vector ( 5 ).
  • the extraction unit 150 further generates: the error sequence G 2 - 1 including the test vectors ( 1 ) to ( 3 ); and the error sequence G 2 - 2 including the test vectors ( 1 ), ( 2 ), and ( 4 ).
  • FIG. 10 illustrates a non-error sequence G 3 -N, an error sequence G 3 - 1 , and an error sequence G 3 - 2 that are generated by the extraction unit 150 from the partial sequence G 3 .
  • the extraction unit 150 excludes test vectors ( 2 ) and ( 6 ) that cause the error from test vectors ( 1 ) to ( 7 ) included in the partial sequence G 3 , and thereby generates the non-error sequence G 3 -N including the test vector ( 1 ), the test vectors ( 3 ) to ( 5 ), and the test vector ( 7 ).
  • the extraction unit 150 further generates: the error sequence G 3 - 1 including the test vectors ( 1 ) and ( 2 ); and the error sequence G 3 - 2 including the test vectors ( 1 ) and ( 3 ) to ( 6 ).
  • the extraction unit 150 supplies the extracted non-error sequences and the extracted error sequences to the test sequence generation unit 160 .
  • test sequence generation unit 160 generates a test sequence having a length equal to or larger than a designated sequence length designated in advance, on the basis of the extracted non-error sequences.
  • the test sequence generation unit 160 further generates a test sequence on the basis of the error sequences.
  • the test sequence generation unit 160 generates, as the test sequences, ( 1 ) repetitions of each of the non-error sequences, ( 2 ) a sequence obtained by coupling ( 1 ) the repetitions of each of the non-error sequences, ( 3 ) cyclic groups of the non-error sequences, and/or ( 4 ) a sequence obtained by coupling the cyclic groups of the non-error sequences.
  • FIG. 11 illustrates an example processing in S 160 by the test sequence generation unit 160 .
  • the operations resulting from concatenating the test vectors in the non-error sequences G 1 -N, G 2 -N, and G 3 -N in order that are generated in S 140 by the extraction unit 150 are respectively defined as O 1 , O 2 , and O 3 , and “.” in FIG. 11 is defined as an operator for coupling operations before and after “.”.
  • the test sequence generation unit 160 generates, as the test sequences, ( 1 ) an operation in which O 1 is repeated a predetermined number of times, an operation in which O 2 is repeated a predetermined number of times, and an operation in which O 3 is repeated a predetermined number of times, ( 2 ) an operation in which O 1 is repeated a predetermined number of times, O 2 is then repeated a predetermined number of times, and O 3 is then repeated a predetermined number of times, ( 3 ) an operation in which O 1 , O 2 , and O 3 are executed in the stated order, an operation in which O 2 , O 3 , and O 1 are executed in the stated order, and an operation in which O 3 , O 1 , and O 2 are executed in the stated order, and ( 4 ) an operation in which O 1 , O 2 , O 3 , O 2 , 03 , O 1 , O 3 , O 1 , and O 2 are executed in the stated order.
  • FIGS. 12 to 14 each illustrates an example test sequence generated by the test sequence generation unit 160 according to the present embodiment.
  • the test sequence generation unit 160 generates a test sequence in which each of the operations O 1 to O 3 respectively corresponding to the partial sequences G 1 -N to G 3 -N is repeated, as ( 1 ) the operation in which each of O 1 to O 3 is repeated the predetermined number of times.
  • the test sequence generation unit 160 generates a test sequence in which three test vectors of [1, UPDATE, 101, Mike], [1, UPDATE, 102, John], and [1, DELETE, NULL, NULL] included in O 1 are repeated a plurality of number of times.
  • the test sequence generation unit 160 generates a test sequence in which three test vectors of [2, INSERT, 102, Mike], [2, UPDATE, 100, Alice], and [2, DELETE, NULL, NULL] included in O 2 are repeated a plurality of number of times.
  • the test sequence generation unit 160 generates a test sequence in which five test vectors of [3, INSERT, 101, Alice], [3, UPDATE, 100, John], [3, UPDATE, 102, Alice], [3, DELETE, NULL, NULL], and [3, INSERT, 100, Mike] included in O 3 are repeated a plurality of number of times.
  • FIGS. 15 to 17 each illustrate another example test sequence generated by the test sequence generation unit 160 according to the present embodiment.
  • the test sequence generation unit 160 generates a test sequence in which O 1 , O 2 , and O 3 respectively corresponding to the partial sequences G 1 -N to G 3 -N are coupled in order, as ( 3 ) the operation in which O 1 to O 3 are executed in cyclic order.
  • the test sequence generation unit 160 generates a test sequence O 1 .O 2 .O 3 in which [1, UPDATE, 101, Mike], [1, UPDATE, 102, John], and [1, DELETE, NULL, NULL] included in O 1 , [2, INSERT, 102 , Mike], [2, UPDATE, 100, Alice], and [2, DELETE, NULL, NULL] included in O 2 , and [3, INSERT, 101 , Alice], [3, UPDATE, 100, John], [3, UPDATE, 102, Alice], [3, DELETE, NULL, NULL], and [3, INSERT, 100 , Mike] included in O 3 are coupled in the stated order.
  • the test sequence generation unit 160 generates a test sequence O 2 .O 3 .O 1 in which [2, INSERT, 102, Mike], [2, UPDATE, 100, Alice], and [2, DELETE, NULL, NULL] included in O 2 , [3, INSERT, 101, Alice], [3, UPDATE, 100, John], [3, UPDATE, 102, Alice], [3, DELETE, NULL, NULL], and [3, INSERT, 100, Mike] included in O 3 , and [1, UPDATE, 101, Mike], [1, UPDATE, 102, John], and [1, DELETE, NULL, NULL] included in O 1 are coupled in the stated order.
  • the test sequence generation unit 160 generates a test sequence O 3 .O 1 .O 2 in which [3, INSERT, 101, Alice], [3, UPDATE, 100, John], [3, UPDATE, 102, Alice], [3, DELETE, NULL, NULL], and [3, INSERT, 100, Mike] included in O 3 , [1, UPDATE, 101, Mike], [1, UPDATE, 102, John], and [1, DELETE, NULL, NULL] included in O 1 , and [2, INSERT, 102, Mike], [2, UPDATE, 100, Alice], and [2, DELETE, NULL, NULL] included in O 2 are coupled in the stated order.
  • test sequence generation unit 160 may add the error sequence received from the extraction unit 150 , to the test sequence.
  • the generation device 10 selects combinations of levels from factors according to the pairwise method or the like to generate a test vector, and treats the test vector as an input to the test target. As a result, the generation device 10 can secure completeness of factors included in a test vector as a unit operation.
  • the generation device 10 applies coupling rules to generated test vectors, to further generate a partial sequence in which the test vectors are ordered based on the commutativity of operations.
  • the generation device 10 can apply an output of a combinational test method such as the pairwise method to the test target for receiving an ordered sequence, without defining a state machine.
  • the generation device 10 generates a test sequence based on a partial sequence from which test vectors corresponding to operations that cause the error are removed, and generates another test sequence for the error portion. As a result, the generation device 10 can separately execute a test of an error process while omitting a useless operation from the test sequences.
  • the generation device 10 applies a cyclic group to a partial sequence from which the error is removed, to generate a test sequence.
  • the generation device 10 can secure completeness of a state transition of the test target using the test sequence of a realistically testable length.
  • a test sequence generated according to a naive method has a length of k k
  • the generation device 10 of the present embodiment can generate a test sequence having a length of approximately k 2 .
  • the generation device 10 may store these pieces of information in a storage device in advance, and may read out the stored pieces of information from the storage device as needed.
  • FIG. 18 illustrates a proof of the completeness of the state transition of the test target by the generation device 10 .
  • the test target has k states and that ⁇ O 1 , O 2 , . . . , Ok ⁇ covers the possibility cases of operations that provoke the test target into causing an error (failure).
  • test vectors in ( 4 ) the sequence (O 1 .O 2 .O 3 .O 2 .O 3 .O 1 .O 3 .O 1 .O 2 ) obtained by coupling the cyclic groups of the non-error sequences by the generation device 10 cover the possibility of provocation of every error (failure), that is, it is proved that the generation device 10 secures the completeness of the state transition of the test target.
  • Q is a set of a finite number of states, and
  • k.
  • ⁇ is a residue class of 1 types of operations received by the test target, and satisfies ⁇ O 1 ⁇ (homomorphism).
  • the transition is defined by ⁇ ⁇ Q ⁇ .
  • T k (k ⁇ 1) (a set of test vectors)
  • T k covers all states of S k,l . This is denoted by T k ⁇ S k,l .
  • a state that is newly added at S k+1,l is represented by q k+1 .
  • a path from S k,l to q k+1 is represented by O x1 , . . . , O xm .
  • FIG. 19 illustrates an example hardware configuration of a computer 1900 that functions as the generation device 10 .
  • the computer 1900 includes: a CPU peripheral unit including a CPU 2000 , a RAM 2020 , a graphics controller 2075 , and a display device 2080 that are connected to one another by a host controller 2082 ; an input/output unit including a communication interface 2030 , a hard disk drive 2040 , and a CD-ROM drive 2060 that are connected to the host controller 2082 by an input/output controller 2084 ; and a legacy input/output unit including a ROM 2010 , a flexible disk drive 2050 , and an input/output chip 2070 that are connected to the input/output controller 2084 .
  • the host controller 2082 connects the RAM 2020 to the CPU 2000 and the graphics controller 2075 that access the RAM 2020 at high transmission rates.
  • the CPU 2000 operates and controls each unit on the basis of programs stored in the ROM 2010 and the RAM 2020 .
  • the graphics controller 2075 acquires image data that is generated by the CPU 2000 and the like on a frame buffer provided in the RAM 2020 , and displays the image data on the display device 2080 .
  • the graphics controller 2075 may include a frame buffer for storing image data generated by the CPU 2000 and the like.
  • the input/output controller 2084 connects the host controller 2082 to the communication interface 2030 , the hard disk drive 2040 , and the CD-ROM drive 2060 that are relatively high-speed input/output devices.
  • the communication interface 2030 makes communications with other devices via a network through wires or radio waves. Moreover, the communication interface functions as hardware for making communications.
  • the hard disk drive 2040 stores therein programs and data used by the CPU 2000 in the computer 1900 .
  • the CD-ROM drive 2060 reads a program or data from a CD-ROM 2095 , and provides the program or data to the hard disk drive 2040 through the RAM 2020 .
  • the ROM 2010 and the flexible disk drive 2050 and the input/output chip 2070 that are relatively low-speed input/output devices are connected to the input/output controller 2084 .
  • the ROM 2010 stores therein a boot program executed at the time of activation of the computer 1900 and/or programs depending on hardware of the computer 1900 .
  • the flexible disk drive 2050 reads a program or data from a flexible disk 2090 , and provides the program or data to the hard disk drive 2040 through the RAM 2020 .
  • the input/output chip 2070 connects the flexible disk drive 2050 to the input/output controller 2084 , and connects various input/output devices to the input/output controller 2084 through, for example, a parallel port, a serial port, a keyboard port, and a mouse port.
  • the programs that are provided to the hard disk drive 2040 through the RAM 2020 are provided by a user in the state where the programs are stored in a recording medium such as the flexible disk 2090 , the CD-ROM 2095 , or an IC card.
  • the programs are read out from the recording medium, are installed onto the hard disk drive 2040 in the computer 1900 through the RAM 2020 , and are executed by the CPU 2000 .
  • the programs that are installed onto the computer 1900 to cause the computer 1900 to function as the generation device 10 include a first reception module, a test vector generation module, a second reception module, a third reception module, an extraction module, and a test sequence generation module. These programs or modules may act on the CPU 2000 and the like to cause the computer 1900 to function as the first reception unit 100 , the test vector generation unit 120 , the second reception unit 130 , the third reception unit 140 , the extraction unit 150 , and the test sequence generation unit 160 .
  • Information processing described in these programs is read onto the computer 1900 to thereby function as the first reception unit 100 , the test vector generation unit 120 , the second reception unit 130 , the third reception unit 140 , the extraction unit 150 , and the test sequence generation unit 160 that are specific means achieved by cooperation of software and the above-mentioned various hardware resources. Then, computation or processing of information suited to the intended use of the computer 1900 according to the present embodiment is achieved by these specific means, whereby a special generation device 10 suited to its intended use is constructed.
  • the CPU 2000 executes a communication program loaded onto the RAM 2020 , and instructs the communication interface 2030 to perform a communication process, on the basis of a processing content described in the communication program.
  • the communication interface 2030 reads out transmission data stored in a transmission buffer region or the like provided on a storage device such as the RAM 2020 , the hard disk drive 2040 , the flexible disk 2090 , or the CD-ROM 2095 , and transmits the transmission data to the network. Otherwise, the communication interface 2030 writes reception data received from the network into a reception buffer region or the like provided on the storage device.
  • the communication interface 2030 may transfer the transmission/reception data with respect to the storage device according to a direct memory access (DMA) method.
  • the CPU 2000 may read out data from the transfer-origin storage device or communication interface 2030 , and may write the data into the transfer-destination communication interface 2030 or storage device, to thereby transfer the transmission/reception data.
  • DMA direct memory access
  • the CPU 2000 reads the entire or a necessary portion from files, databases, or the like stored in an external storage device such as the hard disk drive 2040 , the CD-ROM drive 2060 (the CD-ROM 2095 ), or the flexible disk drive 2050 (the flexible disk 2090 ), onto the RAM 2020 through DMA transfer or the like, and performs various processing on the data on the RAM 2020 . Then, the CPU 2000 writes the processed data back into the external storage device through DMA transfer or the like. In such processing, the RAM 2020 can be considered to temporarily hold the contents of the external storage device, and hence, in the present embodiment, the RAM 2020 , the external storage device, and the like are generally referred to as a memory, a storage unit, a storage device, and the like.
  • Various pieces of information such as various programs, pieces of data, tables, and databases in the present embodiment are stored in such a storage device to be subjected to information processing.
  • the CPU 2000 may hold part of the RAM 2020 on a cache memory, and may read and write data on the cache memory.
  • the cache memory takes on part of the function of the RAM 2020 , and hence, in the present embodiment, the cache memory is also included in the RAM 2020 , the memory, and/or the storage device unless distinctively described.
  • the CPU 2000 performs, on data read out from the RAM 2020 , various processes including various types of computation, information processing, condition determination, and information retrieval/translation that are described in the present embodiment and are designated by a sequence of instructions in a program, and writes the processed data back into the RAM 2020 .
  • the CPU 2000 makes condition determination
  • the CPU 2000 determines whether or not a condition that various variables described in the present embodiment are larger than, smaller than, equal to or larger than, equal to or smaller than, or equal to another variable or a constant is satisfied. Then, if the condition is satisfied (or is not satisfied), the CPU 2000 moves on to a different sequence of instructions, or invokes a sub-routine.
  • the CPU 2000 can retrieve information stored in a file, a database, or the like in the storage device. For example, in the case where entries in each of which an attribute value having a second attribute is associated with an attribute value having a first attribute are stored in the storage device, the CPU 2000 retrieves an entry in which the attribute value having the first attribute coincides with a designated condition, from among the entries stored in the storage device, and reads out the attribute value having the second attribute stored in the retrieved entry, whereby the CPU 2000 can obtain the attribute value having the second attribute associated with the first attribute that satisfies the designated condition.
  • the above-mentioned programs or modules may be stored in an external recording medium.
  • the usable recording medium include an optical recording medium such as a DVD or a CD, a magneto-optical recording medium such as a MO, a tape medium, and a semiconductor memory such as an IC card, in addition to the flexible disk 2090 and the CD-ROM 2095 .
  • a storage device such as a hard disk or a RAM provided to a server system connected to a private communication network or the Internet may be used as the recording medium, and the programs may be provided to the computer 1900 via the network.
  • 10 generation device 20 user terminal, 100 first reception unit, 120 test vector generation unit, 130 second reception unit, 140 third reception unit, 150 extraction unit, 160 test sequence generation unit, 1900 computer, 2000 CPU, 2010 ROM, 2020 RAM, 2030 communication interface, 2040 hard disk drive, 2050 flexible disk drive, 2060 CD-ROM drive, 2070 input/output chip, 2075 graphics controller, 2080 display device, 2082 host controller, 2084 input/output controller, 2090 flexible disk, 2095 CD-ROM

Landscapes

  • Engineering & Computer Science (AREA)
  • General Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Hardware Design (AREA)
  • Quality & Reliability (AREA)
  • Debugging And Monitoring (AREA)
US14/627,409 2014-02-20 2015-02-20 Generation device, generation method, and program Abandoned US20150234007A1 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US15/415,331 US10394676B2 (en) 2014-02-20 2017-01-25 Generation device, generation method, and program

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2014-030275 2014-02-20
JP2014030275A JP5963316B2 (ja) 2014-02-20 2014-02-20 生成装置、生成方法、及び、プログラム

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US15/415,331 Continuation US10394676B2 (en) 2014-02-20 2017-01-25 Generation device, generation method, and program

Publications (1)

Publication Number Publication Date
US20150234007A1 true US20150234007A1 (en) 2015-08-20

Family

ID=53797942

Family Applications (2)

Application Number Title Priority Date Filing Date
US14/627,409 Abandoned US20150234007A1 (en) 2014-02-20 2015-02-20 Generation device, generation method, and program
US15/415,331 Active 2035-03-09 US10394676B2 (en) 2014-02-20 2017-01-25 Generation device, generation method, and program

Family Applications After (1)

Application Number Title Priority Date Filing Date
US15/415,331 Active 2035-03-09 US10394676B2 (en) 2014-02-20 2017-01-25 Generation device, generation method, and program

Country Status (2)

Country Link
US (2) US20150234007A1 (ja)
JP (1) JP5963316B2 (ja)

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2017037914A1 (ja) * 2015-09-03 2017-03-09 三菱電機株式会社 情報処理装置及び情報処理方法
JP2019066987A (ja) * 2017-09-29 2019-04-25 富士通株式会社 解析装置および解析方法
US10394676B2 (en) 2014-02-20 2019-08-27 International Business Machines Corporation Generation device, generation method, and program
CN110291588A (zh) * 2017-02-13 2019-09-27 高通股份有限公司 用于安全关键汽车用例的具有存储的测试模式的现场多核自测试控制器
WO2021171117A1 (en) * 2020-02-28 2021-09-02 International Business Machines Corporation Executing tests in deterministic order

Families Citing this family (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109643818A (zh) 2016-09-02 2019-04-16 昭和电工株式会社 氧化还原流动二次电池及其电极
JP6916327B1 (ja) * 2020-02-28 2021-08-11 三菱電機インフォメーションシステムズ株式会社 派生テスト装置、派生テスト方法、および、派生テストプログラム
JP7106597B2 (ja) * 2020-04-27 2022-07-26 アンリツ株式会社 移動端末試験装置及び移動端末試験方法
CN113466675B (zh) * 2021-05-26 2022-06-21 中国电子科技集团公司第五十四研究所 一种测试向量生成方法
JPWO2022264486A1 (ja) * 2021-06-16 2022-12-22

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20020073374A1 (en) * 2000-09-28 2002-06-13 Danialy Givargis A. Method,system and program product for testing and/or diagnosing circuits using embedded test controller access data
US20030115033A1 (en) * 2001-12-13 2003-06-19 Aitken Robert C. Flow for vector capture
US20030145297A1 (en) * 2002-01-25 2003-07-31 Jean-Francois Cote Method and program product for completing a circuit design having embedded test structures
US20070055665A1 (en) * 2005-08-01 2007-03-08 Youichi Kitahara Sequential pattern extracting apparatus
US20070259256A1 (en) * 2004-11-29 2007-11-08 Jean-Marc Le Canut Systems and methods for detecting and indicating fault conditions in electrochemical cells

Family Cites Families (22)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5050091A (en) * 1985-02-28 1991-09-17 Electric Editor, Inc. Integrated electric design system with automatic constraint satisfaction
US5598344A (en) * 1990-04-06 1997-01-28 Lsi Logic Corporation Method and system for creating, validating, and scaling structural description of electronic device
JP2786780B2 (ja) * 1992-06-26 1998-08-13 松下電器産業株式会社 テスト仕様生成装置および方法
DE69326004T2 (de) * 1993-09-20 1999-11-25 Hewlett Packard Gmbh Testapparat mit grosser Kapazität
US6292765B1 (en) * 1997-10-20 2001-09-18 O-In Design Automation Method for automatically searching for functional defects in a description of a circuit
US8001439B2 (en) * 2001-09-28 2011-08-16 Rambus Inc. Integrated circuit testing module including signal shaping interface
US7017095B2 (en) * 2002-07-10 2006-03-21 International Business Machines Corporation Functional pattern logic diagnostic method
US20080072112A1 (en) * 2006-09-14 2008-03-20 Texas Instruments Incorporated Sequential Scan Technique Providing Reliable Testing of an Integrated Circuit
JP4705060B2 (ja) * 2007-02-27 2011-06-22 三菱電機株式会社 試験仕様生成装置及び試験システム及び試験仕様生成方法及びプログラム
WO2008125998A1 (en) * 2007-04-12 2008-10-23 Nxp B.V. Analog circuit testing and test pattern generation
US8190953B2 (en) * 2008-10-03 2012-05-29 Chakravarthy Sameer H Method and system for selecting test vectors in statistical volume diagnosis using failed test data
JP5120565B2 (ja) 2009-02-20 2013-01-16 三菱自動車工業株式会社 ソフトウェア試験方法
JP2011034274A (ja) * 2009-07-31 2011-02-17 Hitachi Solutions Ltd テスト自動実行システム
US20110046938A1 (en) * 2009-08-19 2011-02-24 Fujitsu Limited Verification apparatus and design verification program
JP5244750B2 (ja) 2009-09-18 2013-07-24 株式会社東芝 テストケース生成装置およびその方法
US8438442B2 (en) * 2010-03-26 2013-05-07 Freescale Semiconductor, Inc. Method and apparatus for testing a data processing system
JP5669630B2 (ja) 2011-03-04 2015-02-12 インターナショナル・ビジネス・マシーンズ・コーポレーションＩｎｔｅｒｎａｔｉｏｎａｌ Ｂｕｓｉｎｅｓｓ Ｍａｃｈｉｎｅｓ Ｃｏｒｐｏｒａｔｉｏｎ テスト・ケース生成方法、プログラム及びシステム
US8661305B2 (en) * 2011-07-10 2014-02-25 Ravishankar Rajarao Method and system for test vector generation
JP5944258B2 (ja) * 2012-07-26 2016-07-05 株式会社東芝 テストケース生成支援装置
CN104078082B (zh) * 2013-03-29 2017-11-14 芯成半导体（上海）有限公司 用于测试存储器件的电路和方法
CA2922701C (en) * 2013-08-28 2021-05-04 Stc.Unm Systems and methods for leveraging path delay variations in a circuit and generating error-tolerant bitstrings
JP5963316B2 (ja) 2014-02-20 2016-08-03 インターナショナル・ビジネス・マシーンズ・コーポレーションＩｎｔｅｒｎａｔｉｏｎａｌ Ｂｕｓｉｎｅｓｓ Ｍａｃｈｉｎｅｓ Ｃｏｒｐｏｒａｔｉｏｎ 生成装置、生成方法、及び、プログラム

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20020073374A1 (en) * 2000-09-28 2002-06-13 Danialy Givargis A. Method,system and program product for testing and/or diagnosing circuits using embedded test controller access data
US20030115033A1 (en) * 2001-12-13 2003-06-19 Aitken Robert C. Flow for vector capture
US20030145297A1 (en) * 2002-01-25 2003-07-31 Jean-Francois Cote Method and program product for completing a circuit design having embedded test structures
US20070259256A1 (en) * 2004-11-29 2007-11-08 Jean-Marc Le Canut Systems and methods for detecting and indicating fault conditions in electrochemical cells
US20070055665A1 (en) * 2005-08-01 2007-03-08 Youichi Kitahara Sequential pattern extracting apparatus

Cited By (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10394676B2 (en) 2014-02-20 2019-08-27 International Business Machines Corporation Generation device, generation method, and program
WO2017037914A1 (ja) * 2015-09-03 2017-03-09 三菱電機株式会社 情報処理装置及び情報処理方法
JPWO2017037914A1 (ja) * 2015-09-03 2018-03-08 三菱電機株式会社 情報処理装置及び情報処理方法
CN110291588A (zh) * 2017-02-13 2019-09-27 高通股份有限公司 用于安全关键汽车用例的具有存储的测试模式的现场多核自测试控制器
JP2019066987A (ja) * 2017-09-29 2019-04-25 富士通株式会社 解析装置および解析方法
WO2021171117A1 (en) * 2020-02-28 2021-09-02 International Business Machines Corporation Executing tests in deterministic order
US11182282B2 (en) 2020-02-28 2021-11-23 International Business Machines Corporation Executing tests in deterministic order
CN115176233A (zh) * 2020-02-28 2022-10-11 国际商业机器公司 以确定性顺序执行测试
GB2609110A (en) * 2020-02-28 2023-01-25 Ibm Executing tests in deterministic order
AU2021227739B2 (en) * 2020-02-28 2023-11-23 International Business Machines Corporation Executing tests in deterministic order

Also Published As

Publication number Publication date
JP5963316B2 (ja) 2016-08-03
JP2015156076A (ja) 2015-08-27
US10394676B2 (en) 2019-08-27
US20170132103A1 (en) 2017-05-11

Similar Documents

Publication Publication Date Title
US10394676B2 (en) Generation device, generation method, and program
US10692065B2 (en) Using a mixture model to generate simulated transaction information
US10489604B2 (en) Searchable encryption processing system and searchable encryption processing method
CN102611692B (zh) 多承租人数据中心中的安全计算方法
WO2017020590A1 (zh) 一种芯片验证方法和装置、设备、存储介质
US10282312B2 (en) Integrated circuit, method for protecting an integrated circuit and computer program product
US11593362B2 (en) Secure aggregate function computation system, secure computation apparatus, secure aggregate function computation method, and program
JP6689283B2 (ja) 装置指紋をインターネット装置に割り当てるための方法及び機器
CN111401572B (zh) 基于隐私保护的有监督特征分箱方法及装置
CN112514349B (zh) 利用对加密匹配索引进行的精确和模糊匹配来检测重复
EP3218811A1 (en) Testing insecure computing environments using random data sets generated from characterizations of real data sets
US8813036B2 (en) Visual representation of a difference between Cartesian product models
Azriel et al. Revealing on-chip proprietary security functions with scan side channel based reverse engineering
EP3675086A1 (en) Registration terminal, search terminal, search server, search system, registration program, and search program
CN111221690B (zh) 针对集成电路设计的模型确定方法、装置及终端
CN111949655A (zh) 表单显示方法、装置、电子设备和介质
US20140289178A1 (en) Information processing apparatus, information processing method and program
CN111427759A (zh) 测试数据生成方法及装置、电子设备和计算机可读介质
EP3364592A1 (en) Method for generating a prime number for a cryptographic application
JP5875717B1 (ja) 乱数生成装置、乱数生成方法、およびプログラム
US10540469B2 (en) Verifying sequential equivalence for randomly initialized designs
CN111538663A (zh) 测试用例的生成方法、装置、计算设备以及介质
JP2019101723A (ja) 試験シナリオ生成装置、試験システム、試験シナリオ生成方法及び試験シナリオ生成プログラム
US9274173B2 (en) Selective test pattern processor
US9477800B1 (en) System, method, and computer program product for automatically selecting a constraint solver algorithm in a design verification environment

Legal Events

Date Code Title Description
AS Assignment

Owner name: INTERNATIONAL BUSINESS MACHINES CORPORATION, NEW Y

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:AMANO, SHUNICHI;MIYASHITA, HISASHI;TAI, HIDEKI;SIGNING DATES FROM 20150216 TO 20150220;REEL/FRAME:034994/0981

STCB Information on status: application discontinuation

Free format text: ABANDONED -- FAILURE TO RESPOND TO AN OFFICE ACTION