JP2010044622A - Information processor test program and method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a technique capable of enhancing the comprehensiveness of a test such as a competition test of a device to be tested. <P>SOLUTION: This test method has processing for generating a test data using a random number (S20), processing for generating an expected value of a test result (S21), processing for starting up an assertion checker for monitoring the test (S22), processing for executing the test for the device to be tested (S23), processing for accumulating monitored information by the assertion checker (S25), processing for outputting the test result (S27), processing for collecting and analyzing the monitored information, to update coverage information (S26, S32, S33), and processing for regulating a content of the test by the test data, based on the coverage information (S34) or the like. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to a test method for a function (logic) of a logic circuit or the like in an information processing apparatus such as a semiconductor integrated circuit (IC), and in particular, a method of performing a random test using a random number. And relates to a method of testing a plurality of elements (such as a processor) of a device under test.

  The information processing apparatus testing method in the prior art is as follows. A test is performed on a logic circuit or the like constituting a logical control unit of an information processing apparatus (hereinafter referred to as a device under test). The device under test of this test has a configuration including a processor (CPU), a storage device (memory), and peripheral devices such as an input / output device. In addition, a device (system) such as a bus (system bus) that connects these elements and enables communication is included.

  In the above test, test items are extracted by a tester (for example, considered and determined based on a hardware manual). If this test item can cover all the logic circuit operations, the accuracy of the test is completely ensured.

  However, since the current information processing apparatus has a complicated logical configuration, it is difficult to perform a test by extracting all the test items, and there is a possibility of test item extraction omission or execution omission. Therefore, even if not all tests, it is required to improve the accuracy of the test as the second best.

  Therefore, conventionally, a test method (random number test method) for performing a random test using a random number is used. That is, with respect to a plurality of elements and a plurality of test instructions and parameters, a lot of test contents (test data) are randomly generated using random number data and applied to the test. This compensates for test item extraction omission and execution omission.

  Regarding the above random number test method and the like, there are conventional examples as follows.

  (1) Japanese Patent Laid-Open No. 7-253901 (Patent Document 1) discloses a technique for repeatedly generating a test instruction sequence using random number data as input and causing the information processing apparatus to execute the instruction sequence to test the apparatus. Are listed. In this technology, when the next test instruction sequence is generated with random number data as input, a test instruction with a low occurrence frequency is generated at random, using random number data as input, in the repeatedly generated test instruction sequence Is inserted in the test instruction sequence.

  (2) In addition, in Japanese Patent Application Laid-Open No. 2005-108007 (Patent Document 2), in an LSI design verification apparatus and method, in the middle stage of RTL (register transfer level) simulation or in an LSI circuit to be verified, It is described that an assertion check means for verifying whether or not they are consistent by extracting an operation history in the operation simulation is described. This technique is a method (test method using assertions) in which a device for verification (assertion check means) is incorporated in the logic in order to detect a contradiction in the logic.

(3) JP 2002-149440 A (Patent Document 3) aims to suppress a decrease in verification efficiency of logic simulation in random number verification (random number test method) in a logic circuit verification method and apparatus. It is described that it is determined whether or not to execute verification. This technique accumulates and updates verified pattern history information, which is useful for test execution judgment and failure analysis.
Japanese Patent Laid-Open No. 7-253901 JP 2005-108007 A JP 2002-149440 A

  In the above-described prior art, in order to improve the accuracy of the test, in particular, in the configuration including a plurality of elements (CPU, peripheral device, storage device, etc.) and system bus in the device under test, It is desirable to exhaustively test parallel (simultaneous parallel) operation.

  For that purpose, it is necessary to strengthen tests for individual elements (individual tests) such as a test for CPU, a test for peripheral devices, and a data transfer test to a storage device, that is, a competitive test. It is done.

  Therefore, a high load was applied to each element by combining the instruction execution of the individual test of each element above, and by complicated combination considering timing conditions such as instruction issue timing and start timing of peripheral devices via the system bus. Testing under conditions (competitive testing) is important.

  In the above-described prior art example, for example, in the random number test method, tests using various combinations of algorithms are realized by utilizing the characteristics of random numbers.

  In the test of the elements of the device under test, especially complex tests that must be aware of the operation of many elements such as CPUs and peripheral devices, there is a method of generating individual tests for each of those elements and simply executing them. It was adopted. In addition, the competitive operation test for a plurality of elements has been performed to some extent by executing a test based on a combination of elements in parallel.

  However, the generation of individual tests for each of these elements is independent and there is no correlation between the tests. Therefore, in particular, there is a lack of completeness (in other words, coverage) related to a test of competing operations (parallel operations) of a plurality of elements, and there is a possibility that a test leak or the like may occur. There is also a possibility that the randomness may be biased. In addition, the degree of completeness is unclear, and it is unclear how far the test was actually achieved or its effectiveness and sufficiency.

  Conventionally, it has been difficult to comprehensively test competing operations by combining elements, and it cannot be denied that one would expect chance or time.

  The technique of Patent Document 1 relates to a random number test method, but does not relate to an assertion (assertion check method), timing (timing test method), or a method using coverage. The technique of Patent Document 2 relates to an assertion (assertion check method), but does not relate to a random number test method or the like. The technique of Patent Document 3 relates to a random number test method, but is not a method for adjusting the test content (a method using coverage).

  The present invention has been made in view of the above problems, and an object of the present invention is to provide a technique capable of enhancing the completeness of a test such as a competition test of an information processing apparatus (device under test). .

  Of the inventions disclosed in the present application, the outline of typical ones will be briefly described as follows. In order to achieve the above object, the present invention provides a method (information processing) for testing a logic circuit or the like (device under test) of an information processing device (IC or the like) by logic simulation or FPGA (Field Programmable Gate Array) or the like. Device test method), a computer for performing the processing work, and a program (information processing device test program), which uses a random number test method, an assertion check method, etc., and has the following configuration To do.

  This test method introduces quantitative indicators and means that are the basis for judging test completeness and failure. This test method not only monitors (assertion check) tests such as the competitive operation status of multiple elements (logical devices) of the device under test, but also manages the comprehensiveness of the test based on the monitoring information. The coverage information is updated, and a mechanism for driving adjustment of test contents (test data) is provided by feedback (reflection) based on the updated coverage information. Specifically, the coverage information is information (event information) for managing the occurrence of a predetermined signal condition, information (timing information) for managing the operation timing of elements, and the like. As a result, it is possible to automatically generate a competitive operation in each test in a plurality of tests and to prevent test leakage, thereby improving the test coverage.

  (1) This method and the corresponding program specifically have the following processing steps.

  (A) A processing step of generating test data (test instruction sequence (group) and parameters thereof) corresponding to the test content using a random number by a test program.

  (B) A processing step of generating an expected value of a test result by executing the test data on a logic simulator.

  (C) A processing step of starting an assertion checker, which is an assertion program prepared for monitoring the test (assertion check), immediately before the start of the test.

  (D) A processing step for executing a test based on the test data simultaneously on a plurality of elements such as a processor, a peripheral device, and a storage device in the device under test.

  (E) A processing step of performing monitoring processing by the assertion checker during execution of the test and accumulating monitoring information based thereon. The assertion check (monitoring) includes extraction (trace) of a history of operations such as activation of each element.

  (F) A processing step of comparing the execution result of the test with the expected value and outputting the result.

  (G) A processing step of collecting the monitoring information, analyzing the collected monitoring information, and updating coverage information, which is information for managing test coverage and the like, using the analysis result.

  (H) A processing step of adjusting the test data (the next test content) based on the analysis result or the updated coverage information (adjustment of at least one of a test instruction sequence or a parameter). If there is no need, the test content is not changed and a new test is generated.

  (I) A processing step of executing a new test (next test) by the adjustment.

  In the processing steps (H) and (I), the test execution result (contents of coverage information) is reflected by feedback on the test data generated in the previous test. Thereby, for example, a test in which the same test instruction is reused and only its parameters are adjusted is generated.

  The monitoring information includes information such as the operation time and the number of times of the elements to be monitored. Further, the coverage information includes event information related to occurrence of a predetermined signal condition in the assertion. The event information includes the number and frequency of occurrence of the signal condition, the number of adjustments of the test, and the like.

  (2) Further, in the processing step of updating the coverage information of (G) above in (1), the (A), (H) The contents (history) of the test data generated by the above and the timing information regarding the simultaneous parallel operation (competitive operation) of a plurality of elements are added. The timing information includes information such as element operation timing (start time, end time, execution time), contention time, contention frequency, and contention rate of peripheral devices operating at the same time. In particular, using the timing information, a portion having a low competitiveness among a plurality of elements is selected as an adjustment target.

  (3) Further, in (1) or (2) above, in the adjustment process step of the test content in (H) above, the elements to be adjusted (coverage information and further random numbers are used in combination) Select the test instructions and parameters, etc.) and make adjustments. In the determination for this purpose, an untested part or a part with low competitiveness is selected preferentially, and test data for executing the test is generated.

  Among the inventions disclosed in the present application, effects obtained by typical ones will be briefly described as follows. According to the present invention, it is possible to improve the completeness of a test such as a competitive test of a device under test.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. Note that components having the same function are denoted by the same reference symbols throughout the drawings for describing the embodiment, and the repetitive description thereof will be omitted.

<Overview>
In this embodiment, as shown in FIG. 1 to FIG. 11, particularly FIG. 4, etc., a test such as a competition test is performed using the monitoring information by the test assertion check and the coverage information updated based on the monitoring information. The test contents are changed by feedback (for example, parameter adjustment).

<1: Information processing system>
FIG. 1 shows a basic configuration example of an information processing system (logic simulation environment) including an information processing apparatus (EWS) that is a computer to which the test method and program of the present embodiment are applied. This system includes a general-purpose information processing apparatus such as an EWS (engineering workstation) 100 and a console terminal 200 connected thereto. The tester (U) operates the console terminal 200 which is an information processing terminal device used for input / output processing, and information processing in the EWS 100 is performed according to the operation.

  The EWS 100 includes hardware such as a control unit 101 (CPU and the like), a storage unit 102 (memory such as RAM), a communication unit 103 (network interface and the like), an auxiliary storage unit 104 (HDD and the like), an input unit 105, a display unit 106, and the like. Is provided. The EWS 100 includes a logic simulator 10 realized by the control unit 101 or the like. On the logic simulator 10, a program such as a test program 30 (random number test program) is executed. The test program 30 corresponds to the program of the present embodiment.

  The logic simulator 10 is a comprehensive environment in which a target system (device under test 20) can perform a logic simulation using a model in which all elements such as a processor and a memory are connected. The logic simulator 10 can artificially operate the hardware operation of the target system by the logic simulation process. In the logic simulation by the logic simulator 10, the device under test 20 is logically configured. In the logic simulation process on the logic simulator 10, a test (random number test) process using the test program 30 is performed on the device under test 20 with respect to its processing function and competitive operation.

  The control unit 101 realizes processing such as logic simulation by processing the data read from the auxiliary storage unit 104 onto the storage unit 102. The communication unit 103, the input unit 105, the display unit 106, and the like have an interface with the console terminal 200, receive input information from the keyboard and mouse of the console terminal 200, perform input processing, and output information to the console terminal 200. For example, output processing to a display screen or a printer is performed.

  The logic simulator 10 includes an OS and other programs in addition to the test program 30. In the logic simulator 10, the test program 30 is executed on the device under test 20 on the OS to supply test resources. The test resource is a register value or the like.

  The device under test 20 is described as a logical file in a hardware description language (HDL) such as VHDL. The logical file is created by a designer or the like. The tester (U) causes the logic simulator 10 of the EWS 100 to be executed from the console terminal 200 and inputs the logic file of the device under test 20 to the logic simulator 10. The device under test 20 is a logic that configures each element such as a processor to be tested, which will be described later.

  The test program 30 generates test data 50 for each test, and includes a program that generates random test contents using random numbers. The device under test 20 is tested using the test data 50. The test for the device under test 20 is repeatedly performed as many times as necessary.

  Further, in association with the test program 30 (test data 50), an assertion checker 6 for assertion check (monitoring) is described in the device under test 20 by HDL. The assertion checker 6 is described and controlled by an assertion checker control program 60 described later. The assertion checker 6 is a logic capable of monitoring and detecting a specific signal and signal condition relating to the test and outputting information (assertion monitoring information 31 to be described later) representing it.

  The logic simulator 10 stores the assertion monitoring information 31 and information (coverage information 32 to be described later) for managing the completeness (range) of the test created / updated based on the assertion monitoring information 31. It has a function and a function to change the test contents using the information.

  As a result, the system guides the policy for changing the test contents, and feeds back (reflects) the test results (coverage information 32) to the test contents that have already been generated and executed. Can be generated and executed. By using the functions of this system, the tester can easily guide the policy for changing the test contents (in general, improving the completeness). In addition, the test contents can be automatically changed by feedback according to a predetermined policy. In this system, it is possible to preferentially execute a test on a part of the system bus 5 where there is not much competition. As a result, it is possible to automatically generate a competing operation in each test in a plurality of tests and prevent test leakage and the like.

<2: Device under test>
FIG. 2 shows a configuration example of the device under test 20 on the logic simulator 10 and a state of the test program 30 (test data 50) at the time of executing the test in relation thereto. In FIG. 2, for the sake of explanation, the physical configuration (storage unit 102) of FIG. 1 is connected to the device under test 20 that is a logical configuration. The device under test 20 includes a processor (CPU) 1, a peripheral device 2 (input / output device (IO), etc.), a storage device (memory) 3, etc., and a system bus 5 for connecting them.

  The processor 1 is logic such as a CPU or a microprocessor that processes instructions. The storage device 3 is the logic of a memory (main memory, secondary memory, storage, etc.) that stores instructions and data. The processor 1 reads out and executes an instruction (test instruction) from the storage device 3 and stores the result in the storage device 3. The peripheral device 2 is logic such as an input / output device (IO), and communicates with the processor 1 and the storage device 3. The system bus 5 controls the flow of data and signals between elements such as the processor 1, the peripheral device 2, and the storage device 3. Although only one element such as the processor 1 is shown in FIG. 2, a configuration in which a plurality of each element exists (for example, a multiprocessor system) is also possible.

  On the other hand, the physical storage unit 102 stores data such as the test program 30, the assertion monitoring information 31, and the coverage information 32. The test program 30 automatically generates test data 50 when each test is executed, and places the test data 50 at an arbitrary address in the storage device 3. The test data 50 arranged on the storage device 3 is sequentially fetched and executed by the processor 1, thereby executing the test. The test data 50 includes a test instruction sequence 51 (code) and its parameters 52 (variable data and the like). The test instruction sequence 51 includes a test instruction group. The parameter 52 is substituted into the test instruction sequence 51. The test instruction includes general instructions such as data read and write.

  Moreover, it has the assertion checker 6 (6-1, 6-2, 6-3) connected with each element (1, 2, 3) of these test objects. The assertion (assertion checker 6) describes a function specification, design intention, premise, and the like, and monitors whether the logical description of the target element is as per the specification. Each assertion checker 6 (6-1, 6-2, 6-3) is logically connected in such a manner as to be interposed in a signal line between the target element and the system bus 5 as shown in FIG. (Embedded or associated in the logic of the target element) The assertion checker 6 monitors a control signal in the bus protocol of the system bus 5. Each assertion checker 6 has a function of monitoring a corresponding element (its signal) and extracting (tracing) an operation history. The assertion checker 6 activated by the test program 30 at the time of the test is between each element, that is, between the processor 1 and the system bus 5, between the system bus 5 and the peripheral device 2, and between the storage device 3 and the system bus 5. Each signal issued between and is monitored in parallel. The assertion checker 6 outputs assertion monitoring information 31 (monitoring information), which is information extracted by monitoring, and stores (accumulates) it at an arbitrary address prepared in the storage unit 102 (or storage device 3).

<3: Logic simulator, test program>
FIG. 3 shows a configuration including the correlation of each processing unit in the characteristic test control unit of the present embodiment, centering on the logic simulator 10 and the test program 30. On the logic simulator 10, the test of the device under test 20 as shown in FIG. 2 is monitored by the assertion checker 6, and the assertion monitoring information 31 and the coverage information 32 are accumulated in the storage unit 102.

  The test program 30 includes a test generation processing unit 11, an expected value generation processing unit 12, a test execution processing unit 13, a test result comparison / output processing unit 14, an assertion activation processing unit 15, a monitoring information collection / analysis processing unit 16, and coverage information. It has processing units such as an update processing unit 17 and a test data adjustment processing unit 18. Each processing unit exchanges information data with each other as indicated by a line.

  The processing units 11 to 14 also exist in the conventional method, and are functions that presuppose in this embodiment. These processing units perform processing related to the test generation / execution cycle in each test, generation of test contents (test data 50), generation of expected values of the test, execution of the test, and the test It performs basic processing such as output of results. Further, the processing units such as 15 to 18 are processing units necessary for realizing this feature, and perform processing related to assertion, coverage, and feedback.

  The test generation processing unit 11 creates each test from the beginning. The test generation processing unit 11 generates test data 50 (test instruction sequence 51, parameter 52).

  The expected value generation processing unit 12 generates an expected value as a result of the test based on the test data 50.

  The test execution processing unit 13 executes a test on the device under test 20 such as the processor 1, the peripheral device 2, the storage device 3, and the system bus 5 under the monitoring of the assertion checker 6. The test execution processing unit 13 places the test data 50 generated by the test generation processing unit 11 in the storage device 3, and causes the processor 1 to execute a test using the test data 50.

  The test result comparison / output processing unit 14 compares the test result based on the test data 50 with the expected value generated by the expected value generation processing unit 12 and outputs the result.

  The assertion activation processing unit 15 performs a process of activating the assertion checker 6 at the start of the test. The assertion activation processing unit 15 activates the assertion checker 6 (6-1, 6-2, 6-3) by writing to a specific address of the storage device 3 at the start of the test.

  The monitoring information collection / analysis processing unit 16 collects the assertion monitoring information 31 related to the test in the storage unit 102 based on the output of the assertion checker 6 at the end of the test based on the test data 50 (current test). Then, processing for analyzing the coverage information 32 is updated.

  The coverage information update processing unit 17 manages the coverage information 32 and, based on the analysis result of the assertion monitoring information 31, coverage information as information used for generating (changing) the next test content (test data 50). 32 is updated. In this update, the additional information (analysis result data) is merged with the past (accumulated) coverage information 32 in the storage unit 102.

  The test data adjustment processing unit 18 has a role of adjustment with respect to the test data generation processing unit 11. Based on the coverage information 32, the test data adjustment processing unit 18 generates the test data 50 (the test instruction sequence 51 or particularly the parameter 52) of the generated test (previous test content) in order to generate the test content after the next time. Perform the adjustment process. For example, the test data adjustment processing unit 18 reuses the test command sequence 51 of the generated test data 50 and creates (regenerates) only the parameter 52 for the test command sequence 51 for adjustment.

  For example, the test data adjustment processing unit 18 controls the adjustment of an untested part or a part with low competitiveness based on the coverage information 32 according to a policy for increasing the competitiveness of a plurality of elements. The test data 50 is adjusted to be a candidate. For example, in particular, the execution timing of each element is adjusted so as to be in a competitive operation state. In some cases, the test instruction sequence 51 is changed, the parameter 52 is changed, or both are changed.

<4: Test program, assertion checker>
FIG. 4 shows a schematic processing flow of the test program 30 and the assertion checker 6 (S represents a processing step). In this flow, the processing cycle of one test is shown. Each test is executed by the test execution processing unit 13 of FIG.

  Information input operations by the examiner (U) include setting information input and insertion of a start instruction (execution command) for starting a test. Other than that, the operation is basically realized by automatic processing by a computer. .

  First, in test data generation processing S20, the test generation processing unit 11 generates test data 50 for the device under test 20 (each element) using random numbers. The test data 50 includes a test instruction sequence 51 (test instruction group) and a parameter 52 attached thereto. For example, an element to be tested, a command, a parameter, and the like are selected by the random number.

  Regarding the test for the processor 1 (CPU), as resources to be tested, resources such as general-purpose registers, control registers, interrupt registers, floating-point registers, TLB (Translation Look-aside Buffer), caches, and other resources existing in each CPU There are CPU common area and individual area data on the apparatus 3.

  The test instruction sequence 51 for the processor 1 is generated by random numbers and is created while simulating each instruction. This has the purpose of creating an efficient and effective instruction sequence, and the instruction sequence is created while grasping changes in the program counter and each register.

  The number of registers is limited, and the larger the number of test instructions related to the test instruction sequence 51, the more registers are used. When a register is simply used with random numbers, overwriting to the same register can frequently occur due to the use of biased registers. In that case, the value carried over on the register may be lost by overwriting. As a result, the previous test may be wasted. In order to avoid this, a register (for example, register A) whose value has been temporarily changed by an instruction (test instruction) is used as a source register, and the value is transferred to another register (for example, register B). As a result, even if the value changes and then, for example, the register A is overwritten, it becomes evidence that the instruction sequence (test instruction sequence 51) has been executed in order without omission.

  Regarding the test for IO (peripheral device 2), the test target resources include selection of a channel / device to be tested, registers for each channel / device, transfer source / transfer destination addresses and data, interrupt registers, and the like. . For these, test data 50 is created using random numbers.

  Regarding the generation of the test data 50, all of them are basically generated by random numbers so as to be variable values, but candidates are prepared as fixed values when there are restrictions or limitations due to the specifications of the device under test 20. . For example, in the present system, this fixed value can be selectively set by a tester. In addition, depending on the content of the test, there is a case where assignment with a weight (bias) is given not uniformly but with respect to generation by random numbers. In this system, the tester can selectively set the weighting of the random number. Various settings in this system can be performed by an information input operation while the examiner looks at the screen from the console terminal 200 or the like.

  With respect to the test instruction sequence 51 for the peripheral device 2, an instruction sequence suitable for the purpose of use, such as a start-only instruction sequence serving as a trigger for moving the peripheral device 2 and an end-determination dedicated instruction sequence, is arranged at an arbitrary location.

  Next, in the expected value generation processing S21, the expected value generation processing unit 12 generates an expected value corresponding to the test content based on the test data 50. In this case, an expected value corresponding to the test content is created for each device under test 20 by a software simulator built in the test program 30.

  The expected value generation method related to the processor 1 is created in accordance with the CPU architecture, the IEEE standard, etc., and all the instruction functions related to the processor 1 can be supported by this system (expected value generation is possible). .

  With regard to the expected value generation method for the peripheral device 2, a dedicated hardware manual is implemented as the expected value generation process S21.

  Next, before the test is started (executed), the assertion start processing unit 15 starts the assertion checker 6 in the assertion start processing S22. The activation method is to write a predetermined bit to a predetermined address of the storage device 3. By this activation, the assertion checker 6 becomes a hardware monitoring operation state completely independent from the test program 30.

  Subsequently, in the test execution process S23, the test execution processing unit 13 starts a test based on the test data 50. When the test is started, the operation of various signals is started by each element of the device under test 20.

  On the other hand, the assertion checker 6 continuously monitors the flow of the test operation on the bus (system bus 5) in the target test operation monitoring process S24. During the monitoring, the assertion checker 6 monitors the signal condition being monitored (for example, particularly the competition condition) and the signal under test in monitoring the target signal such as the request signal and the address signal on the bus (system bus 5). It is determined whether or not the state matches. If they match, in the assertion monitoring information extraction process S <b> 25, the assertion checker 6 extracts (creates) the assertion monitoring information 31 relating to the occurrence (event and timing) of the match and stores it in the storage unit 102.

  The assertion monitoring information 31 includes the operation time and number of times for each element to be monitored (peripheral device 2 and the like), the time and number of simultaneous parallel operations (competitive operation) of a plurality of elements, and the establishment (match) of the signal condition of the monitoring target There are times and times.

  After the test of all the devices under test 20 in S23 is completed, in the monitoring information collection process S26, the monitoring information collection / analysis processing unit 16 collects the assertion monitoring information 31 stored in the storage device 3, and the coverage information Prepared as information for an input value to the update process S33.

  At the end of the test of S23, in the test result comparison / output process S27, the test result comparison / output processing unit 14 performs a process of comparing the test result with the expected value. In this process, all of the test instruction sequence 51 and result resources at the end of the test (at the end of the operation of the element) are collected and compared with the expected value data generated in the expected value generation process S21. To do.

  Based on the comparison result, the normality of the tests of all the devices under test 20 is checked in S30. If even one error is detected in the comparison result, an error message is output in S31. For example, information such as the error message is displayed on the screen of the console terminal 200 and stored in a file. If the test is normal, go to the next step.

  In the next analysis process S32, the monitoring information collection / analysis processing unit 16 analyzes the assertion monitoring information 31 obtained in the monitoring information collection process S26, and generates analysis result data for use as an input value to the coverage information 32. Put together. The analysis process S32 may be performed by including it in S26, S33, or the like.

  Using the analysis result data, the coverage information update processing unit 17 updates the data of the coverage information 32 in the coverage information update processing S33. In this update process, the necessary conditions to be obtained from the assertion checker 6 and information obtained by collation and calculation from the assertion monitoring information 31 are merged into the coverage information 32. In this update process, for example, based on the assertion monitoring information 31, all the operation times of each element of the device under test 20 are totaled, and information such as the elements that have been operated in parallel (competitive operation) and their time and number of times is obtained. Are stored in the coverage information 32.

  Next, in the test data adjustment process S34, the test data adjustment processing unit 18 adjusts the test contents from the next time onward based on the results of the current test and monitoring (that is, the assertion monitoring information 31 and the coverage information 32 representing the result). (Change) judgment and adjustment processing are performed. In other words, for the next and subsequent tests, whether the test content is newly created as a random number from the beginning, or the test content (test data 50 that has already been generated) that was performed earlier (in this case) is taken over and reused. Use judgment to determine whether the adjustment should be made. This determination is made in particular by referring to information relating to the signal condition and competition condition occurrence results included in the coverage information 32, information relating to test adjustment results, etc. (event information 32a and timing information 32b described later).

  In the test data adjustment process S34, by referring to the coverage information 32, a part that has not been tested at that time, a part with a small number of test occurrences, a part with a little competition performance, and the like are found. If such a part is found that complies with the adjustment policy (if adjustment execution conditions are met), it is decided to adjust the test content. Then, in the test data adjustment process S34, for example, the parameter 52 of the test data 50, for example, is adjusted in accordance with the policy regarding the test content for the part (adjustment target element). In this case, adjustment of the parameter 52 refers to test competitiveness by adjusting the start timing, transfer destination address, transfer data length, etc. to the peripheral device 2 without changing the large test contents for the device under test 20. The load is adjusted.

  Finally, in the end determination S35, it is determined whether or not execution of a test (for example, a routine by a plurality of tests) required by the operator (tester) as an end condition has been completed. When the request is satisfied, the process is terminated. When the request is not satisfied, when the test (next test) is executed again, the process returns to the test data generation process S20, and the processes after S20 are performed again as described above. Do. In the test after the next time, the test content of the previous time (this time) is executed in the same manner according to the content fed back (reflected).

  As described above, the test according to the series of processing cycles (S20 to S34) can be repeatedly executed a plurality of times while feedback for adjusting the test contents (from S34 to S20). More specific description will be given below.

<5: Assertion and coverage information update processing>
FIG. 5 shows a schematic flow regarding processing (corresponding to S32, S33, etc.) for updating the coverage information 32 based on the assertion monitoring information 31 by the assertion checker 6 in the processing of the test program 30.

  In assertion activation and test activation processing S51 (corresponding to S22 and S23), the assertion checker 6 enters a hardware monitoring operation state completely independent of the test program 30 by activation of the assertion. In the device under test 20, each element starts to move individually when the test is started, and various data and signals flow on the system bus 5 through the elements such as the processor 1, the peripheral device 2, and the storage device 3. .

  As an example of a program structure for controlling the assertion checker 6, an assertion checker control program 60 is shown on the right side. This is a hardware operation coded by software in HDL. There is a declaration statement (assertion module declaration, ..., constant declaration, etc.) at the beginning, and signal conditions (assertion check conditions, request conflict conditions, etc.) are described in the subsequent conditional statements (if / then statements, etc.), and then the operation (Specific address collection, update data collection, update time collection, update count collection, etc.) are described.

  When the assertion checker control program 60 detects a control signal to be monitored, the detected time and number of times are extracted and recorded in a format such as the assertion monitoring information 31 shown below. Regarding the example of the assertion monitoring information 31 in FIG. 5, “monitoring target” indicates an identification item of the monitoring target, and there are items corresponding to this, for example, storing “time” and “number of times”. The “monitoring target” item includes, for example, a start register and a completion flag register related to the input / output device (IO) that is the peripheral device 2, establishment of a specific signal, transfer IO conflict, signal condition pattern, and the like. These classifications include (1) transmission / reception time of each IO, time and frequency of each request signal, (2) contention time of transmission / reception and frequency of contention, and (3) contention by combination (pattern) of request signal conditions. Time, and the number of competitions. Note that “time (time)” is represented by a clock cycle value, for example.

  When the assertion checker 6 detects the monitoring target during the test, a value is written in the items “time” and “number of times” in the assertion monitoring information 31. For example, when the monitoring target is the activation register of the IO-1 that is the peripheral device 2, the value determined for the address of the activation register of the IO-1 is stored, so that the data transfer by the IO-1 is performed. After the completion of this data transfer, the hardware sets a flag in the completion flag register of the IO-1. For such hardware specifications, by monitoring the address by assertion, the operation time of the hardware (device) (the time from the start to the end of operation) can be measured.

  The assertion checker 6 detects writing to the IO-1 activation register from the test operation based on the test data 50, and writes the time (activation time). Thereafter, writing to the hardware IO-1 completion flag register is detected, and the time (end time) is written. The operation time (time from the start time to the end time) of the peripheral device 2 (IO-1) can be obtained from the IO-1 start register and the IO-1 completion flag register. Similarly, information such as operation time related to the entire device under test 20 (a plurality of elements) is obtained. As a result, it is possible to know how much each element has operated in parallel (competitive operation) during the test.

<6: Assertion check>
FIG. 6 shows an example of the signal condition (competition condition) of the test target signal in the assertion check. Two examples (signal condition 1 (pattern 1), signal condition 2 (pattern) in the case of monitoring competition conditions by a plurality of factors (signal conditions) from pattern 1 to pattern 40 in the assertion monitoring information 31 of FIG. 2)).

  In this example, in FIG. 6A, items (SCLK and the like) under Signal are signals (monitoring target signals) described by the assertion checker 6, and these are the assertion activation and test activation processing S51. After that, it is always in the monitoring state. FIG. 6B shows a description example of each signal condition. The signal condition is represented by, for example, a logical operation of a plurality of signal values.

  When the signal condition 1 (for example, snoop processing) is satisfied, it is regarded as a conflict (occurrence of a race condition), and the time and the number of occurrences at this time are stored in the “time” and “number of times” items in the assertion monitoring information 31. The Similarly, when the signal condition 2 (for example, bus request processing) is satisfied, it is regarded as a conflict, and information is stored in the assertion monitoring information 31.

  In the next assertion monitoring information reading process S52 of S51, the monitoring information collection / analysis processing unit 16 reads the assertion monitoring information 31 already collected.

  Then, in the next assertion monitoring information data conversion process S53, the data conversion process is performed by using the assertion monitoring information 31 as an input, and collation, calculation, analysis, etc. of the information (corresponding to the analysis process S32).

  The coverage information 32 is updated by the coverage information update processing unit 17 in the next coverage information recording update processing S54 using the data (analysis result data) obtained in this way. At the time of update, the content (test instruction sequence 51, parameter 52) of the generated test data 50 and the operation time (start time, end time) of the generated test data 50 with respect to the past (accumulated) coverage information 32, Add timing information such as contention rate (competition time, number of times of contention).

<7: Coverage information>
FIG. 7 shows a configuration example of the coverage information 32 regarding the coverage information update process (S33). The coverage information 32 in FIG. 7 is roughly divided into (a) event information 32a and (b) timing information 32b.

  (A) The event information 32a includes items such as signal condition (pattern), number of occurrences, adjustment flag (number of adjustments), occurrence frequency (%), and cumulative number of occurrences. The event information 32a represents the number of occurrences and frequency of establishment of a plurality of signal conditions (patterns) to be monitored, a flag (number of times) of occurrence of test content adjustment, and the like.

  The signal condition (pattern) is as described above. There are predetermined signal conditions (patterns) 1 to 40 related to the control signal to be monitored. The number of occurrences is the number of occurrences in a plurality of repeated tests (routines). The cumulative number of occurrences is updated, for example, as a cumulative total value of the number of occurrences in the past in the results of many tests by repeating the routine multiple times. All values such as the number of occurrences are updated by cumulative addition. The occurrence frequency is updated by calculating what percentage of the occurrence frequency of the pattern is based on the number of tests and the number of occurrences.

  The adjustment flag (adjustment count) indicates that the generated test data 50 (test instruction sequence 51 and parameter 52) has been updated by adjustment, or represents the number of times. The item is not updated at the first time point and is 0. For example, when the preparation related to the pattern 3 occurs, the item becomes 1.

  (B) The timing information 32b includes items such as a competition time and the number of competitions for each device under test 20. In this example, the case is related to a competition of transfer operations of the peripheral device 2 (IO). In the timing information 32b, for each peripheral device 2, the values obtained by reading the start time and the end time are totaled, and the contention time for transfer is obtained. For example, when the peripheral device 2 includes three IOs IO-1 to IO-3, the peripheral device 2 has an item of each combination. For example, combinations (“1⇔2”, “2⇔3”, “3⇔1”) focusing on two conflicts such as IO-1 and IO-2, and items related to conflicts between three IOs (“1,2, , 3 ") and holds the time of operation (contention time) at the same time.

  The contention rate is obtained by calculation from [contention time] / [time of peripheral device in operation]. When the contention time and the number of times of contention are large, the contention rate becomes high. Parts with a high competition rate are preferentially selected as candidates in the determination of test content adjustment. When the timing is adjusted by selection, the number is added to the item of the number of adjustments in the timing information 32b and updated.

<8: Competition timing>
FIG. 8 shows an example of contention timing (operation timing) related to a plurality of elements related to the timing information 32b. The time is a clock cycle value, and the rectangular portion indicates the activation state (operation time) of the element (IO-1, IO-2, IO-3). Based on the collected data of the assertion monitoring information 31, the start time and end time in activation (operation) of an element such as an IO and the operation time that is the difference between them are obtained. As an example, IO-1 start time: 100, end time: 160, IO-2 start time: 120, end time: 170, IO-3 start time: 50, end time: 150.

  The contention information (timing information 32b) is obtained by conversion by calculation as follows. First, the operation times of IO-1, IO-2, and IO-3 are 60, 50, and 100 from the difference between the start time and the end time. As contention information between IO-1 and IO-2, contention time T (1⇔2), which is the time (period) during which they operate simultaneously in parallel, is between 120 and 160, and the length is 40. . Similarly, T (2⇔3) for IO-2 and IO-3 is between 120 and 150, and the length is 30. T (3⇔1) for IO-3 and IO-1 is between 100 and 150, and the length is 50. Further, for the three elements IO-1, IO-2, and IO-3, the contention time T (1, 2, 3) is between 120 and 150, and the length is 30. In accordance with such conversion, a value is stored in each item (such as “1⇔2”) of the contention time in the timing information 32b. The first line of the contention time is the T value (40, 30, 50, 30) in the above example.

  Further, when there is a conflict between elements corresponding to the value of the conflict time, the number of conflicts in the timing information 32b is added. In this example, since all the target peripheral devices 2 (IO) have conflicts, the respective conflict counts are added as indicated by 1 in the first row.

  In this way, the contention time data and the contention frequency data, which are all obtained by the difference between the magnitude comparisons, are recorded and accumulated in the timing information 32b. In the coverage information record update process S54, the contents of the coverage information 32 related to the above example are updated.

  Further, although not shown based on the data of the timing information 32b, the maximum value, minimum value, average time value, value of the contention rate, etc. of the contention time are also obtained by simple calculation, and coverage information is also obtained. 32, and is used for determination of test content adjustment. Note that the maximum, minimum, and average times regarding the contention time, and the number of times of contention are managed by data update.

<9: Test data adjustment processing>
FIG. 9 shows a schematic process flow relating to the test data adjustment process (corresponding to S34). Based on the accumulated coverage information 32, in the processing of the test program 30 (the test data adjustment processing unit 18, the test generation processing unit 11, etc.), the generated test data 50 (the test instruction sequence 51, etc.) is used. An example of how to create a test resource will be described.

  In determination processing (test data adjustment determination / candidate determination processing) S81, the test data adjustment processing unit 18 determines whether or not to adjust the test data 50 based on the coverage information 32 (in the case of the above-described new generation). As an example of the adjustment, as described above, only the parameter 52 is regenerated by reusing the test instruction sequence 51.

  In this determination, information such as the number of occurrences in the event information 32a in the coverage information 32, an adjustment flag (adjustment number), the occurrence frequency, and the number of competitions in the timing information 32b are used. In this determination, it may be determined according to circumstances according to an AND (logical product) condition or an OR (logical sum) condition. This condition may be, for example, a determination based on a condition for each test, or may be determined in advance by a system setting or the like.

  In this determination, the following cases (parts) are recognized based on the coverage information 32. That is, an untested case (a case that has not yet been realized as a test target, for example, a part where the number of occurrences is 0), a case where the test is insufficient (for example, a part where the number of occurrences is low), or a competition on the system bus 5 There are cases where the operation has not yet occurred or is small (for example, a portion where the number of times of competition is small), a combination of signals to be tested, and the like.

  When the adjustment is made based on the recognition of the case (part), it is determined whether a part (element or the like) to be preferentially adjusted for the test contents is a plurality of candidates. If new generation is selected in S81, the process ends without performing the processes of S82, S83, etc., and the process proceeds directly to the test data generation process S20. If the adjustment is selected in S81, the process proceeds to an adjustment target selection process S82 using a random number.

  As an example of the case of new generation, for example, when the number of occurrences in the event information 32a is 1000, the adjustment flag is 10,000, the occurrence frequency is 90%, etc. to decide. Then, the adjustment processing up to that point (reuse of the generated test instruction sequence 51) is stopped, and the next test content is generated by generating new test data 50 (reset). For example, the following different signal conditions (patterns) ) Will begin to run the test.

  As another example in the case of a new generation, when the test content is narrowed down to one point of a certain condition and the test is performed on the one point, the number of occurrences and the adjustment flag ( If the expiration condition is satisfied, the next test is newly generated.

  In the adjustment target selection process S82, based on the generated test instruction sequence 51, a random adjustment target is randomly selected from a plurality of candidates of the device under test 20 to be adjusted obtained in S81. Is selected. Since there are many combinations that can be adjusted, a random number is used for selection.

  In the test data adjustment (parameter regeneration) process S83, the adjustment target selected in S82 is reused in the generated test instruction sequence 51 to regenerate the parameter 52 as the next test content. Do. Thereby, adjusted test data 50 for the next test is obtained.

  After these processes, in the test data generation process S20, a test based on the adjusted test data 50 or a new generation test is generated again by the test generation processing unit 11. Note that the same applies to the case where S83 is performed as a part or modification of S20.

<10: Contention timing adjustment example>
FIG. 10 shows, as a time chart, examples of (a) before adjustment and (b) after adjustment regarding the test data adjustment processing (S34) of FIG. This example is particularly a case where the competition operation is automatically adjusted by adjusting the execution timing of elements for the competition test. The rectangle indicates the activation state (operation time) of each element, and is determined by the test instruction sequence 51 and the parameter 52.

  The elements to be tested in this example include four CPU-0 to CPU-3 related to the processor 1 and four IO-0 to IO-3 related to the peripheral device 2. The generated test instruction sequence 51 is an execution timing of each test for the four processors 1 and an execution timing of each test for the four IOs (peripheral devices 2).

  The execution timing of each target element is as illustrated. For example, IO-1 is executed first, and immediately after the completion of the execution of IO-3 and IO-2, it continues almost in parallel with IO-0, CPU-2, and CPU-3, and after a little delay, CPU-1 CPU-0 is executed. Although not shown, the series of execution timing information is included in the coverage information 32 (timing information 32b). For example, it is assumed that (a) pre-adjustment information indicates the result of the previous (current) test and is already stored as the content of the timing information 32b.

  In the test data adjustment processing of FIG. 9, as an object, one effective for adjustment (regeneration of the parameter 52) is selected from these target elements, and the execution timing is adjusted. Timing information 32b is referred to as an input to select an effective target for adjustment. As a specific example, the timing information 32b of FIG. 7 stores the numerical values of the results of the contention time and the number of times of contention of the peripheral device 2. Those with less contention time and contention number (or contention rate) are preferentially determined as candidates (S81). An adjustment target is randomly selected from the candidates (S82). Then, the parameter 52 is regenerated (adjusted) for the adjustment target by reusing the test instruction sequence 51 of the test data 50 by the test data adjustment (parameter regeneration) process S83.

  As a method of adjustment, for example, there are a method of adjusting the activation time of each target element (method of adjusting the activation schedule) and a method of adjusting increase / decrease (length) of the operation time (test period). One or both of them may be employed.

  Furthermore, the method for adjusting the activation time is roughly classified into a distributed (series) type and a competitive (parallel) type. In the distributed (series) type, element activation times (operation times) are allocated at different times in series as much as possible from the start to the end of the test. On the other hand, in the competitive (parallel) type, element activation times (operation times) are assigned so as to be operated at the same time while being aware of parallel execution as much as possible. In addition, a distributed conflict mixed type is possible in which both types are mixed using random numbers.

  In the method of adjusting the increase / decrease in the operation time (test period), the test of the processor 1 adjusts the test instruction sequence 51 generated by random numbers, the type of data (parameter 52), the arrangement location, and the like. Specifically, the increase / decrease in the number of instructions, the exchange of instructions, the repetition of instruction execution, the instruction placement environment, the data placement environment, the data type, and the like are adjusted. In addition, interrupts and snoop disturbances may be added.

  For testing of peripheral devices 2 such as I / O devices, transfer length increase / decrease, transfer source / destination placement environment, start command placement environment, repeat transmission / reception by start command, mode switching for each protocol, external interrupt There are disturbances, etc.

  After the adjustment as described above, the adjustment flag (the number of adjustments) in the event information 32a of the coverage information 32 in FIG. 7 is incremented and updated. As a result, how many times the existing test data 50 has been reused, that is, the results of adjustment, remain as data. The data is used in the above-described determination (FIG. 9).

  In the example of FIG. 10, in order to improve the degree of contention of the test instruction sequence 51 of a plurality of elements of the device under test 20 in the adjustment and test by reusing the generated test instruction sequence 51, ) Is an example of the competitive type, which is one of the methods for adjusting.

  In the test data adjustment process of FIG. 9, when it is determined that the number of times of competition of the timing information 32b of FIG. Are randomly selected from the candidates and adjusted (S82, S83).

  For example, (a) From the state before adjustment, adjustment is made in the direction of the arrow so that the start timings of CPU-2, IO-0, CPU-3, and CPU-1, CPU-0 indicated by broken lines are advanced in time. (Adjustment of corresponding parameter 52). Thereby, (b) it will be in the state after adjustment. The start time is shifted forward while the length of the operation time of each adjustment element remains unchanged. It can be seen that the degree of competition is improved after (b) adjustment compared to (a) before adjustment. For example, a competition test is performed in which elements such as CPU-2 and IO-0 and elements such as IO-3 operate simultaneously in parallel.

  In this system, the tester can easily grasp the state such as the competitiveness by performing the process of displaying the time chart as shown in FIG. 8 or 10 on the screen of the console terminal 200 or the like. FIG. 11 shows a display example of a time chart showing such competitiveness.

  As described above, in the system according to this embodiment, the test data 50 of the executed test is reused to adjust the next test content. In this system, the contents of the test are automatically adjusted by feeding back (reflecting) the test results up to the previous test in accordance with the policy using assertions and coverage. By repeating such a test, it is possible to improve the completeness with respect to the test on the function and timing related to the device under test 20. In particular, it is possible to avoid and reduce the execution of meaningless tests (for example, overlapping tests), and to efficiently execute an effective test such as a competitive test with an increased competition rate. Therefore, the test period can be shortened and the quality of the product can be improved.

  As mentioned above, the invention made by the present inventor has been specifically described based on the embodiment. However, the present invention is not limited to the embodiment, and various modifications can be made without departing from the scope of the invention. Needless to say.

  The present invention can be used in an information processing system for IC design and testing.

1 is a diagram illustrating a basic configuration example of an information processing system (logic simulation environment) including an information processing apparatus (EWS) to which an information processing apparatus testing method and program according to an embodiment of the present invention are applied. It is explanatory drawing which shows the example of a structure of the to-be-tested apparatus on the logic simulator in this test method and a program, and the state of the test program (test data) at the time of a test relevant to it. It is a figure which shows the structure containing correlation, such as each process part in the characteristic control part centering on a logic simulator and a test program. It is a figure which shows the schematic processing flow of a test program and an assertion checker. It is a figure which shows the example of the schematic flow regarding the coverage information update process by assertion monitoring information, an assertion checker control program, and assertion monitoring information. It is a figure which shows the example of the signal condition (competition condition) of the test object signal in an assertion check, (a) is an example of a signal condition, (b) is a description example of a signal condition. It is a figure which shows the structural example of coverage information, Among these, (a) shows event information and (b) shows timing information. It is a figure which shows the example of the timing of the competition operation | movement in the several element of a to-be-tested apparatus. It is a figure which shows the schematic process flow regarding a test data adjustment process (S34). It is a figure which shows the example before (a) adjustment and after (b) adjustment regarding a test data adjustment process (S83) as a time chart. It is a figure which shows an image regarding the process which displays the time chart showing competitiveness on the screen of a computer.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 1 ... Processor (CPU), 2 ... Peripheral device (IO etc.), 3 ... Memory | storage device (memory), 5 ... System bus, 6 (6-1, 6-2, 6-3) ... Assertion checker, 10 ... Logic Simulator, 11 ... Test generation processing unit, 12 ... Expected value generation processing unit, 13 ... Test execution processing unit, 14 ... Test result comparison / output processing unit, 15 ... Assertion activation processing unit, 16 ... Monitoring information collection / analysis processing unit , 17 ... Coverage information update processing unit, 18 ... Test data adjustment processing unit, 20 ... Device under test (logical file), 30 ... Test program (random number test program), 31 ... Assertion monitoring information, 32 ... Coverage information, 32a ... Event information, 32b ... Timing information, 50 ... Test data, 51 ... Test instruction sequence, 52 ... Parameter, 60 ... Assertion checker control program, 00 ... EWS (engineering workstation), 101 ... controller, 102 ... storage unit, 103 ... communication unit, 104 ... auxiliary storage unit, 105 ... input unit, 106 ... display unit, 200 ... console terminal.

Claims (6)

An information processing apparatus test program for causing a computer to execute a process of testing an apparatus under test of an information processing apparatus on a logic simulator according to a tester's operation,
(A) processing for generating test data for testing the device under test using a random number by a test program;
(B) processing for generating an expected value of a test result based on the test data;
(C) a process of starting an assertion checker, which is a program for monitoring the test based on the test data, with the start of the test;
(D) a process of performing a test using the test data simultaneously on a plurality of elements including a processor, a peripheral device, and a storage device in the device under test of the information processing device;
(E) A process of monitoring the element by the assertion checker during execution of the test and storing monitoring information extracted by the monitoring;
(F) a process of comparing the execution result of the test with the expected value and outputting the result;
(G) collecting the monitoring information, analyzing the collected monitoring information, and using the analysis result to update coverage information for managing the comprehensiveness of the test;
(H) Based on the updated coverage information, it is determined whether or not to adjust the content of the test. When adjusting, the target to be adjusted is selected, and the content of the test based on the test data regarding the target to be adjusted The process of adjusting
(I) An information processing apparatus test program that causes the computer to execute a process of executing a test based on the adjusted test data. The information processing device test program according to claim 1,
In the processing step of (G), the content of the test data generated by (A) and timing information related to the simultaneous parallel operation of the plurality of elements for the coverage information accumulated at the time of the update, And add
The timing information includes information on the operation timing of the element, contention time and number of times of contention regarding the concurrent operation,
In the processing step of (H), an information processing apparatus test program characterized in that, using the timing information, a portion having low competitiveness regarding the simultaneous and parallel operation of the plurality of elements is selected as the adjustment target. . In the information processing apparatus test program according to claim 1 or 2,
In the processing step (H), the element to be adjusted is selected by using the coverage information and a random number in combination.
In the selection, information is characterized by preferentially determining, as a candidate, a portion having low competitiveness regarding the simultaneous and parallel operation among the plurality of elements, and selecting the adjustment target by the random number from the candidates. Processing equipment test program. In the information processing apparatus test program according to any one of claims 1 to 3,
In the processing step (H), with respect to the object to be adjusted, the test instruction sequence in the test data generated in the processing step (A) is reused to regenerate its parameters. An information processing apparatus test program characterized by adjusting the contents of the information processing apparatus. The information processing device test program according to claim 1,
An information processing apparatus test program, comprising: performing a process of displaying, on a computer screen, a time chart representing a competitiveness related to execution timing and concurrent operation for the plurality of elements in the device under test in the test. An information processing apparatus test method for testing an apparatus under test of an information processing apparatus on a logic simulator according to a tester's operation,
(A) a processing step of generating test data for testing the device under test using a random number by a test program;
(B) a processing step of generating an expected value of a test result based on the test data;
(C) a processing step of starting an assertion checker, which is a program for monitoring the test based on the test data, with the start of the test;
(D) a processing step of performing a test based on the test data simultaneously on a plurality of elements including a processor, a peripheral device, and a storage device in the device under test of the information processing device;
(E) a processing step of monitoring the element by the assertion checker during execution of the test and storing monitoring information extracted by the monitoring;
(F) a processing step of comparing the execution result of the test with the expected value and outputting the result;
(G) processing steps for collecting the monitoring information, analyzing the collected monitoring information, and updating coverage information for managing the comprehensiveness of the test using the analysis result;
(H) Based on the updated coverage information, it is determined whether or not to adjust the content of the test. When adjusting, the target to be adjusted is selected, and the content of the test based on the test data regarding the target to be adjusted Processing steps to adjust,
(I) A processing step of executing a test based on the adjusted test data.

Images