KR102335401B1 - Generating Test Plan Automatically - Google Patents

Abstract

Disclosed is an automatic generation method for a test plan of a test system. According to one embodiment of the present specification, the automatic generation method for a test plan may select a plurality of test cases including a controller group as the controller group to be generated for the test plan is selected, set a test order for each of the plurality of test cases according to a predetermined criterion, and update a database by reflecting the test execution results for the plurality of test cases according to the set order. Accordingly, it is possible to more efficiently generate a test plan by reflecting the test history according to the characteristics of the controller.

Description

Generate Test Plan Automatically {Generating Test Plan Automatically}

This specification relates to a method for automatically generating a test plan.

The vehicle electronic controller developer must test whether the controller is developed according to the required specifications. The vehicle electronic controller test is conducted focusing on the vehicle electronic controller development requirements. In the electronic controller test, the tester conducts the test using Test Cases to check whether each REQuirements is properly developed, and submits the test report.

The controller has many requirements. There may be more than one test case to test one requirement specification, and it may take a long time to test each test case. In order to proceed with the controller test, it is necessary to conduct the test case tests in an efficient order in order to test a large number of test cases corresponding to complex requirements. In addition, for efficient controller testing, it is necessary to share the test progress and results with others involved in testing. To this end, there is a need for a test plan in which information on test progress or test results can be built quickly and efficiently.

An object of the present specification is to provide a method capable of automatically generating a test plan by reflecting a required specification and a test result history.

The technical problems to be achieved by the present invention are not limited to the technical problems mentioned above, and other technical problems not mentioned are clear to those of ordinary skill in the art to which the present invention belongs from the detailed description of the invention below. can be understood clearly.

A method for automatically generating a test plan of a test system according to an embodiment of the present specification includes: selecting a plurality of test cases including the controller group as a controller group to be generated of the test plan is selected; setting a test order of each of the plurality of test cases according to a predetermined criterion including a test success rate, an execution time rate, and the importance of a required specification; and updating the predetermined criteria by reflecting test execution results for the plurality of test cases according to the set order.

The test case may be stored in advance in a database for each type of the controller group, and may include at least one test case duplicated in different controller groups among the plurality of test cases.

The database is

For each test case, the test execution time, the importance information of the required specification, the number of successes, the number of failures, the related controller group information to which the test case belongs, the time score according to the execution time, the importance score according to the importance information, the success The success rate score and the final total score according to the number of times and the number of failures may be stored in a matched form.

The predetermined criterion includes the final summation score obtained by summing the time score, the importance score, and the success rate score, and the setting of the test order includes determining the test order in order of decreasing the final summed score. can

The test success rate is a ratio of the number of successes to the total number of tests.

The execution time ratio is the execution time of the test case to be scored compared to the test execution time of each of the plurality of selected test cases.

The method for automatically generating the test plan may include: checking whether the test case is a test result, success or failure; counting the number of test successes or failures with respect to the controller group to which the test case belongs; and re-calculating the test success rate of the controller group to which the test case belongs.

The importance of the requirement is pre-registered in the system by the user, and a score corresponding to the importance of the requirement may have a lower score as the importance of the requirement increases.

At least one request specification (REQuirements) corresponding to the controller group is previously registered in the test system, and at least one test case for testing the request specification may be stored in the database.

A plurality of test cases may exist for one requirement specification, and test cases for testing each requirement and each requirement specification may be stored in advance in the system.

The method for automatically generating the test plan may include: checking whether the test case is a test result, success or failure; counting the number of test successes or failures with respect to the controller group to which the test case belongs; and re-calculating the test success rate of the controller group to which the test case belongs.

The test system may include a system for testing an electronic controller of a vehicle, and the controller group may include at least one of a driveline controller group, a convenience function controller group, a chassis controller group, and a communication network controller group.

According to the method for automatically generating a test plan according to an embodiment of the present specification, the actual controller test is performed by first testing a test case that has a high probability of being determined to be a failure and a short test time as a result of a test by reflecting the test history according to the characteristics of the controller If the result is judged to be a failure, the controller development department can share the test results correctly, and errors can be corrected quickly.

In addition, according to the method for automatically generating a test plan according to an embodiment of the present specification, an efficient test can be performed by excluding test cases having a high importance of a required specification and a high probability of being determined to fail with a low priority test case. let it be

The effects obtainable in the present invention are not limited to the above-mentioned effects, and other effects not mentioned will be clearly understood by those of ordinary skill in the art to which the present invention belongs from the following description. .

BRIEF DESCRIPTION OF THE DRAWINGS The accompanying drawings, which are included as a part of the detailed description to help the understanding of the present specification, provide embodiments of the present specification, and together with the detailed description, explain the technical features of the present specification.
1 shows a schematic system for implementing a method for automatically generating a test plan according to an embodiment of the present specification.
2 is a flowchart of a method for automatically generating a test plan according to an embodiment of the present specification.
3 is a diagram for describing a relationship between a controller group and a test case according to an embodiment of the present specification.
4 is a flowchart for describing S220 shown in FIG. 2 in more detail.
5 illustrates an example of data storage of an initial database applied to implement a method for automatically generating a test plan according to an embodiment of the present specification.
FIG. 6 is a diagram for describing in more detail a process of generating a test plan by selecting test cases of a first specific controller group using the database of FIG. 5 and updating the database after performing the test.
7 is a view for explaining an example of generating a test plan for a second specific controller group after updating the database of FIG. 6 and reflecting the test execution result.
8 is a view for explaining an example in which a test item value is changed by a user input according to an embodiment of the present specification.
9 is a view for explaining an example in which the priority of all test cases is changed according to test plan generation, test execution, and test result reflection based on FIGS. 5 to 8 .
FIG. 10 is a diagram for explaining an example of generating a test plan for a third specific controller group after a priority of all test cases is changed according to FIG. 9 .
BRIEF DESCRIPTION OF THE DRAWINGS The accompanying drawings, which are included as a part of the detailed description to facilitate the understanding of the present invention, provide embodiments of the present invention, and together with the detailed description, explain the technical features of the present invention.

Hereinafter, the embodiments disclosed in the present specification will be described in detail with reference to the accompanying drawings, but the same or similar components are assigned the same reference numbers regardless of reference numerals, and redundant description thereof will be omitted. The suffixes "module" and "part" for components used in the following description are given or mixed in consideration of only the ease of writing the specification, and do not have distinct meanings or roles by themselves. In addition, in describing the embodiments disclosed in the present specification, if it is determined that detailed descriptions of related known technologies may obscure the gist of the embodiments disclosed in the present specification, the detailed description thereof will be omitted. In addition, the accompanying drawings are only for easy understanding of the embodiments disclosed in the present specification, and the technical idea disclosed herein is not limited by the accompanying drawings, and all changes included in the spirit and scope of the present invention , should be understood to include equivalents or substitutes.

Terms including an ordinal number such as 1st, 2nd, etc. may be used to describe various elements, but the elements are not limited by the terms. The above terms are used only for the purpose of distinguishing one component from another.

When an element is referred to as being “connected” or “connected” to another element, it is understood that it may be directly connected or connected to the other element, but other elements may exist in between. it should be On the other hand, when it is said that a certain element is "directly connected" or "directly connected" to another element, it should be understood that the other element does not exist in the middle.

The singular expression includes the plural expression unless the context clearly dictates otherwise.

In the present application, terms such as “comprises” or “have” are intended to designate that a feature, number, step, operation, component, part, or combination thereof described in the specification exists, but one or more other features It should be understood that this does not preclude the existence or addition of numbers, steps, operations, components, parts, or combinations thereof.

The above-described specification, it is possible to be implemented as computer-readable code on a medium in which the program is recorded. The computer-readable medium includes all types of recording devices in which data readable by a computer system is stored. Examples of computer-readable media include Hard Disk Drive (HDD), Solid State Disk (SSD), Silicon Disk Drive (SDD), ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical data storage device, etc. There is also a carrier wave (eg, transmission over the Internet) that is implemented in the form of. Accordingly, the above detailed description should not be construed as restrictive in all respects but as exemplary. The scope of this specification should be determined by a reasonable interpretation of the appended claims, and all modifications within the equivalent scope of the present invention are included in the scope of this specification.

The tester should determine the test target requirements according to the development stage, select test cases related to the requirements, and then proceed with the test. Test case selection and execution plans for executing a specific purpose controller test are called test plans.

1 shows a schematic system for implementing a method for automatically generating a test plan according to an embodiment of the present specification.

The method for automatically generating a test plan according to an embodiment of the present specification can be implemented through the system shown in FIG. 1 . The system may include a test device 100 and a controller 1 to be tested.

The test apparatus 100 may include an input unit 110 , a database 120 , a display 130 , an audio output unit 140 , and an interface unit 150 .

The input unit 110 may include a user input unit (eg, a touch key, a push key, etc.) for receiving information from a user. The input unit 110 may include a camera or an image input unit, a microphone for inputting an audio signal, or an audio input unit in addition to the user input unit. According to an embodiment of the present specification, when a new controller is registered in the database or when registration for a specific controller's requirements, etc. is required, registration is possible through the user input unit.

The database 120 may store test cases and information related to each test case in the form of a table. The information related to the test case may include test execution items, for example, test execution time, importance information, test result success count, and failure count. In addition, the information related to the test case may further include a time ratio, a critical ratio, a time score obtained by converting the success rate information into a score, an importance score, a success rate score, and a final sum score that is the sum of the scores. The system may further include a memory, and the memory may store data supporting various functions of the system 100 . The memory may store a plurality of application programs driven in the system 100 , data for system operation, and instructions. At least some of these application programs may be downloaded from an external server through the communication unit.

The display unit 130 may output information processed by the system 100 . For example, the display unit 130 may display execution screen information of an application program for test execution or other UI information on the execution screen information. Also, the display unit 130 may display on the screen whether the test result was successful or not.

The audio output unit 140 may output an audio signal related to a test. The audio signal related to the test may include an alarm signal corresponding to an event such as error detection and success or failure during the test.

The interface unit 150 serves as a passage with all external devices connected to the system 100 . The interface unit 150 receives data from the test object controller 1, receives power and transmits it to each component inside the test object controller 1, or transmits data inside the system 100 to the test object controller 10 ) can be sent to

The controller 160 controls the overall operation of the system 100 . The controller 160 may provide a test result for a test case by processing signals, data, information, etc. input or output through the above-described components or driving a test program stored in a memory (not shown).

Although not shown in FIG. 1 , the system 100 may further include a power supply. The power supply unit receives external power and internal power under the control of the controller 160 to supply power to each component included in the system 100 .

At least some of the respective components may operate in cooperation with each other in order to implement the operation, control, or control method of the system 100 . In addition, the operation, control, or control method of the system 100 may be implemented on the system 100 by running at least one application program stored in a memory.

Meanwhile, the test target controller 1 may include a drive system controller, a chassis controller, a convenient function controller, a communication network controller, and the like. It goes without saying that the controller is exemplary and various controllers related to the vehicle electronic controller may be included.

2 is a flowchart of a method for automatically generating a test plan according to an embodiment of the present specification. 3 is a diagram for describing a relationship between a controller group and a test case according to an embodiment of the present specification. The method for automatically generating the test plan may be implemented through the system 100 of FIG. 1 , more specifically, the control unit 160 of the system 100 .

Referring to FIG. 2 , the system 100 may select a test case belonging to a test target controller group ( S200 ).

The test case is for testing the required specification of a specific controller, and a plurality of test cases may exist for testing one specific controller. That is, in order to complete the test on a specific controller, it may mean that tests on different types of test cases must be completed.

Referring to FIG. 3 , the test target controller group may include, for example, a drive system controller, a chassis controller, a convenient function controller, and a communication network controller. Meanwhile, test cases include input/output function test, dynamic model test, CAN communication network test, operating voltage test, CAN communication general test, user UI test, load test, diagnostic communication test, CAN routing test, voltage waveform test, constant temperature and humidity test, etc. may include Here, the input/output function test case may be included in different controller groups, that is, a drive system controller group, a chassis controller group, and a convenient function controller group. Also, for example, the constant temperature and humidity data may be included in a drive system controller group, a chassis controller group, a convenience function controller group, and a communication network controller group.

Accordingly, when a specific controller group is selected first, test cases necessary for testing for the specific controller group can be selected. By adjusting the test sequence according to To this end, it is necessary to register the test case and the required specification on the system 100 before step S200 of FIG. 2 . The test case and required specification registration may be registered through a user input unit or the like. The registered test cases and requirement specifications may be accumulated in a database.

In addition, before S200 of FIG. 2 , it may be necessary to input information about the controller to be tested. According to an exemplary embodiment, when information on a controller to be tested is input, the system 100 may check a controller group to which the controller belongs, and a plurality of test cases included in the controller group may be extracted.

The system 100 may set a test order for each of a plurality of test cases according to a predetermined criterion.

The predetermined criterion may include a test success rate, an execution time rate, and the importance of a required specification. For example, a test case having a low test success rate among a plurality of test cases may be set to have a high priority. In addition, a higher priority may be set for a test case that takes a long time to perform the test. In addition, according to the importance of the required specification of the controller, a higher priority may be set for test cases related to the required specification with high importance. According to the present specification, the test order may be set as the priority for the test case having the high priority.

Meanwhile, the system 100 may determine the test order based on data stored in the database 120 . The database 120 may include a test execution time for each test case, information on the importance of the required specification, the number of test results successes, the number of failures, and related controller group information to which the test case belongs.

Here, the related controller group information may mean a name of a controller group to which a corresponding test case can be applied. For example, when the first test case is applied to all of group A, group B, and group C, the associated controller group information for the first test case may be group A, group B, and group C. The test result pattern of the controller for each controller group may be different. For example, the convenience function controller group shows a pattern where the success rate of voltage-related test cases is high while the success rate of communication-related test cases is low. can be low. Therefore, it is meaningful to manage the success rate of test cases for each controller group, and the accumulated success rate can be used when generating the next test plan by reflecting the history of past test results in the test cases for each controller group.

Meanwhile, the database may include score information for determining the final test order. The above-described execution time, information on the importance of the requirements, the number of successes and the number of failures, etc. may be configured with a predetermined score and used to generate a test plan.

The test performance score can be calculated as a time ratio according to the following equation.

And, the score according to the importance of the requirement specification may be composed of the following example.

In addition, the test success rate may be calculated as a success rate according to the following equation.

The system 100 may calculate a final summation score of the time ratio (time score), the importance ratio (importance score), and the success rate (success rate score), and the lower the final summed score, the higher the priority.

The system 100 may update data stored in the database 120 by reflecting test execution results for a plurality of test cases according to the test order of each automatically generated test case ( S220 ).

The system 100 may update the data to be used in a subsequent test by automatically updating the test success count and failure count among the data stored in the database 120 to recalculate the test success rate item.

4 is a flowchart for describing S220 shown in FIG. 2 in more detail.

The system 100 may check whether the test is successful or failed as a result of performing a test on the test case to be tested according to the automatically generated test plan ( S400 ).

The system 100 may recalculate the success rate of the pre-stored test case by counting the number of test successes or failures (S420).

The system 100 may update the final summed score of each test case according to the recalculated success rate. The updated final score can be used when generating the next test plan.

Hereinafter, a process of automatically generating a test plan according to an embodiment of the present specification will be described in more detail with reference to FIGS. 5 to 10 .

5 illustrates an example of data storage of an initial database applied to implement a method for automatically generating a test plan according to an embodiment of the present specification.

Referring to FIG. 5 , the database includes a test case 510 , a test item 520 for the test case, for example, a test execution time 521 , a requirement specification importance 522 , the number of successes 523 , and failure. count 524 . In addition, the database includes various related controller group information 500 including the test case. In addition, the database includes a final summation score 540 according to each test result together with a test result item 530 , for example, a time ratio 531 , an important ratio 532 , and a success rate 533 .

Among the information included in the database, the test case 510 and the associated controller group 500 are registered in the system through a user input unit, etc., the test item 520 is information according to the test result, and the test result item ( 530) is automatically calculated by the above equation.

FIG. 6 is a diagram for explaining in more detail a process of generating a test plan by selecting test cases of a first specific controller group using the database of FIG. 5 and updating the database after performing the test.

Referring to FIG. 6 , according to an exemplary embodiment, a test plan for the gateway controller of the first vehicle may extract test cases to be tested based on the initial database shown in FIG. 5 . The gateway controller belongs to the communication network controller group. The system 100 includes test cases (TC-03, TC-04, TC-05, TC-08, TC-09, TC-10, TC -11) can be extracted (FIG. 6 (a)). The extracted test cases are sorted in the order of the lowest final sum score, and the sorted order is the test order, which is an example in which the test plan is generated.

On the other hand, referring to FIG. 6B , the system 100 performs a test on each extracted test case, counts the number of successes and the number of failures as a result of the test, respectively, and records them in the database (610) ). The system 100 updates the success rate 620 information based on the updated number of successes and failures, and updates the final sum score 630 as the success rate is updated. As the database is updated, it can be seen that the earliest priority for the gateway controller is changed from TC-03 to TC-05 after performing the test.

7 is a view for explaining an example of generating a test plan for a second specific controller group after updating the database of FIG. 6 and reflecting the test execution result.

Meanwhile, FIG. 7 is for explanation of generating a test plan for the box controller of the second vehicle, and the selection box controller belongs to the communication network controller group as well. Accordingly, the type of test case used in FIG. 6 may be equally applied to FIG. 7 .

The system 100 extracts a test case based on the database in which the success rate is updated after the gateway controller test of the first vehicle and generates a test plan (FIG. 7(a)). The system 100 performs a test on each test case based on the generated test plan, counts the test result, the number of successes and the number of failures, and records it in the database ( 710 ). As shown in FIG. 6 , as the number of successes and the number of failures for each test case is updated, the success rate and the final summed score may change, and accordingly, the test priority may be changed.

8 is a view for explaining an example in which a test item value is changed by a user input according to an embodiment of the present specification. Referring to FIG. 8 , in the process of generating a test plan for the gateway controller of the third vehicle, since the gateway controller belongs to the communication network controller group, the database finally updated in FIG. 7 may be utilized. However, for the TC-09 test case, the test execution time has been reduced from 1440 minutes to 240 minutes due to technology development, and this can be reflected before testing in the process of generating the gateway controller test plan for the third vehicle. For example, the execution time of TC-09 can be changed to 240 minutes through the user input unit, and accordingly, the time ratio score is improved, and the final total score is also changed, indicating that TC-09 is changed in the earliest order. have. That is, according to an embodiment of the present specification, the test execution result for the test case may be reflected, but by manually changing the test item before the test is performed, the technology development may be immediately reflected to generate a test execution plan. .

9 is a view for explaining an example in which the priority of all test cases is changed according to test plan generation, test execution, and test result reflection based on FIGS. 5 to 8 . In the initial database, the test plan is generated with TC-01 as the first priority, but it can be seen that TC-09 is changed to the first priority as the test results are cumulatively updated through FIGS. 6 to 8 . This will allow the re-prioritization of test cases by considering the final combined score for all test cases, not just a specific test group.

FIG. 10 is a diagram for explaining an example of generating a test plan for a third specific controller group after a priority of all test cases is changed according to FIG. 9 . Referring to FIG. 10 , unlike the aforementioned communication network controller group, the seat controller belongs to the convenient function controller group, and the system 100 extracts the test cases associated with the convenient function controller group from the database re-established in FIG. 9 to generate a test plan. can be newly created.

The present invention described above can be implemented as computer-readable codes on a medium in which a program is recorded. The computer-readable medium includes all types of recording devices in which data readable by a computer system is stored. Examples of computer-readable media include Hard Disk Drive (HDD), Solid State Disk (SSD), Silicon Disk Drive (SDD), ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical data storage device, etc. There is also a carrier wave (eg, transmission over the Internet) that is implemented in the form of. Accordingly, the above detailed description should not be construed as restrictive in all respects but as exemplary. The scope of the present invention should be determined by a reasonable interpretation of the appended claims, and all modifications within the equivalent scope of the present invention are included in the scope of the present invention.

delete

Test device: 100

Claims (6)

A method for automatically generating a test plan of a test system, the method comprising:
selecting a plurality of test cases including the controller group as a controller group to be generated of the test plan is selected;
setting a test order of each of the plurality of test cases according to a predetermined criterion including a test success rate, an execution time rate, and the importance of a required specification;
checking whether a test result of the test case succeeds or fails according to the set order;
counting the number of test successes or failures with respect to the controller group to which the test case belongs;
recalculating the test success rate of the controller group to which the test case belongs; and
updating the predetermined criteria by reflecting test execution results for the plurality of test cases according to the set order;
How to automatically generate a test plan that includes.
The method of claim 1,
The test cases are stored in advance in the database for each type of the controller group,
The database is
For each test case, the test execution time, the importance information of the required specification, the number of successes, the number of failures, the related controller group information to which the test case belongs, the time score according to the execution time, the importance score according to the importance information, the success A method for automatically generating a test plan, characterized in that the success rate score and the final total score according to the number of times and the number of failures are stored in a matched form.
3. The method of claim 2,
The predetermined criterion includes the final summation score obtained by summing the time score, the importance score, and the success rate score,
The setting of the test order comprises determining the test order in an order of decreasing the final summed score.
The method of claim 1,
checking whether the test case is a test result, success or failure;
counting the number of test successes or failures with respect to the controller group to which the test case belongs; and
recalculating the test success rate of the controller group to which the test case belongs;
A method for automatically generating a test plan, comprising:
4. The method of claim 3,
The importance of the required specification is registered in advance in the system by the user,
The test plan automatic generation method, characterized in that the score corresponding to the importance of the requirement specification is lower as the importance of the requirement specification is higher.
delete

Images