KR20070082222A - Api test method and apparatus of embedded software - Google Patents

Abstract

A method and a device for testing an API of embedded software are provided to offer a test result even if a system is crashed by storing information before system crash and failed test case information. The device includes a test suit(510) for registering the test case, a test manager(610) for storing the failed test case information, and an output manager(640) for providing a test result. A log before the system crash and the failed test case information are stored to a log file.

Description

API test method and apparatus of embedded software

1 illustrates a CppUnit configuration according to the prior art.

FIG. 2 is a diagram illustrating a unit test framework configuration using CppUnit according to the prior art. FIG.

3 is a flow chart of a unit test framework according to the prior art.

4 is a diagram illustrating a system test frame work configuration according to the prior art.

5 is a flow diagram of a system test framework according to the prior art.

6 is a diagram illustrating a GTT configuration of the present invention.

7 is a diagram illustrating a configuration of a system test framework using GTT of the present invention.

8 is a test harness core class diagram in accordance with an embodiment of the present invention.

9 is a flow diagram of a system test framework in accordance with an embodiment of the present invention.

10 illustrates a test setup file in accordance with an embodiment of the present invention.

11 illustrates test results in accordance with an embodiment of the present invention.

<Explanation of symbols for main parts of the drawings>

400: GTT 410: test harness core

440: interface 430: test harness platform

The fast-growing embedded software industry is emerging in a new and dynamic way through technological innovation and fierce competition in the market.

Since embedded systems are designed for specific applications rather than general purpose, the advent of this industry has forced the industry to create new and creative solutions for dynamic structural changes for more complex system designs.

An embedded system is a system in which a software that performs predetermined specific functions is embedded in a microprocessor installed in various electronic products or information devices, rather than a general computer, and the embedded software is embedded software.

Embedded software has become the core software for diversifying and adding value to industrial and military control devices, digital information appliances, automatic sensor equipment, etc. Recently, more powerful encryption capabilities, improved multimedia APIs, and communication APIs Increasing demand from handheld devices, such as core sets, is increasing the usability of operating systems for device vendors working to provide wireless performance and require features to support the OS embedded within the API. .

1 is a diagram illustrating a CppUnit configuration according to the prior art.

CppUnit (100) according to the prior art is an open source porting Junit used in Java to C ++ is one of the implementation of the C ++ unit test is a library for testing the framework of C ++.

The CppUnit 100 may include a test suite template class 110, a test suite repository 120, a test result collector 130, and a test result output manager. Test Result Output Manager (140), Runner (150), and other modules, such as modules that provide a Graphical User Interface (GUI) in a Microsoft Foundation Class (MFC) environment, Windows, Linux, VxWorks OS It is composed of modules such as portability module.

In the test suite template class 110, a test suite may be created, and the methods of the test suite template class 110 become test cases.

A test case is a minimal unit for testing that generates several tests for a module in one method. The test case has the code for the test and an assertion for checking whether the test is satisfied.

The test suite is a set of test cases related to a specific component and assembly. The test suite is added to manage a test case and connects test cases to automate the test.

The test suite storage unit 120 is a storage unit that manages test suites created by the test suite template class 110, and the test result collection unit 130 stores information of the failed test case whenever the test case fails.

The test result output manager 140 outputs the test execution result in the form of a test file, an Extensible Markup Language (XML) file, or a console window.

The runner 15 is responsible for testing the selected test suites in the test suite storage.

2 is a diagram illustrating a unit test framework configuration using a CppUnit according to the prior art.

The unit test framework is a self-test in which a unit of minimum execution unit is developed as a test technique used in extreme programming (XP), and a test case is separated by separating each element of the program. After the test, each unit is tested to make sure that the element is working.

The unit test framework according to the related art includes a library or a DLL (Dynamic link library, hereinafter referred to as a DLL) according to a test suite 200 and a test environment created using the test suite template class 110 of the CppUnit 100. The CppUnit 100 in the form performs unit tests while operating on the target platform 220.

3 is a flowchart of a unit test framework according to the prior art.

An actor (not shown), which is a subject that performs unit tests, may be added directly to the source code of the target platform 210 as a function of performing unit tests, or may be separately written and connected to the target platform.

When the unit test is started, the actor function adds the test result collector 130 to the CppUnit 100 so as to store information of the failed test case (S300), and adds the test suite to be performed to the test suite storage unit of the CppUnit 100. Obtained at 120 (S310) and delivered to the runner 150 of the CppUnit (100) (S320), by specifying the test result collector 130 added to the test through the runner 150 of the CppUnit (100) It performs (S330).

When the test is completed, the test result output manager 140 of the output form created by the actor function is selected and the test result is output (S340).

4 is a diagram illustrating a system test framework configuration according to the prior art.

The system test framework is a logical extension of unit testing, integrating previously tested units into one component and testing the interface between the two units.

The system test framework according to the prior art performs the test using the test suite 200 and the CppUnit (100) created in the CppUnit (100), similar to the unit test framework, actor 230 function that is the subject of the test execution Test harness 300, which serves as a separate implementation, adds. The system test requires a separate test harness because the test needs to be performed to reflect more information such as a regression test or a release target of the test target.

The test harness 300 is a component that executes each step of the test suite, and interacts with other components to drive test cases, such as a configuration manager 310, a test manager 320, and an output. It consists of an output manager 330.

The configuration manager 310 analyzes and manages system test information, for example, a test version, a test date, a test target module, and a list of test suites to be performed from the configuration file, and the test manager 320 starts from the configuration manager 310. With this setup, the developer can decide which tests to run, create and manage test lists, and perform tests according to the list of test suites to be executed.

When the test is completed, the output manager 330 obtains test information from the setting manager 310, obtains a test performing result from the test manager 320, and outputs the result.

5 is a flowchart of a system test framework according to the prior art.

First, when the actor 230, which is the main unit of the unit test, requests the test manager 320 to perform a test (S500), the test manager 320 of the test harness 300 sends a configuration manager 310 to the test performance information configuration. Make a request (S502).

The configuration manager 310 receiving the configuration request reads the test name, the test target module, and the test suite information to be performed from the configuration file, configures the test information (S504), and notifies the test manager 320 of the completion of the test performance information configuration. In operation S506, the test manager 320 requests the runner 150 to start the test in operation S508.

After the test start request of the test manager 320, the runner 150 requests a listener (listener) registration to the CppUnit (100) (S510), and registers the corresponding listener (S512), and finishes the listener registration to the runner 150. Notify (S514).

The listener is a class waiting to be accessed from the outside, and waits for an external request to access the Cppunit (100), and connects when a request comes in.

The runner 100 notified of the completion of the listener registration requests the setting manager 310 for a list of test suites to be performed (S516), and the setting manager 310 executes the test suite information to be performed from the test information that it has. Provided to 150 (S518).

The runner 150 provided with the information of the test suite provides the CppUnit 100 with a list of test suites to be tested (S520), and the test suite storage unit 120 provides a list of test suites provided by the runner 150. Import and register as a test suite to be performed (S522).

When the test suites are registered in the CppUnit 100 (S522), the registration completion report is reported to the runner 150 (S524), and the runner 150 receiving the test suite registration report requests the CppUnit 100 to perform the test ( S526), the CppUnit 100 performs a test (S528) and reports the completion of the test to the runner 150 when the test is completed (S530).

The runner 150 that receives the test performance completion from the CppUnit 100 reports the test performance completion to the test manager 320 (S532), and the test manager 320 requests the output manager 330 to report the test result. (S534).

The output manager 330, which has been requested to report the test result, requests test information from the setting manager 310 (S536), and receives test information from the setting manager 310 (S538).

When the output manager 330, which has received the test information, requests the necessary test performance result from the CppUnit 100 (S540), the test performance result is provided from the CppUnit 100 (S542).

The output manager 330 reports the test result based on the test information received from the configuration manager 310 and the test result provided from the CppUnit 100 (S544), and notifies the test manager 320 that the test result report is completed. If (S546), the test manager 320 reports to the actor 230 that all the process of the test is completed (S548).

As described above, the unit test or system test in the related art uses CppUnit to perform a test to separately configure a test harness corresponding to an actor or actor that is a test subject, and to generate a library or a DLL according to a test environment. Modification of CppUnit is inevitable because it is not easy, but it is difficult to modify because the source of CppUnit, which is open source, is complicated.

Test cases that verify non-functional requirements run for a long time, and because of the high probability of system crash, the test framework for system testing must be able to cope with system crashes. However, if you use CppUnit to test, you can obtain test results after performing tests on all test suites passed to CppUnit's runner, so if a system crash is caused by a test case during test execution, The test result until the occurrence cannot be guaranteed.

In addition, if a test case fails while the test is running, you must terminate the running test case to perform the test for the next test case, but you must modify the CppUnit internals to handle the termination and use the success or failure macro directly. There is a disadvantage.

The present invention has been devised to improve the above-mentioned problems. Independence of the OS for various test environment changes, and a system test framework that can register and execute test cases can be configured to perform system crash even before a system crash occurs. This is to store test information and failed test case information to provide test results.

The objects of the present invention are not limited to the above-mentioned objects, and other objects that are not mentioned will be clearly understood by those skilled in the art from the following description.

In order to achieve the above object, the API test apparatus of the embedded software according to an embodiment of the present invention includes a test suite for registering test cases, an output manager for storing failed test case information, and an output manager for providing a test execution result. .

The API test method of the embedded software according to an embodiment of the present invention includes registering a test case, storing failure information of the test case, and reporting a test performance result.

Specific details of other embodiments are included in the detailed description and the drawings.

Advantages and features of the present invention and methods for achieving them will be apparent with reference to the embodiments described below in detail with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below, but can be implemented in various different forms, and only the embodiments make the disclosure of the present invention complete, and the general knowledge in the art to which the present invention belongs. It is provided to fully inform the person having the scope of the invention, which is defined only by the scope of the claims. Like reference numerals refer to like elements throughout.

6 is a diagram illustrating a GTT configuration of the present invention.

The GTT 400 of the present invention may be composed of a test harness core 410, an interface 420, and a platform 430 as an application related to test case or test suite execution.

The test harness core 410 is a piece of code and data generated for the purpose of supporting a test as part of an environment for testing a system and system components. The test harness core 410 refers to code used for unit test or system test. It may be composed of a manager, a runner, an output manager, a storage unit, and a test suite, which will be described in detail in FIG. 8 described later.

The interface 420 may be composed of a class for writing a test case in accordance with the test harness, a class providing a macro, and a main class providing a macro header file and an interface for executing a test harness. It can consist of structures, public classes, and global header files that contain global variables.

The test harness platform 430 is a layer independent of the OS and the target platform to make system calls dependent on each OS, and to control access to a shared class in the OS, a list class that can be bidirectionally linked. It can be composed of semaphore class that implements semaphore, signal used, thread that is execution code of system test, time class that implements time functions, and waiting object class.

7 is a diagram illustrating a system test framework using the GTT of the present invention.

The system test framework 500 according to an embodiment of the present invention refers to a unit test is logically expanded, integrating already tested units into one component, and testing an interface between the units. This is to identify a problem that occurs when it is combined may be composed of a test harness core 410, a test suite 510, a test harness platform 430, a target platform 520, multiple users create a test case In order to perform a test, a utility (not shown), which is a set of utilities, can be additionally configured to perform a specific function.

The test harness core 410 refers to code and data generated for the purpose of supporting a test as part of an environment for testing a system and system components, and may be generated by a user for use in unit test or module test. In the test harness core 410 performing the above-described purpose, the test suite 510 is analyzed and managed, and the test is performed according to the analyzed list, and when the test is completed, the test information and the execution result are obtained and the result is output. You can manage the process.

The test suite 510 is a set of test cases, and the test case is a documented test case specification that is the basis for performing a test. This can be evidence that the test has been performed, or it can express the range covered by the test. In general, a test case may include information about a measurable state, a situation, an event, and an input / output value.

As described in FIG. 7, the test harness platform 430 is a layer that is independent of the OS and the target platform 520, which is a target platform for testing, and can be linked in both directions. It can consist of a list class, a semaphore class that is a signal used to control access to shared resources in the OS, a thread that is the execution code of system tests, a time class that implements time functions, and a wait target class.

Since the test harness core 410 and the test case code are inserted into modules of the target platform rather than libraries or DLLs, they are compiled together with the target platform.

8 is a test harness core class diagram according to an embodiment of the present invention.

The test harness core class according to the embodiment of the present invention includes a main 600, a test manager 610, a setup manager 620, a runner 630, an output manager 640, a storage unit 650, and a test suite 510. ), May be configured as a platform 430.

The main 600 is connected to the test manager 610 is configured a start class for starting a test, the test manager 610 is the main 600, runner 630, configuration manager 620, and the output manager ( It is connected to the 640, and consists of a starter, a configuration manager, an output manager, and a runner class to determine which tests to run and to create and manage a test list.

The configuration manager 620 is connected to the test manager 610 and the output manager 640. The test target, summary file, log file, iteration count, time, module, and test suite list classes necessary for test setting are configured and managed. can do.

The runner 630 is connected to the test manager 610, the storage unit 600, the test suite 530, and the start, the total number of test cases in which the test is performed, failure, success, error, start time, end time, It can be composed of functions that can obtain the execution time, the number of iterations actually performed, and the like.

The output manager 640 may be configured with a log failure information class that is connected to the test manager 610 and the setting manager 620 to output corresponding information when a test fails. The test suite 530 may include a test suite name and a test. It can consist of a case list class.

The storage unit 650 is a storage unit for managing test suites. The storage unit 650 adds or stores test cases from the test suite 510 to store information about failed test cases whenever a test case fails.

9 is a flowchart of a system test framework in accordance with an embodiment of the present invention.

First, test suites, which are a set of test cases, are declared as global variables when a program is executed for test automation (S900), and test cases owned by the test suite are registered in their test case list (S902).

The test suites request their registration to the storage unit 600 (S904), and when the storage unit 600 reports the success of registration of each test suite to the test suite (S906), the test suite requests a test execution to the main 600. (S908). The main 600 may be an entry point to an externally provided interface class.

The main 600 requests the test manager 610, which is a real manager, to perform a test (S910), and the test manager 610 sends test information to the configuration manager 620, for example, a test version and a test. Date, the configuration request for the test target module and the test suite to be performed (S912).

The configuration manager 620 receiving the configuration request reads the test information and the test suite list to be performed from the configuration file to configure the test information (S914), and reports the completion of the configuration of the test information to the test manager 610 (S916). .

When the configuration of the test information is completed (S916), the test manager 610 requests the test runner to perform the test (S918), and the runner 630 requests the test suite list from the setting manager 620 (S920). ). The setting manager 620 provides test suite information to the runner 630 according to the request (S922).

The runner 630 requests the test suite of the corresponding list from the storage unit 600 (S924), receives the corresponding test suite (S926), and the runner 630 based on the test suite provided from the storage unit 600. The test suite requests the test suite to perform a test case that the test suite has (S928), and the test suite performs a test on its test case (S930).

When the test suite delivers the test execution result to the runner 630 (S932), the runner 630 stores the test performance result (S934).

The execution method of the execution result is delivered in an exception manner, and in case of failure, it can be delivered in a failure exception.

The exception approach here is that the exception classes in the C ++ library provide the programmer with a basic error model. Based on this, you can write methods for error detection and notification that are appropriate for your application.

The standard C ++ library provides classes for error handling, which use the exception handling capabilities of C ++. The standard C ++ library implements the error model by largely dividing it into logic errors and runtime errors.

The runner 630 stores the received test execution result (S934), and requests the output manager 640 to record the failure information (S936), and the output manager 640 stores the failure information (S938). In operation 940, the terminal 630 reports completion of the failure information storage.

When the storage completion report for the failure information is reported to the runner (630) (S940), the test request for the other test case is requested (S942).

The runner 630 requests a test suite to perform a test case for all test cases of the test suite, performs a test on the test case, passes a test result of a test case, and a runner 630. Steps from the execution result storage step, the failure information recording request and the failure information storage step, and the failure information storage completion report are performed (S928 to S940).

When the test run request for the test case is received in the runner 630 (S942), the runner 630 requests the test suite to the storage unit 600 for all the test suites, the corresponding test suite delivery step, and the test case execution. Request step, Test execution step of test case, Execution result delivery step of test case, Execution result storage step, Record request step for failure information, Completion reporting step for saving failure information and saving failure information, Request for other test case execution The process until the step is performed (S924 ~ S942).

When the runner 630 reports that the test manager 610 has completed all of the tests (S946), the test manager 610 requests the output manager 640 to report the test result (S948).

The output manager 640 requests test information, for example, test version information, test name, and test module name information, from the setting manager 620 (S950), and the setting manager 620 transmits the corresponding information to the output manager (620). 640) (S952).

When output manager 640 asks runner 630 for a list of test performance information, for example, the total number of test cases that performed a test, the number of successful test cases, the number of failed test cases, and the failed test case information list ( In operation S954, the test performance information is transmitted to the output manager 640 (S956), and the result is stored (S958).

When the result is stored in the output manager 640 (S958), the completion of the result storage is reported to the test manager 610 (S960), and the test manager 610 indicates that all the test processes are completed in the GTT main 600. By reporting (S962) and reporting that the test has been completed in the test harness (S964), all the processes of the test may be completed.

10 illustrates a test setup file according to an embodiment of the present invention.

The test configuration file according to an embodiment of the present invention includes test version information 700, test name 710, summary file 720, log file 730, repetition number 740, module 750, test suite ( 760), and can be used as basic information of the result report.

In the test version information 700, the version name of the test subject and the title name of the test to be performed may be described in the target, and the summary file 720 may record the entire test result, and the log file 730 may indicate each test case. Each time is executed, the log and failed test case information can be recorded.

The internal log file (not shown) records a message related to the test harness. If a system crash occurs during the test, the summary file 720 is not generated, but the log file 730 is a log file before the crash and failed test case information. You can record all of them.

If you do not specify the name of the log file (730), and the default is written, the summary file 720 is recorded as, for example, TS_YYYYMMDDHHMM.csv YYYY year, MM is month, DD is date, HH is hour, and MM is minute. The log file 730 is generated in two formats, for example, TL_YYYYMMDDHHMM.log and TI_YYYYMMDDHHMM.log. The former is log file 730 and the latter is internal log file. In both cases, as in the summary file 720, it may be recorded as year, month, day, hour, and minute.

The number of iterations 740 is represented by a constant number of iterations of the selected test suites, and module 750 is the target module name of the test and can be used when reading from other applications. The test suite 760 may consist of a list of test suites to be performed.

11 is a diagram illustrating test results according to an embodiment of the present invention.

The test result report according to an embodiment of the present invention is a summary file indicating a test result when a configuration file is set in the configuration manager as shown in FIG. 10. The test result report is stored in a CSV file format in consideration of a Linux environment as well as a Windows environment and failed. If you have a test case, you can send it to the program.

The summary file 720 includes a test information section 800 representing test information, a test result statistics section 810 representing test results, a failure list section 820 listing failed test case information, an error list section in which an error occurred ( 830 may be displayed.

The test information section 800 displays a test version name, a test title, a test target module name, a selected test suite iteration number, an actual number of iterations, and an execution time.

The test result statistics section 810 may display a total number of test cases performed, a number of successful test cases, a number of failed test cases, and a test case in which an error occurs.

The test case in which the error occurred means that the test case was interrupted because an exception occurred other than the exception caused by the pass or fail decision macro.

The failure list section 820 contains the name of the failed test suite and the name of the test case included in the test suite, the name and location of the source file where the failure occurred, the line number of the source where the failure occurred, and the status of the results that were actually performed by the description message created in the macro. Information can be displayed.

The error list section 830 may display detailed information of a test case in which an error occurs, for example, a test suite name, a test case name, and exception type information.

Although the embodiments of the present invention have been described above with reference to the accompanying drawings, those skilled in the art can understand that the present invention can be implemented in other specific forms without changing the technical spirit or essential features. will be. Therefore, it should be understood that the embodiments described above are exemplary in all respects and not restrictive.

The embedded software API test apparatus and method of the present invention as described above has one or more of the following effects.

First, since the test harness and test case code are inserted into the software module of the target platform rather than the library or DLL type, there is an advantage that the test harness and the test case code are compiled together without the configuration.

Second, the test harness has a platform independent of each OS, and even for OSs not supported by the platform, the system call of the corresponding OS can be added to the mapping without modifying test case code written using the test harness and platform. There is also an advantage.

Third, the test harness consists of a configuration manager that can select the test suite to test, set test information, the number of iterations, repeat time, etc., and an output manager that can specify the output form of the test results. Therefore, it is easy to modify, and if there is a system crash, the test case log before the crash and the failed test case information are recorded in separate log files, which can satisfy various test requirements.

Claims (5)

Registering a test case; Storing failure information of the test case; And And reporting the result of performing the test. The method of claim 1, The test harness is composed of OS independent layers so that even if the OS is not supported by the test harness, the software is configured separately or the test harness is added to an independent layer of the test harness without modification of the test case. API test method. The method of claim 2, The API test method of the embedded software to save the log and the failed test case information until the system crash occurs in a log file. A test suite unit for registering a test case; An output manager for storing failed test case information; And An API test device in embedded software that includes an output manager that provides test performance results. The method of claim 4, wherein The API test apparatus of the embedded software for storing the log and the failed test case information until the system crash occurs in a log file.

Images