KR20100042825A - Method and system for performing automatic test in embedded platform - Google Patents

Abstract

PURPOSE: A method and a system for performing an automatic test in an embedded platform are provided to stabilize a test and reduce a test cost by using the storage and recovery of an embedded platform state. CONSTITUTION: An RTOS(Real Time Operating System)(200) processes an event inputted in a PC emulator or a hardware device based on priority. An embedded platform(300) provides an API(Application Programming Interface) by which an application program can be executed. An event monitor(400) stores the previous state of the embedded platform and performs a test. When performing a reproduction test, the event monitor performs the test by recovering the embedded platform into the previous state.

Description

Method and system for performing automatic test in embedded platform

The present invention relates to a test automation method in a system using an embedded platform, and more particularly to a test automation method and system that helps to minimize the error of the application by automatically testing the application developed using the embedded platform. .

An embedded system refers to a system having only special functions by embedding software for operating the system in hardware. Unlike an personal computer, an embedded system has specific requirements and performs only predefined tasks. In other words, unlike PCs that have built-in operating systems in mass storage devices, such as hard disks, embedded systems store operating systems and applications in the form of images in ROM (flash) and then create RAM disks at boot-up. A system on which the OS and applications run.

As such, embedded systems are closely related to the application and vary greatly as the system changes. In addition, the test work to find the error of the application frequently occurs. If the program is not easy to update, if the program is not developed reliably from the beginning, it may lead to a large cost increase later, and it may be very difficult to restore the reliability.

Therefore, embedded system needs to be developed very carefully from the beginning of development, but it is not easy to test, so many tests are not carried out, and many test personnel are invested after development, but there are not enough tests due to time or cost problems. many.

Common test methods include unit box-based white box tests to test the integrity of the code itself, and black box tests that compare specifications and results without understanding the development of the program. box test).

White box testing is a way for a developer to inject test code into a program during development to continuously test whether the code is stable during development. In a black box test, a tester who accepts a result compares an input with its output, It's a way to compare whether or not it is.

Embedded systems often perform repetitive black box tests based on test scenarios created by professional testers, and have very large test scenarios depending on the number and complexity of applications. As a result, the number or time of test personnel increases exponentially. Test automation methods are used to solve this problem.

Test automation is a method that helps users repeatedly test the input operation by automating the test method by manually inputting a key such as a key input or folder opening and closing. At this time, the input operation can be edited, and the embedded platform can be executed by executing commands other than the operation.

However, in the case of using the test automation method, since the state of the embedded system is not applied but only the input event is constantly input, the test can be performed differently according to the state of the embedded system. It's not working.

In addition, the test results of the conventional test, while the tester directly tests the embedded system, the tester checks with the eyes directly to determine when the error occurred, and the debugging is performed in consideration of the cause of the error. It is not possible to detect the accurate information according to the test results, and also the result of the tester results are different, there is a problem that the debugging efficiency is lowered.

Accordingly, the present invention has been made to solve the above problems, and by reducing the time and manpower required for testing while reducing the cost and manpower required for the test by automated storage and recovery of the state of the embedded platform in the automated test of the result made on the embedded platform The purpose is to provide a test automation method and system in an embedded platform.

Another object of the present invention is to embed the evaluation system in the embedded platform to leave accurate results after the test and the test content to the communication between the developer and the tester becomes clear, even when the test is carried out remotely, the complex test smoothly To provide a test automation method and system in an embedded platform that can be used.

Another object of the present invention is to emulate the embedded system on the PC using the system-independent features of the embedded platform to test and test the same test scenarios in the embedded system to help reproduce and solve the problem. To provide a test automation method and system in an embedded platform that can provide a solution.

Features of the test automation method in the embedded platform according to the present invention for achieving the above object is (A) collecting and analyzing the state of the embedded platform to store in a state that can be loaded later, (B) of the embedded platform Performing a test; (C) restoring an embedded platform to a previous state of the stored embedded platform; and (D) performing steps (B) to (C) with the embedded platform restored to a previous state. And repeatedly performing a reproducing test (repeating test).

Preferably, when the reproduction test is completed, performing a tolerance evaluation by comparing the test result with a tolerance value based on a pre-stored automatic evaluation equation, and for all values whose test results deviate from the tolerance value through the evaluation. And outputting a pre-standardized output value.

Preferably, in step (A), when the state storage request of the embedded platform is received through an event monitor, collecting and analyzing the state of the current embedded platform, and changing the requested state information into a form capable of transmitting the requested state. And transmitting the event information to the event monitor, analyzing the state information of the embedded platform, and storing the state information in a state file DB for later loading.

Preferably, the step (B) generates a character string according to the key log format when the event input through the key input unit in the real hardware (H / W) device or the PC emulator is a key event. Create event log by generating in general log form and record in event file DB, interpret event by playing event log written according to event or format recorded in event file DB, and interpreted event is embedded platform And an event playing step of analyzing the received event and pushing the analyzed event to an event queue to perform the same event processing as that of the H / W device. It is done.

Preferably, the step (C) includes loading the stored state of the previously embedded platform into an event monitor and loading it into a memory, and processing the file from the loaded memory into a state where a file is analyzed and transmitted to the embedded platform. And receiving the processed data from the embedded platform, recovering the state of the embedded platform after analyzing the received data, and delivering a completion message to the event monitor.

Features of the test automation system in the embedded platform according to the present invention for achieving the above object is a Real Time Operating System (RTOS) for processing the events input from the hardware device or PC emulator based on the priority and the application program An embedded platform providing an executable application program interface, and an event for restoring and executing the embedded platform to the stored previous state when performing a reproducing test (repeating test) while storing and performing a previous state of the embedded platform It includes a monitor.

Preferably, the embedded platform includes an application API module for providing an application program interface for executing an application program, a test window management module for outputting a test result of the embedded platform, and event management for processing an event transmitted through the RTOS. A module, a status management module for collecting and analyzing the status information of the embedded platform requested from the event monitor, converting the analyzed information into a form that can be transmitted, and transmitting the result to the event monitor; It is characterized in that it comprises an event evaluation module for evaluating the value of the allowable value internally based on each time, and performing the evaluation according to an event or a specific input and transmitting the content to the event monitor when a problem occurs.

Preferably, the event monitor includes a state processor for storing and restoring a state of an embedded platform based on a test performance, and an event processor for testing an embedded platform on an embedded event.

Preferably, the state processing unit loads state information of a previous embedded platform previously stored in a state file DB in a state file DB, analyzes a file, and transmits the state information to the embedded platform, and the embedded platform received from the embedded platform. Receiving the status information, and analyzes the status information, characterized in that it comprises a status recorder for storing in the state file DB in a state that can be loaded later.

Preferably, the event processing unit determines an event to be input and changes it to a key log form or a general log form, and outputs the event changed in the event reproducing unit to a window of an event monitor and stores it in an event file DB. And an event transmitter configured to load an event stored in the event file DB into an event loader to analyze an event and transmit the analyzed event to an embedded platform.

Test automation method and system in the embedded platform according to the present invention as described above has the following effects.

First, it is possible to stabilize the embedded system, which is inconvenient to test, by repeatedly and quickly performing accurate tests by supporting the automatic test of the manual test on the embedded platform.

Second, unlike the existing test automation system, by using the function of saving and restoring the state of the embedded platform, it can be tested in the state of the same embedded platform even during the reproducible test (repeated test), which enables more accurate testing of events. It works.

Third, in the case of conducting many kinds of tests by embedding evaluation expressions in the embedded platform, it is possible to evaluate not only the malfunction of the GUI or the operation itself but also the use of the wrong factor so that the malfunction can be accurately identified in advance. There is.

Other objects, features and advantages of the present invention will become apparent from the following detailed description of embodiments with reference to the accompanying drawings.

A preferred embodiment of the test automation method and system in the embedded platform according to the present invention will be described with reference to the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below, but can be embodied in various forms, and only the present embodiments are intended to complete the disclosure of the present invention and to those skilled in the art to fully understand the scope of the invention. It is provided to inform you.

1 is a block diagram showing the configuration of a test automation system in an embedded platform according to an embodiment of the present invention.

As shown in FIG. 1, a test automation system real-time multiple programming of a hardware (H / W) device or PC emulator 100 and an event input from the H / W device or PC emulator 100 at a high priority potential. In the lower priority, the RTOS (Real Time Operating System) 200 for batch processing, the embedded platform 300 for providing an application program interface for executing an application, and the previous state of the embedded platform 300 And an event monitor 400 for restoring and executing the embedded platform to the stored previous state when performing a reproducing test while storing and performing a test.

 In this case, the embedded platform 300 includes an application API module 310 for providing an application program interface for executing an application program, a test window management module 330 for outputting a test result of the embedded platform, and the RTOS. The event management module 320 for processing the event delivered through the event monitor 400 collects and analyzes the status information of the embedded platform requested from the event monitor 400, and changes the analyzed information into a form that can be transmitted to the event monitor 400. Built-in automatic evaluation formula and state management module 340 to transmit to the internal evaluation based on the test results each time the value for the allowable value, and performs the evaluation according to the event or a specific input to monitor the contents of the event when a problem occurs It includes an event evaluation module 350 to deliver.

In this case, the automatic evaluation formula includes a range of operations to be predicted according to the characteristics of various functions or UI components of the embedded platform, and outputs a message when the range is exceeded or falls short of this range to indicate at which point the malfunction began. Means.

In addition, the event monitor 400 stores and restores the state of the embedded platform 300 based on the test performance, and the input event for testing the embedded platform on the embedded platform 300. It includes an event processor 430.

At this time, the state processing unit 410 loads the state information of the previous embedded platform 300 stored in the state file DB 420 in advance when performing the reproduction test, and analyzes the file and transmits the state information to the embedded platform ( 412 and a state recorder 414 which receives the state information of the embedded platform received from the embedded platform 300, analyzes the state information, and stores the state information in the state file DB 420 in a state where it can be loaded later.

In addition, the event processing unit 430 determines an event to be input and changes it to a key log form or a general log form, and an event reproducing unit 434 and outputs the event changed in the event reproducing unit to a window of an event monitor. An event recorder 438 storing the event file DB and an event stored in the event file DB 440 for loading the event into the event loader 432 to analyze the event and transmit the analyzed event to the embedded platform. And an event transmitter 436.

The operation of the test automation system in the embedded platform according to the present invention configured as described above will be described in detail with reference to the accompanying drawings.

2 to 7 are flowcharts illustrating a test automation method in an embedded platform according to an exemplary embodiment of the present invention.

Referring to FIG. 2, before testing the embedded platform, the event monitor 400 first collects and analyzes the state of the embedded platform 300 and stores the state in a state that can be loaded later (S100).

At this time, referring to Figure 3, the process of storing the embedded platform state in detail, first, when the event monitor 400 requests the state storage to the embedded platform 300 (S110), the embedded platform 300 is a state management module ( In step 340, the request command is received through serial or other communication means (S120), and the state of the current embedded platform is collected and analyzed (S130). At this time, the state of the embedded platform 300 depends on the call stack of the state machine, various environment variables and the DB, all of these functions are recorded in the embedded platform.

The embedded platform 300 changes the state information analyzed by the state management module 340 into a form that can be transmitted and transmits it to the requested event monitor 400 (S140).

The event monitor 400 receives the transmitted state information of the embedded platform (S150), analyzes the state information through the state recorder 414, and changes the state to a useful file storage state (S160). The state information analyzed by the state recorder 414 is stored in the state file DB 420 in a state where it can be loaded later (S170).

As such, when the storage of the state information of the embedded platform 300 is completed (S100), the test of the embedded platform is then performed (S1000).

1. Log the event.

In step S110 of testing the embedded platform, event log writing is performed (S200). Looking at the process of creating an event log with reference to Figure 4,

When a key is pressed through the key input unit in the real hardware (H / W) device or the PC emulator 100, a corresponding input signal is transmitted to the Real Time Operating System (RTOS) 200, and then the RTOS 200 Transmits an event according to the transmitted input signal to the embedded platform 300 based on the priority. In this case, the H / W device receives the event through the serial, and the PC emulator receives the event through the window event.

As such, the embedded platform 300 starts event processing through the event management module 320, and the processed event is converted into an ASCII string through a serial device or other communication means to the event monitor 400. Transfer (S210).

Accordingly, the event monitor 400 reproduces the event converted into an ASCII character string through the event processing unit 430 and determines whether the input event is a key event (S220).

In the determination result (S220), if the input event is a key event, a string is generated in which state was pressed according to the key log format (S240). To generate (S230).

The generated key event and the general log are recorded in the event file DB 440 through the event recorder 438 to perform event log creation (S250).

2. Play the event.

Subsequently, the event is reproduced as the test of the embedded platform (S1000). Looking at the event playback process in detail with reference to Figure 5,

When the event playback proceeds, first, an event log written according to an event or format already recorded in the event file DB 440 is loaded into the memory (S310). The event inputted through the event reproducing unit 434 is reproduced to analyze whether it is a key event, a command, or other contents (S320).

If the analyzed result (S320), the analyzed event is a command code is performed internal command processing (S340). One embodiment of the internal command processing refers to a command operation such as jumping to a specific line or waiting for time. In addition, the analysis result (S320), if the analyzed event is a key event, the key event is interpreted (S330).

In addition, the command processed or interpreted event is transmitted to the embedded platform 300 through the event transmitter 436 (S350).

2. Perform the event.

Subsequently, an event is performed as the progress of the test of the embedded platform (S1000). Looking at the event execution process in detail with reference to Figure 6,

The embedded platform 300 is received through the event monitor 400 (S410), and analyzes the received event (S420).

Then, it is determined whether the analyzed event is a command or a key event (S430). If the determined event is a command, internal command processing is performed (S440). If the determined event is a key event, the corresponding event is pushed to an event queue. In operation S450, the same event processing as that of the H / W device is operated is performed.

Subsequently, when the received event processing is finished, a result of the processing task is delivered as an event again and the processing performed is terminated (S460).

When the process of the test of the other embedded platform (S1000) is ended in the received event as described above, the embedded platform is restored to the previous state of the embedded platform stored in the state file DB.

In this case, referring to FIG. 7, the process of storing the embedded platform state is described in detail. First, the event monitor 400 loads the state of the previous embedded platform stored in the state file DB 420 and loads the state in the memory (S510). .

The file is analyzed in the loaded memory and processed in a state in which transmission is possible (S520). The processed data is transmitted to the embedded platform 300 (S530), and the embedded platform 300 is received through a serial or other transmission medium (S540).

Subsequently, the embedded platform 300 recovers the state of the embedded platform after analyzing the received data (S550). When the restoration of the state information is completed, the embedded platform 300 transmits a completion message to the event monitor 400 (S560).

Then, the event monitor 400 receives the transmitted message and completes recovery (S570).

In addition, the embedded platform 300 reproduces the event stored in the event file DB 440 in a state in which the embedded platform 300 is restored to a previous state, and performs a reproducing test by processing an event performance (S600).

As described above, after the state of the embedded platform 300 is tested, the state of the embedded platform 300 is restored, and when the test is performed after the state is restored to the state of the first embedded platform stored, the test can be more accurate. At this time, all the applications operate on the embedded platform and are dependent on the application programming interface (API) provided by the embedded platform, so the state of the embedded platform can be changed to the state before the test by saving and restoring.

In addition, the event evaluation module 350 in the embedded platform 300 includes an allowable value according to an input value of GUI control or various functions of the embedded platform through an automatic evaluation expression, and the allowable value is exceeded by comparing the tested result with the allowable value. Tolerance evaluation is performed through whether or not (S700). In this case, the automatic evaluation formula includes a range of operations to predict according to the characteristics of various functions or UI components of the embedded platform.

The result of evaluating the allowable value evaluation result (S700) and a test result out of the allowable value is evaluated as a problem occurred, and when a problem occurs, the output value standardized and stored in advance for all values is output (S800). At this time, pre-standardized and saved output values are loaded into the output file when a problem occurs in the embedded platform, and the problem occurs in which state, and the event is the stored information. The saved output value is the problem occurrence state, problem occurrence event, problem Causes of occurrence, including the importance and impact of the system.

The generated output value and problem occurrence information are transferred to an event monitor and stored in a file (S900).

As a result, when a problem occurs, the problem can be detected internally first. It outputs a warning message when the embedded platform is operating outside the range expected by the embedded platform, so that it can be combined with the white box test and the black box test during development. help.

Although the technical spirit of the present invention described above has been described in detail in a preferred embodiment, it should be noted that the above-described embodiment is for the purpose of description and not of limitation. In addition, those skilled in the art will understand that various embodiments are possible within the scope of the technical idea of the present invention. Therefore, the true technical protection scope of the present invention will be defined by the technical spirit of the appended claims.

1 is a block diagram showing the configuration of the test automation system in the embedded platform according to an embodiment of the present invention

2 to 7 are flowcharts illustrating a test automation method in an embedded platform according to an embodiment of the present invention.

* Explanation of symbols for main parts of the drawings

100: hardware (H / W) device or PC emulator

200: RTOS 300: Embedded Platform

310: Application API Module 320: Event Management Module

330: window management module 340: state management module

350: event evaluation module 400: event monitor

410: state processing unit 412: state playback unit

414: status log 420: status file DB

430: event processing unit 432: event loading unit

434: event reproducing unit 436: event transmitting unit

438: event log 440: event file DB

Claims (13)

(A) collecting and analyzing the state of the embedded platform and storing it for future loading; (B) testing the embedded platform; (C) restoring an embedded platform to a previous state of the stored embedded platform; (D) repeating the steps (B) to (C) to perform a reproducing test in a state where the embedded platform is restored to a previous state. The method of claim 1, When the performance of the reproduction test is completed, comparing the test result with a tolerance based on a pre-stored automatic evaluation equation, and performing tolerance evaluation; And outputting a pre-standardized output value for all values whose test result exceeds the allowable value through the evaluation. The method of claim 2, The automatic evaluation formula is a test automation method on an embedded platform, characterized in that it includes a tolerance having a range of operations to predict according to the characteristics of the various functions or UI components of the embedded platform. The method of claim 2, The pre-standardized and stored output values are loaded into an output file when a problem occurs in an embedded platform, and a problem occurs in a certain state, and the event is analyzed. The method of any one of claims 1 and 2, wherein step (A) When the state monitor request of the embedded platform is received through the event monitor, collecting and analyzing the state of the current embedded platform; Transmitting the analyzed state information in a form that can be transmitted to the requested event monitor; And receiving the transmitted state information of the embedded platform, analyzing the state information, and storing the state information in a state file DB that can be loaded later. The method according to any one of claims 1 and 2, wherein step (B) If the event input through the key input unit in the real hardware (H / W) device or PC emulator is a key event, a string is generated according to the key log format. If the event is not a key event, the character string is generated in the general log format. Performing event logging by recording in a DB;  An event reproducing step of reproducing an event by reproducing an event log written according to an event or a format recorded in the event file DB, and delivering the interpreted event to an embedded platform; And an event execution step of analyzing the received event and pushing the analyzed event to an event queue to perform the same event processing as that of the H / W device. . The method according to any one of claims 1 and 2, wherein step (C) Loading the stored state of the previously embedded platform into an event monitor and loading it into memory; Processing the file in the loaded memory so that the file can be analyzed and transferred to an embedded platform; Receiving the processed data in an embedded platform; And recovering a state of an embedded platform after analyzing the received data, and transmitting a completion message to an event monitor. Real Time Operating System (RTOS), which processes events input from a hardware device or PC emulator based on their priority, An embedded platform that provides an application program interface on which an application can run, And an event monitor for restoring and executing the embedded platform to the stored previous state when performing a reproducing test while storing the previous state of the embedded platform and performing a test. The method of claim 8, wherein the embedded platform An application API module that provides an application program interface on which the application can run; A test window management module that outputs test results of the embedded platform, An event management module for processing an event delivered through the RTOS; A state management module for collecting and analyzing the state information of the embedded platform requested from the event monitor, changing the analyzed information into a form capable of transmission, and transmitting the state information to the event monitor; Built-in auto-evaluation formula to evaluate the value of the allowable value internally based on the test results each time, and performs an evaluation according to the event or a specific input, and includes an event evaluation module that delivers the contents to the event monitor when a problem occurs Test automation system on an embedded platform. The method of claim 9, The automatic evaluation formula is a test automation system for an embedded platform, characterized in that it includes a range of operations to predict according to the characteristics of the various functions or UI components of the embedded platform. The method of claim 8, wherein the event monitor A state processing unit which stores and restores the state of the embedded platform based on a test execution; Test automation system in the embedded platform, characterized in that it comprises an event processing unit for testing the embedded platform on the embedded platform the input event. The method of claim 11, wherein the state processing unit A state reproducing unit which loads state information of a previous embedded platform previously stored in a state file DB, analyzes a file, and transmits the state information to the embedded platform when a reproduction test is performed; Receiving the status information of the embedded platform received from the embedded platform, the test automation system on an embedded platform, characterized in that it comprises a state recorder for storing the state information in a state file DB to be loaded later. The method of claim 11, wherein the event processing unit An event reproducing unit which determines an event to be input and changes it into a key log form or a general log form; An event recording unit for outputting the event changed by the event reproducing unit to a window of an event monitor and storing the event file in an event file DB; Test automation system in the embedded platform, comprising an event transmitter for loading the event stored in the event file DB to the event loading unit for event reproduction to analyze the event, and transmit the analyzed event to the embedded platform.

Images