KR101091457B1 - Online system test method - Google Patents

Abstract

The present invention relates to an on-line system test method for applying a hooking technique and a profiling concept to an on-line system test to simultaneously perform a system test that minimizes resource overhead and collect test information for identifying a cause and location of a defect. Specifically, the present invention discloses an online system test method comprising defining a defect model, inserting a test agent, hooking a test location, collecting test information, and removing the test agent. do.

Description

Online system test method {ONLINE SYSTEM TEST METHOD}

The present invention relates to a method of testing an online system.

Recently, 'IT convergence', in which IT is applied to existing traditional industries, has been in the spotlight, and IT convergence is being promoted as a solution for strengthening competitiveness in various industries such as medical-IT, automotive-IT, aviation-IT, and construction-IT. In the early days, the medical field's IT grafting began to minimize medical accidents during surgery, and it provides rapid and accurate image information, and precise surgery has shown much fruit in research of 'remote robot image guidance system' replaced by robots. The automotive sector is also attempting to converge IT for the development of intelligent automobiles, focusing on telematics, navigation and location based services (LBS), and human machine interfaces (HMI).

In the IT convergence industry, it is more important than the operation of the software in the operating environment of the fused system rather than the software running independently. In addition, since the part that can confirm the operation by the software itself is relatively limited, the operation by inputting the live environment with real data, which is obtained as the system is actually operated, is not the virtual data for the test. In addition, system testing at online operation time is a very important issue. In particular, online operation time requires testing to find potential defects that occur during the long run of the system or defects that cause the system to crash.

One of the prior art system test methods is a black box test. Black box testing is done by using the system's usage scenario as a test case to verify that the system is operating correctly according to requirements.

However, the black box test has a problem that it is difficult to identify the location and cause of the found defect.

In most industrial sites, short product cycles and short development schedules to preoccupy the market tend to stabilize quality through robust system testing later in development rather than step-by-step testing. This test trend is due to the cost of testing resources and costs, such as the need to equip each of the software constituting the system independently and independently with the replacement of the connected partner components.

In the late stages of development, tests focused on defects must be included in all layers of the system, and the black box test makes it difficult to identify the cause and location of the defects, which increases the test period, thereby increasing the development cost. There is a problem.

There is a method using a debugger as a method for obtaining system test information according to the prior art.

However, the debugger is not easy to use when you need to perform tests in an actual system environment, and when the time-critical characteristics of a real-time system are used, the debugger overhead causes the system to fail. The problem is that it may not work properly.

Another method of obtaining system test information according to the related art is to insert test code into an object.

For example, inserting code that measures CPU or memory usage into a timer routine to profile performance periodically, or insert code that monitors parameters when calling a function for API testing.

However, the method of inserting test code into an object still has problems such as memory space overhead due to additional code and performance overhead due to the execution of the code.

In order to solve the above problems, the present invention introduces a hooking technique to the process management method of the operating system, by incorporating a profiling technique used to collect system state information, to ensure the real-time operation of the system as much as possible, A new on-line system test method is disclosed that tests the dynamic situation of a program (process) and also obtains test information for identifying the cause and location of the defect.

The present invention includes the steps of defining a defect model; Inserting a test agent;

Hooking a test location; Collecting test information; And removing the test agent.

In addition, the step of defining the defect model of the present invention includes the steps of defining a test position according to a defect type; Defining test information according to the defect type; And generating a test code for collecting the test information.

Additionally, inserting the test agent of the present invention comprises: constructing a list for running processes; And inserting the test agent based on the list.

Additionally, hooking the test location of the present invention may comprise identifying a memory region according to the test location; An online system test method is disclosed that includes hooking the identified memory region with the test code.

Additionally, the collecting of the test information of the present invention may include collecting program execution information, process context information, and system state information.

Additionally, removing the test agent of the present invention may include restoring the test location; And removing the test agent.

The online system test method according to the present invention has the advantage that all processes and dynamic situations can be tested while the actual system is operating because the system is booted and hooks a specific memory space while all the images are loaded into the memory.

In addition, the system test method according to the prior art stores the core execution information on the current process and the thread currently running in a specific memory space (CPS) for process management, even if there are many processes running in the system. The on-line system test method according to the present invention has the advantage of minimizing the system overhead due to the test by using a hooking method utilizing the program context information of the CPS rather than inserting individual code for each program through the operating system. have.

Finally, the on-line system test method according to the present invention applies the profiling technique used to measure the performance of the system to the test to detect and repair defects during on-line system testing with severe 'run-time, real-time, resource' constraints. The advantage is that you can get test information.

The meanings of the terms used in the present application are as follows.

Online means a state in which a function or service of a system is actually operating or a state in which a request for a system is immediately available.

Online system testing means testing the system in an environment in which the system is deployed and installed.

Run-time refers to a "dynamic situation in which the system runs while it is running, that is, from system startup to shutdown, compared to compile time." Activities at run time include various dynamic behaviors, such as not only running a program, but also dynamic interworking situations that load and integrate related components, multithreaded operations, and managing shared resources between processes or threads. .

Real-time refers to a situation in which a system or mode of operation immediately processes a request from an external process and returns its results within the promised time.

Hooking is a programming technique that intercepts the system's operation and control at runtime and changes it in the desired direction for a specific purpose.

Hooking technology can be used to improve the functionality of already deployed software products, such as patches, or to create update programs to fix minor bugs, and to access and collect information from other people without your consent, such as spyware. It can also be abused as a modification technique. It is often used for debugging purposes related to the operating system because it can be used to 'peek operating system' using hooking technology.

Hooking is a type of code instrumentation, classified into static and dynamic methods depending on when the code is inserted.

The static approach is to modify the original code before the program is run, by adding a set of test code to the source or binary code at compile time. As a result, a program with static test code inserted is always tested at run-time in a fixed manner, and the resource overhead of memory and CPU is fixed during the system life cycle. In addition, it is not easy to apply to online test because it is not easy to consider dynamic situation such as DLL linkage.

The dynamic approach is to insert test code at run time. Compared to the static approach, it is more suitable for online testing in that it can also test dynamic situations that are identified at run-time.

The on-line system test method according to the present invention refers to a dynamic hooking, and targets the binary code that is actually mounted and operated in the system rather than the source code to target the final distribution code.

Also, since online system test means testing all programs that run on the system, not just one program running on the system, all programs running on the system are not hooked. The concept of OS-hooking is applied through the operating and operating system.

Profiling is a performance analysis technique that measures program behavior or system state at run time.

Profiling is to optimize the system, so it is important to measure the system's performance to find performance bottlenecks. The present invention utilizes profiling to collect test information to identify which program in the system the problem is caused by a test, what part of the program is associated with the defect found in the test.

Hereinafter, with reference to the accompanying drawings will be described an embodiment of the present invention;

1 is a flowchart of an online system test method according to the present invention.

Referring to Fig. 1, an on-line system test method according to the present invention is started.

The online system test method according to the present invention first defines a defect model (S101).

Defining a fault model means defining where to test according to the fault and test information for fault detection and repair.

2 is a flowchart illustrating a specific process of defining a defect model in an online system test method according to the present invention.

 2, the online system test method according to the present invention defines a test position according to the defect type (S1011).

Next, the online system test method according to the present invention defines the test information according to the defect type (S1012).

Then, the online system test method according to the present invention generates the test code for the execution of the original code and the collection of test information for the defined test location (S1013).

The process of defining a defect model of the online system test method according to the present invention is terminated.

After defining the defect model, the test agent is inserted (S102).

Inserting a test agent means inserting (infiltrating) a test agent that performs test location hooking and test information collection into all processes running online.

3 is a flowchart illustrating a specific process of inserting a test agent in the on-line system test method according to the present invention.

Referring to FIG. 3, the online system test method according to the present invention constructs a list of processes operating in the system (S1021).

Next, the online system test method according to the present invention inserts a test agent based on the configured process list information (S1022).

The process of inserting the test agent of the online system test method according to the present invention is terminated.

After inserting the test agent, the online system test method according to the present invention hooks the test position (S103).

Hooking a test location means hooking the OS's Current Process Space (CPS) so that the test code, including the behavior of the original code, is executed when the inserted test agent passes by the test location.

4 is a flowchart illustrating a specific process of hooking a test position in an on-line system test method according to the present invention.

Referring to FIG. 4, the online system test method according to the present invention identifies a memory area to be hooked in the CPS for each test location (S1031).

As a result of identifying the memory area, i) If the test location is a SYSTEM API (Application Process Interface) function, the APISet of the CPS is identified as the hooking area.

ii) In case the test location is DLL (Dynamic Linking Library) EXPORT function, IAT (Import Address Table) of CPS is identified as hooking area.

iii) Otherwise, the code space of the CPS is identified as the hooking area.

Next, the on-line system test method according to the present invention hooks the identified hooking area with the test code (S1032).

The process of hooking the test position of the online system test method according to the present invention is terminated.

After hooking the test position, the online system test method according to the present invention collects test information (S104).

Collecting test information means that when the system executes the hooked location for testing purposes, it collects information for testing.

5 is a flowchart illustrating a specific process of collecting test information in an online system test method according to the present invention.

Referring to FIG. 5, the online system test method according to the present invention collects program execution information necessary for identifying a cause of a defect (S1041).

Next, the on-line system test method according to the present invention collects process context information necessary for defect location identification (S1042).

Then, the online system test method according to the present invention collects system state information necessary for performance analysis (S1043).

The process of collecting test information of the online system test method according to the present invention is terminated.

After collecting test information, the online system test method according to the present invention removes the test agent (S105).

Removing the test airagent means returning the hooked position to the original code for testing purposes and removing the test agent.

6 is a flowchart illustrating a specific process of removing a test agent in the online system test method according to the present invention.

Referring to FIG. 6, the online system test method according to the present invention unhooks test positions that have been hooked with a test code to an original code (S1051).

Next, the online system test method according to the present invention removes the test agent from the processes in which the test agent has been inserted (S1052).

The process of removing the test agent of the online system test method according to the present invention is terminated.

1 is a flowchart of an online system test method according to the present invention.

2 is a flowchart illustrating a specific process of defining a defect model in an online system test method according to the present invention.

3 is a flowchart illustrating a specific process of inserting a test agent in the on-line system test method according to the present invention.

4 is a flowchart illustrating a specific process of hooking a test position in an on-line system test method according to the present invention.

5 is a flowchart illustrating a specific process of collecting test information in an online system test method according to the present invention.

6 is a flowchart illustrating a specific process of removing a test agent in the online system test method according to the present invention.

Claims (12)

Defining a defect model; Inserting a test agent into a currently running process based on program context information of a current process space (CPS); Hooking a test location; Collecting test information; And Removing the test agent from the process. The method according to claim 1, Defining the defect model Defining the test location according to a defect type; Defining the test information according to the defect type; And Generating test code for collecting the test information. The method according to claim 2, Inserting the test agent Constructing a list for running processes; And Inserting the test agent into a currently running process based on the list. The method of claim 3, Hooking the test position Identifying a memory area according to the test location; Hooking the identified memory area with the test code. The method according to claim 4, Collecting the test information An online system test method comprising collecting program execution information, process context information, and system state information. The method according to claim 5, Removing the test agent from the process Recovering the test location; And Removing the test agent. Defining a defect model; Inserting a test agent into a currently running process based on program context information of a current process space (CPS); Identifying a memory area according to a test location; Hooking the identified memory region; Collecting test information; And Removing the test agent from the process. The method of claim 7, Defining the defect model Defining the test location according to a defect type; Defining the test information according to the defect type; And Generating test code for collecting the test information. The method according to claim 8, Hooking the identified memory area Hooking said identified memory region with said test code. The method according to claim 9, And hooking an APISet region of a current process space (CPS) when the test location is a SYSTEM API (Application Program Interface) function. The method according to claim 9, And if the test location is a DLL (Dynamic Link Library) EXPORT function, hooking an IAT (Import Address Table) region of the CPS. The method according to claim 9, Hooking a code space region of a CPS when the test location is not a SYSTEM API or a DLL EXPORT function.

Images