KR20190095717A - Color difference based method for gui test automation and apparatus using the same - Google Patents

Abstract

Disclosed is a GUI test automation method based on a color difference comprising: an application execution module executing a GUI test target application; a color difference reading module reading a color difference displayed in a GUI menu item of the executed application to detect an execution region of the GUI menu item; an input device control module using a reading result of the color difference reading module to control an input device so as to test the GUI menu item; and an analysis module analyzing a handle value assigned to the GUI menu item through the test. According to the present invention, the present invention uses the color difference displayed in the menu to quickly find a menu and execute the corresponding menu so as to dramatically reduce a range of controlling the input device and a test time.

Description

Color difference-based GUI test automation method and device using the same {COLOR DIFFERENCE BASED METHOD FOR GUI TEST AUTOMATION AND APPARATUS USING THE SAME}

The present invention relates to a method for automating color difference-based GUI testing and an apparatus using the same, and more particularly, by checking a function through menu execution of an application, and automatically testing whether the application works properly according to a designed user interface. A method of analyzing results and an apparatus using the same.

Recently developed software, applications or programs are being developed to operate on various platforms and various operating system environments, and also includes various functions according to the convergence of various technologies compared to the past.

These applications are tested during the production stage to ensure that the functionality of the GUI works as designed. That is, a test is performed to determine whether the application performs the expected function on the input through the GUI and outputs reasonable results on various platforms and operating system environments, or whether the device running the application is not stopped due to an infinite loop of errors. do.

However, developers often have to manually perform checks and menu analysis for these tests. This is because errors in the actual environment can be detected by matching the test environment to the environment in which the user uses the application, and performing the test in the same manner as the user uses the application. And in order to establish the reliability of error detection, the test by the same method as that performed by the user in the same use environment as the user's use environment should be repeatedly performed.

In the manual test process, the developer directly plans the check range and sets the test operation by programming to be identical to the environment and the method in which the general user uses the application.

Therefore, in order to proceed with the test, the user must set up and use all the processes of using the application. As the test items are added, the process for preparing the test is complicated, and the preparation time is greatly increased.

Korean Patent Laid-Open Publication No. 10-2014-0004495 (published Jan. 13, 2014) "Automatic test method of an application and its electronic device"

SUMMARY OF THE INVENTION An object of the present invention is to provide a GUI test automation method and an apparatus using the same that can dramatically reduce the scope of controlling an input device by quickly finding a menu using a color difference displayed on a menu and executing the menu. .

The present invention further solves the problem by finding the execution region of the menu by using the color difference displayed on the menu, and simultaneously executing a plurality of tests by using a plurality of input devices in executing the menu, and analyzing the results in combination. To provide a GUI test automation method and an apparatus using the same.

GUI test automation apparatus according to an embodiment of the present invention, the application execution module for executing the GUI test target application; A color difference reading module configured to read a color difference displayed on a GUI menu item of an executed application and detect an execution region of the GUI menu item; An input device control module for testing a GUI menu item by controlling the input device using the reading result of the color difference reading module; And an analysis module for analyzing a handle value assigned to the GUI menu item through the test.

The application execution module may execute the GUI test target application in a local area or an external remote area of a GUI test automation device.

Here, the application execution module, characterized in that for executing each of the GUI test target application in the plurality of external remote areas.

Here, the application execution module is characterized in that for running the respective GUI test target application in a plurality of remote areas of the outside of various platforms composed of heterogeneous operating systems and heterogeneous hardware.

The input device control module may control the input device to move the pointer of the input device to the execution region of the GUI menu item and to execute the menu.

Here, the color difference reading module is characterized by dividing the GUI menu area into a plurality, and detecting the executable menu area and the GUI change for each divided area.

Here, the automation device further includes a storage device for storing default values assigned to elements constituting the application under test during GUI design, and the analysis module obtains a handle value through a GUI test and obtains the handle value. The error in the GUI design is detected by comparing the value with the assigned default value.

Here, the default value may be a handle value assigned to a GUI top menu and a position, a shape of a menu branched from the menu, and a function called by the menu.

Color difference-based GUI test automation method according to an embodiment of the present invention, executing the GUI test target application; Detecting the execution region of the GUI menu item by reading a color difference displayed on the GUI menu item of the executed application; Testing the GUI menu item by controlling the input device using the reading result of the color difference reading module; And analyzing the handle value assigned to the GUI test item through the test.

The step of executing the GUI test target application may include executing the GUI test target application in a local area or an external remote area of the GUI test automation device.

Here, the step of executing the GUI test target application, characterized in that for running each GUI test target application in the plurality of external remote areas.

Here, the executing of the test target application is characterized in that each GUI test target application is executed in a plurality of remote areas outside of various platforms including heterogeneous operating systems and heterogeneous hardware.

The testing of the GUI menu item may include moving the pointer of the input device to an execution region of the GUI menu item and controlling the input device to execute the menu.

The detecting of the execution region of the GUI menu item may include dividing the GUI menu region into a plurality, and detecting an executable menu region and a GUI change for each divided region.

The GUI test automation method may further include storing a default value assigned to an element constituting a test target application during GUI design, and analyzing the handle value may include obtaining a handle value through a GUI test. In addition, an error in GUI design is detected by comparing the acquired handle value with the assigned default value.

Here, the default value may be a handle value assigned to a GUI top menu and a position, a shape of a menu branched from the menu, and a function called by the menu.

According to the present invention, by quickly finding a menu using the color difference displayed on the menu and executing the menu, the range and test time for controlling the input device can be drastically reduced.

Further, the test range can be extended by finding the execution region of the menu using the color difference displayed on the menu, and simultaneously performing a plurality of tests using a plurality of input devices in executing the menu.

1 is a schematic block diagram of a GUI test automation apparatus according to an embodiment of the present invention.
2 is an exemplary diagram of a system environment including a GUI test automation apparatus according to an embodiment of the present invention.
3 is a detailed illustration of FIG. 2.
4 is a block diagram of a GUI test automation apparatus according to an embodiment of the present invention.
FIG. 5 is a detailed block diagram of the controller of FIG. 4.
6 is a block diagram of a computing device corresponding to a target application driving apparatus according to an exemplary embodiment.
7 is a flowchart illustrating a GUI test automation method according to an embodiment of the present invention.
8 is an exemplary diagram for describing a color difference according to an embodiment of the present invention.

Hereinafter, exemplary embodiments of a GUI test automation apparatus and method will be described in detail with reference to the accompanying drawings.

Like reference numerals in the drawings denote like elements. In addition, specific structural to functional descriptions of one embodiment of the present invention are only illustrated for the purpose of describing the embodiment according to the present invention, and unless otherwise defined, all terms used herein including technical or scientific terms are used. The terms have the same meaning as commonly understood by one of ordinary skill in the art. Terms such as those defined in the commonly used dictionaries should be construed as having meanings consistent with the meanings in the context of the related art, and are not construed in ideal or excessively formal meanings unless expressly defined herein. It is preferable not to.

1 is a schematic block diagram of a GUI test automation apparatus according to an embodiment of the present invention.

Referring to FIG. 1, the GUI test automation apparatus 100 according to an embodiment of the present disclosure may store the operating system 176, the test application module 175, and the target application module 179 in a stand-alone form with a memory 177. Can be mounted on

In addition, the GUI test automation device 100 may test the target application module 275 that is run on a plurality of computing devices 200 through a network.

2 is an exemplary diagram of a system environment including a GUI test automation apparatus according to an embodiment of the present invention.

Referring to FIG. 2, the system environment 1 according to an embodiment of the present invention includes a GUI test automation apparatus 100, a plurality of target application driving apparatuses 200, a database 300, and a network 400. . 2 is shown in FIG. 3 in detail.

3 is a detailed illustration of FIG. 2.

Referring to FIG. 3, the GUI test automation apparatus 100 includes an operating system 176 and a test application module 175 in a memory 177, and the target application driving apparatus 200 may include an operating system 276 and a target application module. 275 is mounted in the memory 277.

Network 400 may be a wired and wireless network, such as the Internet, intranet and extranet, cellular, such as a wireless telephone network, a local area network, a wide area network. ), A WiFi network, an ad hoc network, and a combination thereof.

The database 300 may include a database of reference images used to read GUI changes based on reference images for color difference readings and test results.

Network 400 may include a connection of network elements such as hubs, bridges, routers, switches, and gateways. Network 400 may include one or more connected networks, such as a multi-network environment, including a public network such as the Internet and a private network such as a secure corporate private network. Access to network 400 may be provided through one or more wired or wireless access networks. Hereinafter, the GUI test automation apparatus 100 will be described according to the configuration shown in FIG. 3.

4 is a block diagram of a GUI test automation apparatus according to an embodiment of the present invention.

Referring to FIG. 4, the GUI test automation apparatus 100 includes an input device 110, a display device 120, an output device 130, a storage device 140, a communication device 150, a power supply device 160, and It includes a control unit 170. The controller 170 includes a test application module 175. The communication device 150 includes a transmitter 151 and a receiver 152.

The input device 110 includes, but is not limited to, a keyboard, a touch screen, a mouse, an electronic pen, and a voice input device, such as a microphone, for performing input required to drive and manipulate the test application module to test the target application module. It is not limited.

The display device 120 includes a cathode-ray tube (CRT), a liquid crystal display (LCD), a plasma display panel (PDP), which performs a frame output of a screen driving the test application module. Light emitting diode (LED) displays, and organic light emitting diode (OLED) displays include, but are not limited to these.

The output device 130 includes, but is not limited to, a printer, a plotter, a beam projector, a text to sound (TTS) device, a speaker, and an earphone.

The storage device 140 may be any type of removable / non-removable device from the GUI test automation device 100. The storage device 140 may be provided in a magnetic manner such as a magnetic tape, a magnetic drum, a floppy disk, a ZIP drive, and a hard disk drive (HDD). Electrical storage and CD-ROM drives, DVD-ROM drives, blu-ray discs using flash memory such as storage devices, SD cards, USB memory and solid state drives (SSDs) Optical storage devices, such as drives, include, but are not limited to.

The communication device 150 includes, but is not limited to, a communication module corresponding to various communication networks of the network 400, for example, a Bluetooth module, a WiFi module, an Ethernet interface card, a USB module, a cellular wireless communication module, a modem, and a wireless router. It doesn't happen. The communication device 150 includes a transmitter 151 and a receiver 152 therein.

The power supply unit 160 includes, but is not limited to, power supplies, adapters for converting AC to DC, secondary batteries such as SMPS and lithium ion batteries, lead acid batteries, fuel cells, and bioethanol batteries, and other primary batteries. It is not.

The controller 170 includes a memory area in which the processor, the operating system, and the test application module 175 are loaded. The controller 170 executes a command set included in the test application module according to a user selection or setting. Hereinafter, the control unit 170 will be described in detail.

FIG. 5 is a detailed block diagram of the controller of FIG. 4.

Referring to FIG. 5, the control unit 170 includes a processor (central processing unit, CPU) 178, an operating system 176, and a test application module 175 related to a GUI test automation method according to an embodiment of the present disclosure. Memory 177 loaded. In detail, the control unit 170 includes an application execution module 171, a color difference reading module 172, and an input including a computer instruction set for performing a GUI test automation method according to an embodiment of the present disclosure. A device control module 173 and an analysis module 174.

The controller 170 controls each component and modules in the GUI test automation apparatus 100 to implement the GUI test automation method according to an embodiment of the present invention.

The operating system 175 is a computer operating program for managing the file system, various interfaces, and controlling the operation of the components of the GUI test automation apparatus 100. Operating system 175 includes, but is not limited to, operating systems of wired and mobile devices, for example, MS Windows, Unix, Linux, XOS, Android, iOS operating systems.

Test application module 174 is an executable program configured to run on operating system 176 to provide various functions as described herein. The test application module 175 includes an application execution module 171, a color difference reading module 172, an input device control module 173, and an analysis module 174. Detailed descriptions of the various modules 171 to 174 included in the test application module 175 will be described later.

In the GUI test automation device 100 illustrated in FIGS. 4 and 5, the configuration in the test application module 175 is a functionally divided configuration for convenience of description, and is logically handled by one processor in hardware. It can be configured as a phosphorus function, it does not limit the invention by the presented division.

In addition, although the connection relationship between each configuration is shown as a connection line, or the connection relationship between each other is not expressed individually, communication or transmission for control or data exchange may occur even between the configuration where the connection line is not shown, and only as shown. It does not limit the invention.

The target application driving apparatus 200 according to an exemplary embodiment may correspond to the computing device 500. Hereinafter, the target application driving apparatus 200 according to an embodiment of the present invention corresponding to the computing device 500 may be described in more detail through the embodiment.

The computing device 500 may include a computing system consisting of one or more central processing units (CPUs), memory, mass storage, an input interface device, and an output interface device. The elements of the computing system can communicate with each other via a bus.

The hardware platform of the computing device may be a distributed computing environment, such as a cloud based computing system, including any of a personal computer, handheld or laptop device, a multiprocessor system, a microprocessor based system, a program appliance, and any of the above systems or devices. It can be implemented in many forms, including.

6 is a block diagram of a computing device corresponding to a target application driving apparatus according to an exemplary embodiment.

Referring to FIG. 6, the computing device 500 includes an input interface device 510, an output interface device 520, a memory 531, a storage device 532, a power supply 540, a processor 550, and a network interface. Device 560, wireless communication device 570, and bus 580.

The input interface device 510 inputs a character or an object necessary for creating a document according to a user's input. The input interface device 510 includes, but is not limited to, a keyboard, a touch screen, a mouse, a stylus pen, and a pen tablet.

The output interface device 520 includes a display for displaying a user interface related to a document editing application module and the like, and a printer for printing and outputting a document. In addition, the output interface device 520 includes a speaker, a head-phone, and a headset for outputting text in a document as a speech using a text to speech (TTS) engine. do.

The processor 550 is a computer instruction set included in a document editing application / server module 174/274 relating to a data backup method according to an embodiment of the present invention stored in a memory 531 and / or a storage device 532. You can run The processor 550 may refer to a central processing unit (CPU), a graphics processing unit (GPU), or a dedicated processor on which the methods according to the present invention are performed. The memory 531 and the storage device 532 may be configured of a volatile storage medium and / or a nonvolatile storage medium. For example, the memory 531 may be configured as read only memory (ROM) and / or random access memory (RAM).

The wireless communication device 570 includes a device for short range wireless communication, wireless data communication, and wireless voice communication.

Each component included in the computing device 500 is connected by a bus 580 to communicate with each other.

4, 5, and 5, for a GUI test automation method according to an exemplary embodiment of the present invention executed by the above-described GUI test automation device 100 through calculation of a computer instruction set included in a test application module 175. This will be described with reference to 7.

7 is a flowchart illustrating a GUI test automation method according to an embodiment of the present invention.

Referring to FIG. 7, the storage device 140 stores default values allocated to elements constituting a target application (S110).

The default value corresponds to the handle value assigned in the GUI design to the position, type, and function of the top-level menu and the menu branching from it. In this case, the handle value is a number given to a specific object and grammatically an integer value of 32 bits. In DOS programming, almost exclusively file handles were used, so in DOS, handles often meant file handles. But there are many different types of handles in Windows. The created window is attached with a window handle (hWnd) to manage the window by number.

When the menu is selected using the input device 110, the operating system generates a message and sends the ID of the menu item to the window where the menu is selected along with the message. In this case, the analysis module 174 may obtain an ID by using a function. The ID of a menu item is also a handle value. Also, the analysis module 174 may obtain a handle value assigned to the corresponding window using an API function.

Next, the application execution module 171 executes the test target application module (S120). Specifically, the application execution module 171 may be executed in the same manner as a general user actually executes the application to be tested, for example, after double-clicking a mouse or selecting a mouse, and then entering an executable file name through a keyboard enter key or a command input window. The application under test can be executed by the enter key input method.

Next, the display device 120 outputs a GUI screen of the target application module (S130). The GUI screen includes the appearance of the GUI menu to be tested in the present invention.

Next, the color difference reading module 172 detects an execution region of the menu item by reading the color difference representing the GUI menu item (S140). In this case, the color difference reading module 172 may read the color difference using an image reading method.

That is, the color difference reading module 172 senses an execution region of the menu by reading an image of the color difference displayed on the GUI menu displayed on the main window on which the application is executed. Various methods can be used for image reading. Among them, for example, a boundary reading method may already be used for reading. That is, the color difference reading module 172 may detect the menu item by reading a boundary line due to the color difference due to the color distinguished from the surrounding color displayed on the menu item.

The color difference reading performed in an embodiment of the present invention is an operation of analyzing a pattern that changes from one color to another color, rather than an elaborate operation of analyzing the color difference of each pixel. Therefore, black and white GUI images or binarized images can be used to quickly and simplify the color difference analysis.

8 is an exemplary diagram for describing a color difference according to an embodiment of the present invention.

Referring to FIG. 8, the color difference indicates that the background color of the menu name is distinguished from the surrounding color in order to easily identify the menu area.

The input device control module 173 tests the menu item by controlling the input device by using the color difference reading (S150). In particular, the color difference reading method can be effectively used to locate the top menu. In addition to the method of reading color differences, image contrast may be used to determine the name of a menu. In other words, in particular, the mouse may be used alone as the input device 110 in determining the position of the best menu.

One way to quickly find and test menu items is to use split screen. That is, the menu area is divided into a plurality of areas, and the mouse pointer is moved for each area to detect a change in the GUI.

Here, the change of the GUI includes highlighting when a menu item is selected, and includes opening a lower menu window. In other words, GUI change means color change and dynamic change.

Thus, GUI changes can be detected by image contrast. For example, the GUI change may be detected by comparing the color of the menu before executing the menu with the color of the menu when the menu is selected using the image stored in the database 300.

Originally, a pointer of a mouse moving by a user's hand movement and a key of a keyboard pressed, but in an embodiment of the present invention, the input device control module 173 moves a pointer of a mouse, performs a click command, and additionally presses a key of the keyboard. A test is performed (S151). If an executable region of the menu is found based on the color difference, the menu execution may use a mouse and / or a keyboard.

Next, the analysis module 174 analyzes the handle value assigned to the test item through the test (S160). In detail, the analysis module 174 analyzes whether there is a design error by comparing a handle value obtained in the test process with a default value assigned in the GUI design.

 In addition, the analysis module 174 sets a default value for the execution screen of the test program to be executed, divides it into a plurality of blocks, grasps the change of each block, and changes the existence of the menu through the change of each block. Analyze such things as the presence of the function When such an analysis is performed, the analysis module 174 repeatedly executes each function in a predetermined form to test the execution of the application, and generates a report that records errors such as execution and normal operation according to the execution.

The analysis module 174 stores the reference value and the report in the storage device 140.

The storage device 140 stores the reference value generated by the analysis module 174 and provides reference values according to the request of the analysis module 174. In addition, the storage device 140 stores a report generated as the test proceeds from the analysis module 174. Such a report may be provided to another apparatus, such as display device 120, upon the user's request.

As described above, according to an exemplary embodiment of the present disclosure, the range and test time for controlling the input device can be drastically reduced by quickly finding a menu using a color difference displayed on the menu and executing the corresponding menu.

Further, the test range can be extended by finding the execution region of the menu using the color difference displayed on the menu, and simultaneously performing a plurality of tests using a plurality of input devices in executing the menu.

The foregoing description of the present invention is intended for illustration, and it will be understood by those skilled in the art that the present invention may be easily modified in other specific forms without changing the technical spirit or essential features of the present invention. will be. Therefore, it should be understood that the embodiments described above are exemplary in all respects and not restrictive. For example, each component described as a single type may be implemented in a distributed manner, and similarly, components described as distributed may be implemented in a combined form.

The scope of the present invention is shown by the following claims rather than the above description, and all changes or modifications derived from the meaning and scope of the claims and their equivalents should be construed as being included in the scope of the present invention. do.

100: GUI test automation device
110: input device
120: display device
140: storage device
171: program execution module
172: color difference reading module
173: input device control module
174: analysis module
200: target application driving device

Claims (16)

An application execution module for executing a GUI test target application;
A color difference reading module configured to read a color difference displayed on a GUI menu item of an executed application and detect an execution region of the GUI menu item;
An input device control module for testing a GUI menu item by controlling the input device using the reading result of the color difference reading module; And
And an analysis module for analyzing a handle value assigned to a GUI menu item through the test. The method according to claim 1,
The application execution module,
And executing the GUI test target application in a local area or an external remote area within a GUI test automation device. The method according to claim 1,
The application execution module,
A color difference based GUI test automation device, characterized in that each GUI test target application is executed in a plurality of external remote areas. The method according to claim 1,
The application execution module,
A color difference-based GUI test automation device, characterized in that each GUI test target application is executed in a plurality of remote areas outside of various platforms composed of heterogeneous operating systems and heterogeneous hardware. The method according to claim 1,
The input device control module,
And moving the pointer of the input device to the execution region of the GUI menu item and controlling the input device to execute the menu. The method according to claim 1,
The color difference reading module,
Color-based GUI test automation device, characterized in that for separating the GUI menu area into a plurality, and detecting the executable menu area and the GUI change for each divided area. The method according to claim 1,
The automation device,
Further comprising a storage device for storing the default values assigned to the elements constituting the application under test in the GUI design,
The analysis module,
And obtaining a handle value through a GUI test and detecting an error in the GUI design by comparing the obtained handle value with the assigned default value. The method according to claim 7,
The default value is
Color difference-based GUI test automation device, characterized in that the handle value assigned to the position of the GUI top menu and the menu branching therefrom, the type and function called by the menu at the time of GUI design. Executing a GUI test target application;
Detecting the execution region of the GUI menu item by reading a color difference displayed on the GUI menu item of the executed application;
Testing the GUI menu item by controlling the input device using the reading result of the color difference reading module; And
And analyzing the handle value assigned to the GUI test item through the test. The method according to claim 9,
Running the GUI test target application,
And executing the GUI test target application in a local area or an external remote area of a GUI test automation device. The method according to claim 9,
Running the GUI test target application,
Color-based GUI test automation method, characterized in that for running each GUI test target application in a plurality of external remote areas. The method according to claim 9,
Running the test target application,
A color difference based GUI test automation method, characterized in that each GUI test target application is executed in a plurality of remote areas outside of various platforms composed of heterogeneous operating systems and heterogeneous hardware. The method according to claim 9,
Testing the GUI menu item,
And moving the pointer of the input device to the execution region of the GUI menu item and controlling the input device to execute the menu. The method according to claim 9,
Detecting an execution region of the GUI menu item,
Color-based GUI test automation method, characterized in that for distinguishing the GUI menu area into a plurality, and detecting the executable menu area and the GUI change for each divided area. The method according to claim 9,
The GUI test automation method,
Storing the default values assigned to the elements constituting the application under test during GUI design,
Analyzing the handle value,
And obtaining a handle value through a GUI test and detecting an error in the GUI design by comparing the acquired handle value with the assigned default value. The method according to claim 15,
The default value is
A color difference-based GUI test automation method, characterized in that the handle value assigned to the position of the GUI top menu and the menu branched from the GUI, and the function called by the menu.

Images