KR101513662B1 - Search system and method of executable GUI - Google Patents

Abstract

The present invention relates to a search system and method for graphic user interface (GUI) which is executable without sequence dependency. The invention provides a static analysis tool which analyzes basic information about GUI using source code or decompile technique without direct execution, and a dynamic analysis tool which executes each element in consecutive order according to the basic information analyzed in the static analysis; therefore, as a result of using static and dynamic analysis at the same time, erroneous results are not produced and automaticity is increased.

Description

An executable graphical user interface retrieval system and a retrieval method,

The present invention relates to a system and method for locating a graphical user interface (GUI) that can be executed dynamically and in a non-sequential manner. More particularly, the present invention relates to an executable A graphical user interface search system, and a search method.

Rapid technological advances in the IT arena require the development of software technologies that follow. As such, software is an important element in the computer and communications sectors. As a result, the reliability of the software is directly related to the reliability of the system operation, and quality control thereof is required. Software quality control is the most common example of software testing.

Software testing is divided into static testing and dynamic testing. Dynamic testing is a type of software testing in which the implemented software is directly executed to determine whether the result is equal to a desired value. Static testing is a type of software testing in which the software is not actually executed.

Static testing is typically performed by configuration inspection, source code inspection, compile-build processor, and the like.

A shape inspection checks whether a configuration value violates a constraint. When writing software, some variables may delay the input of correct values, and the shape indicates that the setting variables are determined to be concrete values. When writing a program, you can set some variables or constants to affect program compilation or program execution. The variable or constant value is called the shape of the program, and it is the shape inspection that checks whether the shape is as intended.

Source code inspection is an activity that checks the source code for errors. It checks that the signature of the function and method is written in the specification, or checks whether the program variables or header files to be compiled are properly present.

The compile-build process is to report and act on errors that occur during compilation.

In addition, the penetration rate of smartphones based on Android is rising, and the range of application of Android OS is widening in society as a whole, and various functions of Android applications are being launched one by one.

Android applications are structured differently from traditional Java applications and are optimized to work in a limited hardware environment, such as mobile devices. These Android basically provide three kinds of JUnit, Monkey, and MonkeyRunner for testing.

In addition to testing simple APIs, it also provides instrumentation based testing techniques. The function test is performed by expressing the contents of the user's behavior in code. Monkey can send a random event from Android to the target to perform a stress test that tests whether an error occurs or how long it takes. MonkeyRunner is a tool that allows you to create scripted code with Jython for action coordinates and action events.

Examples of such techniques are described in documents 1 and 2 below.

For example, as shown in Fig. 1, the following patent document 1 discloses a method for managing a plurality of configuration source files received from a basic software module 11 and an execution environment module 12 of a vehicle software system A test processor 22, a testing control unit 23, a Satisfaction Modulo Theory Solver (SMT) 24 and a code of satisfaction as a satisfactory module. The pre-processor 21, the test case storage unit 22, A software static testing apparatus 20 that includes an execution unit 25 and performs software static testing on a shape file among files constituting the basic software module 11 or the execution environment module 12 has been disclosed .

Patent Document 2 discloses a method for generating an activity-based state diagram of an Android application, which is a stage prior to creating a scenario for GUI testing of an Android application, wherein reverse engineering of the source of the application is performed, Wherein the step of adding the lifecycle of the activity further comprises the step of adding a lifecycle of the activity to the state diagram, the step of adding the lifecycle of the activity to the activity of each activity And a method for generating an activity-based state diagram of an Android application that includes checking whether a finish function exists for the application and adding an event back.

Korean Patent Publication No. 2012-0072133 (published on July 23, 2012) Korean Registered Patent No. 10-1416065 (Registered on July 1, 2014)

A large number of applications supplied through the app marketplace provide useful functionality, but often contain GUI bugs due to the inefficiency of the verification process. In particular, GUI bugs that cause crashes are very lethal because they cause the application to abruptly stop without saving. Although there are many mobile testing studies, existing studies are dependent on source code, and there is a limit to perform GUI testing for a wide range of applications in the app market, with efficiency and autonomy.

That is, in the conventional technique as described above, there is a problem in that it results in a lot of erroneous results because it is not a direct execution method because it statically analyzes the source code or grasps the relationship between graphic user interfaces through the decompile technique.

In addition, the above-described conventional techniques have been disadvantageous in that the automaticity is reduced and the efforts of many people are finally required.

It is an object of the present invention to provide an executable graphical user interface search system capable of automatically identifying dependencies between graphical user interface elements constituting software and finding executable functions independently, And a search method.

Another object of the present invention is to provide an executable graphical user interface retrieval system and retrieval method that can be used independently for executable graphical user interfaces to achieve high coverage.

In order to achieve the above object, an executable graphical user interface search system according to the present invention is a system for searching an executable graphical user interface in any order, And static analysis means for analyzing the basic information and dynamic analysis means for sequentially executing the elements in accordance with the basic information analyzed by the static analysis means.

According to another aspect of the present invention, there is provided an executable graphical user interface (GUI) search system, wherein the static analysis unit includes a basic information analyzer for reading graphical user interface basic information and a basic information storage unit for storing the basic information, A sequence analysis controller for generating and analyzing tasks for reading basic information from the basic information storage unit and sequentially displaying a graphical user interface, a task manager for taking charge of tasks assigned by the sequence analysis controller to generate an actual graphical user interface And a user interface element storage unit for storing the analyzed graphical user interface elements.

Further, in the executable graphical user interface search system according to the present invention, the dynamic analysis means may further include an execution result determination unit that determines whether information actually displayed on the electronic device is normally output by the graphic user interface indicator .

Further, in the executable graphical user interface search system according to the present invention, the basic information analyzer analyzes information on basic elements representing the graphical user interface from the software specification, the source code, and the executable code, And the basic information analyzing unit is connected to the analyzed graphical user interface basic information.

Also, in the executable graphical user interface search system according to the present invention, the order analysis controller is connected to the basic information storage unit to sequentially read the graphical user interface basic information to display information, The user is instructed to execute the display again using the user interface basic information, and if there is a problem, the job is forcibly terminated and the error is processed.

In the executable graphical user interface search system according to the present invention, the user interface element storage unit is connected to the execution result determination unit and stores analyzed graphical user interface elements.

In an executable graphical user interface search system according to the present invention, the basic information is at least one of a type, a size, a layout, and a color of graphic user interface elements of a button and a text box.

If the execution result determining unit determines that the graphical user interface element is normally displayed, the execution result determining unit determines that the graphical user interface element is executable regardless of the order, And notifies the order analyzing controller of a result that is not a normal display when the normal display is not performed.

In order to achieve the above object, an executable graphical user interface search method according to the present invention is a method for searching an executable graphical user interface in any order, including: (a) extracting a graphical user interface from a software specification, (B) storing the graphical user interface basic information analyzed in the step (a) in a basic information storage unit; (c) (B) generating a graphical user interface (GUI) by sequentially reading the stored basic information from the sequence analysis controller and analyzing the generated graphical user interface; (d) (E) repeating the step (d) Determining whether the information actually displayed on the electronic device is normally output by the execution result determination unit; (f) if it is determined to be normal in the step (e), transmitting the executable graphical user interface element to the user interface And storing it in the element storage unit.

If it is judged that the graphical user interface is not normal in the step (d), the executable graphical user interface searching method according to the present invention is forcibly terminated and the process proceeds to the step (c) And the analysis is performed.

In the graphical user interface searching method according to the present invention, if it is determined in step (e) that it is not determined to be normal, the sequence analyzing controller is notified, and the process proceeds to step (c) And performs analysis on other basic information.

As described above, according to the graphical user interface search system and the search method according to the present invention, both static and dynamic analysis are simultaneously used, so that an erroneous result is not generated and automaticness is enhanced.

That is, according to the present invention, a dynamic method is used to grasp order dependency between graphical user interfaces differently from existing methods. Therefore, when a dynamic method using direct execution is used, it does not produce a wrong result and requires human intervention It is possible to provide perfect automatism.

Further, according to the graphical user interface search system and the search method according to the present invention, graphical user interface elements that can be executed regardless of the analyzed order can be utilized in software testing to achieve higher coverage.

1 shows a configuration of a conventional software static testing apparatus,
Figure 2 shows an automated black box testing architecture applied to the present invention,
3 is a block diagram of an executable graphical user interface retrieval system according to the present invention;
4 is a flowchart illustrating an executable graphical user interface retrieval method according to the present invention.

These and other objects and novel features of the present invention will become more apparent from the description of the present specification and the accompanying drawings.

First, features according to the present invention will be described.

The present invention can be utilized in the field of software verification, and can be utilized in a black box verification technique to be verified without specific internal information, thus achieving high coverage. If an independent graphical user interface is found that does not depend on the order dependency of the graphical user interface, this information can be used as a starting point for each test. Higher coverage can be achieved by using a graphical user interface with many dependencies that is different from black box testing using an existing single-point graphical user interface.

In addition, existing techniques require human intervention. Therefore, we can not embrace the rapidly growing app market. Currently, there are more than one million registered applications in the app market. To validate all of these software, high coverage and automation support technologies are required. The present invention is well suited to the current mobile market because it meets both of these aspects.

Existing software testing is largely divided into white box and black box parts. The present invention can be actively used in the field of black box testing because it automatically analyzes an executable graphical user interface in any order. In addition, testing techniques are essential to ensure the reliability of software, and auto-testing is a very important technology when it comes to markets that require massive testing, such as the current App Store. Therefore, the present invention is highly economical in that it provides automability and can be utilized in black box testing.

Next, the black box testing structure used in the present invention will be described with reference to FIG. 2 is a diagram showing an automated black box testing structure applied to the present invention.

As shown in FIG. 2, the method proposed by the present invention is divided into a host portion and a target portion. The host part consists of installation file reversing, activity sequence analysis, event generation, and logger modules for automated testing. Each module was developed in Python. The target part represents the Android components used for automated testing. The corresponding Android components have been slightly modified for automated testing. We used the Nexus-one version of Cyanogen-mod-7 for modification.

The overall process of auto-testing is as follows. It extracts the activity information from the Android installation file (.apk) file and automatically grasps the order dependency of each activity. After that, GUI testing using random events is performed for each of the executable activities in any order, and the test results are recorded in real time. Finally, the test results are summarized. The detailed description and implementation details of each module are as follows.

Setup File Reversing Module: Obtains a manifest (Manifest.xml) file containing important internal information from the application's installation files using reverse packaging techniques. This file is parsed to determine the package name, version, activity, and permission information of the application, and installs the application on the target.

Activity Sequence Analysis Module: The basic unit that composes the Android GUI is activity, and the application starts from the main activity and goes to different kinds of activities according to the event. The activities of the application are registered and managed by the Activity Manager. In this module, the activity information of the application program and the activity manager are used to automatically analyze the executable activities regardless of the order, and the method is as follows. Create a new process to run the activity. The generated process gets one of the activities defined in the application that has not been analyzed. Then give the activity manager a command to run it. The activity manager normally responds to the host with the execution results in case of an activity that has no dependency. In this case, it is judged to be an executable activity regardless of the order, and the result is stored. However, it requires certain parameters such as database information, and if the activity is sequence-dependent, no results are returned and a GUI crash occurs. In this case, the created process that is responsible for executing the activity is terminated using the signal. Then use the Activity Manager to kill the application. If there are still activities to identify dependencies, the above process is repeated.

Event Generation Module: Unlike conventional Monkey testing start is always the main activity, the method of the present invention performs GUI testing through random event generation for executable activities irrespective of the order of analysis through the analysis. To do this, we modified the Monkey Server part of the Android platform so that we can test for the specified activity.

LOGGER MODULE: Store the results of all the tests in a file for each application. The test results consist of success and failure of installation and removal, activity coverage information, number of GUI crashes, and detailed error message information.

Hereinafter, the characteristic configuration of the present invention will be described with reference to FIG.

FIG. 3 is a block diagram of an executable graphical user interface retrieval system according to the present invention, wherein FIG. 3 illustrates a system for finding executable graphical user interface elements in any order.

As shown in Figure 3, an executable graphical user interface retrieval system according to the present invention is a system for finding a graphical user interface that can be executed in an orderless manner, A static analyzing means 100 for analyzing information and a dynamic analyzing means 200 for sequentially executing the respective elements according to the basic information analyzed by the static analyzing means 100.

The static analysis means 100 and the dynamic analysis means 200 may be executed by a PC, a notebook, a netbook, a PDA, a mobile phone, a smart phone, or a program having a processor and a memory.

The dynamic analyzing means 200 sequentially executes the respective elements using the analyzed basic information and analyzes the execution results to determine whether the corresponding graphical user interface can be executed in any order .

The static analysis unit 100 includes a basic information analyzer 110 that reads basic information of the graphic user interface and a basic information storage unit 120 that stores the basic information.

The basic information analyzer 110 is a basic information analyzer of a graphical user interface, and obtains information for an actual expression such as type, size, layout, color, and the like of elements. This information can be obtained from source code or from binary code. That is, the basic information analyzer 110 obtains basic information about the graphical user interface without directly executing it using the source code or the decompiling technique. Specifically, the type, size, arrangement, and color of each graphic user interface element such as a button, a text box, and the like are extracted and analyzed.

The basic information storage unit 120 stores information on the graphical user interface analyzed by the basic information analyzer 110, draws actual graphical interface elements using the stored information, and outputs the graphical user interface information to the dynamic analysis unit 200 .

The dynamic analysis unit 200 includes a sequence analysis controller 210 for generating a task for sequentially reading basic information from the basic information storage unit 120 and sequentially displaying a graphical user interface, A graphical user interface (GUI) indicator 220 for generating and displaying an actual graphical user interface in charge of a task assigned by the graphic user interface controller 220, and a display unit 220 for displaying whether the information actually displayed on the electronic device is normally output And a user interface element storage unit 240 for storing analyzed graphical user interface elements.

The sequence analysis controller 210 uses direct execution to analyze an executable graphical user interface in any order. At this time, it is responsible for controlling each execution. The sequence analysis controller 210 reads the basic information of the graphical user interface sequentially from the basic information storage unit 120 and sends the information to the graphical user interface indicator 220 which is responsible for displaying information, waiting. If it is normally performed according to the result of the graphical user interface indicator 220, the execution of the display is instructed again using the next graphical user interface basic information. If there is a problem in the graphical user interface indicator 220, the corresponding operation is forcibly terminated Have the error handled. According to the present invention, the analysis can be continued without intervention of the operator stably through such processing.

The graphical user interface indicator 220 performs direct representation through execution using basic information. At this time, if the corresponding graphical user interface is dependent on specific dictionary data, an error occurs during display. The graphical user interface indicator 220 functions to display a general graphical user interface and does not perform any analysis such as dictionary data dependency, but merely displays the graphical user interface using the basic information.

The execution result determination unit 230 determines whether the graphical user interface displayed by the graphical user interface indicator 220 is normally displayed. That is, the execution result determination unit 230 determines whether the graphical user interface is displayed normally, using an error message, call stack information, If it is normally displayed, the corresponding graphical user interface element is determined to be executable regardless of the order and sent to the user interface element storage unit 240, otherwise, the result is notified to the order analysis controller 210.

The user interface element storage unit 240 finally stores executable graphical user interface elements regardless of the order, and this information is utilized in various software testing techniques.

Next, an executable graphical user interface retrieval method of the present invention will be described with reference to FIG.

4 is a flowchart illustrating an executable graphical user interface searching method according to the present invention.

An executable graphical user interface search according to the present invention is a method for finding a graphical user interface that can be executed in an orderly manner. First, information on basic elements representing a graphical user interface from a software specification, source code, ) (S10). By such an analysis, information for actual expression such as kind, size, arrangement, color, etc. of elements can be obtained.

The graphic user interface basic information analyzed in step S10 is stored in the basic information storage unit 120 (S20) and transmitted to the sequence analysis controller 210. [

 The basic information stored in step S20 is read by the order analysis controller 210, and a job of sequentially displaying a graphical user interface is created and analyzed (S30), and it is determined whether the user is normal or not (S40).

If it is determined in step S40 that the operation is normal, the sequence analyzing controller 210 takes charge of the tasks assigned to the sequence analyzing controller 210 to generate an actual graphical user interface and display the graphical user interface on the graphical user interface indicator 220 in operation S50.

On the other hand, if it is determined in step S40 that it is not normal, the process is forcibly terminated (S60), and the process proceeds to step S30 where the analysis is continuously performed on the other basic information stored in step S20.

Next, in step S70, the execution result determination unit 230 determines whether the information actually displayed on the electronic device is normally output in step S50.

If it is determined in step S70 that it is normal, the executable graphical user interface element is stored in the UI element storage unit 240 regardless of the order (S80), and the above-described process is repeated by proceeding to step S30.

On the other hand, if it is not determined to be normal in the step S70, the sequence analysis controller 210 is notified, and the process proceeds to the step S30 and continues to analyze the other basic information stored in the step S20.

As described above, the experimental results of the executable graphical user interface searching method according to the present invention are as follows.

In the present invention, two experiments performed to verify the automaticity and the efficiency will be described. Both experiments compared the results obtained using the proposed method and the existing Monkey. The first is an experiment that measures code coverage for the open source application WordPress and shows the detection rate of known GUI bugs. The second is an experiment that measures activity coverage and detects GUI crashes for the 20 selected app markets.

First, we measured the coverage of open source applications.

In other words, in order to verify the efficiency of the method of the present invention, an experiment was conducted to detect code coverage and known bugs. We used WordPress, an open source application for code coverage measurements, and experimented on the Nexus One. WordPress is an application to support blogging activities, so you can check the change history for 4 years. The version you chose is Git commit hash 66846425. I linked two accounts to WordPress and added several posts to each blog, including multimedia data. To measure code coverage, we also used the Android Instrumentation Class, the Monkey Library, and Emma (EMMA).

The testing scenario for the experiment is as follows. We used the implemented tools to test for an independent activity that automatically analyzed 2,000 random events. There were 14 independent activities and a total of 28,000 random events were tested. For testing with the same conditions, we tested 14 times starting with the main activity using 2,000 random events in the case of traditional Monkey testing.

As a result of the experiment using the two methods, the method according to the present invention achieved syntax and function coverage of 32% and 34%, respectively, and activity coverage achieved 59.2%. These results are 15% higher in syntax coverage than when tested using Monkey under the same conditions. It was also higher than the 25.27% syntactic coverage achieved when testing with 45,000 random events over 4.46 hours using conventional Monkey.

Next, we measured the coverage of market applications.

Experiments were conducted to demonstrate the effectiveness of the present invention for real market applications. The 20 applications used in the experiment have different categories, ratings and cumulative downloads. Because the experiment was carried out with only the installation files of the application, we only got the results at the black box level. First, the results of the activity coverage test showed that the average activity coverage of the method according to the present invention was 64% and that of the conventional Monkey was 22%. For the PerfectViewer application, 100% activity coverage was achieved with the method of the present invention. In addition, some applications have achieved activity coverage three times higher than existing Monkeys. As a result of analyzing the crashes for 20 application programs, the method of the present invention was able to detect crashes in 9 application programs. On the other hand, in the case of the existing Monkey, only four application programs were found to have a crash. In addition, we detected a maximum of four times more crashes for each application. These results show that the present invention is also excellent in terms of detecting a crash.

Although the present invention has been described in detail with reference to the above embodiments, it is needless to say that the present invention is not limited to the above-described embodiments, and various modifications may be made without departing from the spirit of the present invention.

The present invention can provide a high coverage of code coverage and detection of GUI bugs by using an executable graphical user interface search system and a search method according to the present invention. Therefore, it is possible to provide an application program including a GUI bug in the Android Market You will be able to filter out them.

100: static analysis means
110: Basic information analyzer
120: Basic information storage unit
200: Dynamic analysis means
210: Sequence Analysis Controller
220: Graphical user interface indicator
230: execution result judgment unit
240: User interface element storage unit

Claims (11)

A system for finding an executable graphical user interface in any order,
Static analysis means for analyzing basic information about the graphical user interface without direct execution using source code or decompile technology;
And dynamic analysis means for sequentially executing the respective elements according to the basic information analyzed by the static analysis means,
Wherein the dynamic analyzing means comprises: a sequence analyzing controller for generating a task for reading basic information from the basic information storing portion and sequentially displaying a graphical user interface, and analyzing the generated task; A graphical user interface indicator for generating and displaying a user interface, and a user interface element storage for storing analyzed graphical user interface elements. The method according to claim 1,
Wherein the static analysis means includes a basic information analyzer for reading the basic information of the graphic user interface and a basic information storage for storing the basic information. 3. The method of claim 2,
Wherein the dynamic analysis unit further comprises an execution result determination unit that determines whether information actually displayed on the electronic device is normally output by the graphic user interface indicator. 3. The method of claim 2,
The basic information analyzer analyzes information on basic elements representing a graphical user interface from a software specification, a source code, and an execution code,
Wherein the basic information storage unit is connected to the basic information analysis unit and stores the analyzed basic information of the graphical user interface. 3. The method of claim 2,
The sequence analyzing controller is connected to the basic information storage unit to sequentially read the graphical user interface basic information to display information, and when executed normally commands the display execution again using the next graphical user interface basic information, If it exists, forcibly terminates the job and processes the error. The method of claim 3,
Wherein the user interface element storage unit is connected to the execution result determination unit and stores the analyzed graphical user interface element. The method of claim 3,
Wherein the basic information is at least one of a type, a size, a layout, and a color of graphic user interface elements of a button and a text box. The method of claim 3,
The execution result determination unit determines that the graphical user interface element is executable regardless of the order if it is normally displayed in the graphical user interface indicator, and sends the graphical user interface element to the user interface element storage unit. If the normal graphical user interface element is not displayed normally, And notifies a result that is not a display. A method for locating an executable graphical user interface in any order,
(a) analyzing in a basic information analyzer information about basic elements representing a graphical user interface from a software specification, a source code and an execution code,
(b) storing the graphical user interface basic information analyzed in the step (a) in a basic information storage unit,
(c) generating and analyzing a task of sequentially reading the basic information stored in the step (b) from the sequence analysis controller and sequentially displaying a graphical user interface,
(d) generating and displaying an actual graphical user interface by taking charge of the tasks assigned by the order analysis controller,
(e) determining, by the execution result determination unit, whether the information actually displayed on the electronic device is normally output by the step (d)
(f) storing an executable graphical user interface element in the user interface element storage, regardless of the order, if it is determined to be normal in the step (e). 10. The method of claim 9,
And if it is determined in step (d) that it is not normal, execution is terminated and analysis is performed on the other basic information stored in step (b) by proceeding to step (c) Way. 10. The method of claim 9,
And notifies the order analysis controller of the fact that it is not normal in the step (e), and proceeds to the step (c) and analyzes the other basic information stored in the step (b) Graphical user interface search method.

Images