KR101399061B1 - Apparatus and method for determining screen equivalency to test compatibility - Google Patents

Abstract

The present invention performs the same application in smart devices such as smartphones and tablets of different kinds and compares the execution results with an equivalence determination algorithm based on a human visual system (HVS) The present invention provides an apparatus and method for determining screen equivalence that can be easily implemented and can evaluate compatibility at a high speed, and is provided with an application execution unit that executes an application based on a given test scenario in a connected device, A capturing unit for capturing and storing an execution result of the application from the device that is captured by the capturing unit; a matching processing unit for removing or modifying a part (an area other than the area of interest) The two results removed or modified by the processing unit are displayed on a screen based on human visual system (HVS) And an equivalence comparison unit for evaluating the equivalence using an equivalence determination algorithm.

Description

TECHNICAL FIELD [0001] The present invention relates to a screen equivalence determining apparatus and method for evaluating compatibility,

The present invention relates to a framework for determining screen equivalence for evaluating compatibility, and more particularly to a framework for automatically determining a screen equivalence for evaluating compatibility by executing the same app in a smart device such as a smart phone or a tablet, The present invention relates to an equivalence discrimination apparatus and method for evaluating compatibility by comparing execution results of an application through a screen equivalence discrimination algorithm based on HVS.

The global smartphone market is rapidly evolving and smartphone manufacturers and platform makers are launching new products in a very short period of time to quickly accommodate the diverse needs of users. The launch of such a product seems to meet the needs of the user in a timely manner, but it creates compatibility issues for smartphone app developers. Compatibility issues are different when you run the same app on different devices and platforms. For example, the behavior of the Twitter app in version 2.1 should be the same as in version 2.2 of the platform. And the YouTube app should work the same on HTC's Nexus One as well as Samsung's Galaxy S. In particular, this issue is serious in the case of Android with a platform open, also known as "fragmentation".

These smartphone app compatibility issues are expected to become more severe in the future, and even professional developers are working hard to address compatibility issues with their apps. As a result, untrained developers and organizations developing apps are expected to face even more difficulties.

On the other hand, GUI-based testing techniques can be considered to check the compatibility of the app. That is, the user can determine whether the application is compatible with the corresponding smart phones by executing the same application on various smart phones and checking whether the execution result screen is equivalent. However, much effort is required to determine the equivalence of the execution result screen of the application to a large number of smartphones. As the number of smartphones that need to run the same app increases as the fragmentation becomes more and more complicated according to the type of device and the version upgrade of the platform, the cost for compatibility testing will increase exponentially.

Especially, it is very difficult to judge whether the result of the execution of the application is the same on both smartphones. This is because the distortion of the screen image occurs because the GPU and the pixel format used by the manufacturer are different from each other, and the same UI elements may be displayed as different images for different platforms even though the same operation of the functions is the same.

SUMMARY OF THE INVENTION Accordingly, the present invention has been made to solve the above-mentioned problems, and it is an object of the present invention to provide an equivalence determination algorithm based on a human visual system (HVS), which executes the same application in smart devices such as smartphones and tablets of different kinds, The present invention is directed to an apparatus and method for discriminating equivalence of screen equivalents, which can easily evaluate the compatibility of the application by comparing the execution results of the app through the Internet.

It is another object of the present invention to provide a method and apparatus for automatically executing a given arbitrary app on a plurality of smartphones, capturing the result of the app, and performing SIMPLE and VDP techniques using a screen equivalence determination algorithm based on a human visual system (HVS) The present invention also provides an apparatus and method for performing screen equivalence evaluation by discriminating screen equivalence while being less influenced by distortion of a digital image and less affected by minute differences between two images.

According to another aspect of the present invention, there is provided a screen equivalence checking apparatus for evaluating a compatibility of an apparatus, comprising: an application executing unit for executing an application based on a given test scenario in a connected device; A matching processing unit for removing or modifying a part of the execution result of the application captured by the capturing unit that is unnecessary for the experiment (an area other than the area of interest); and a matching unit And an equivalence comparator for evaluating equivalence using two equivalents of the eliminated or modified results using a human visual system (HVS) -based screen equivalence determination algorithm.

Preferably, the device is characterized by including a smart device such as a smart phone or a tablet.

Preferably, the screen equivalence determination algorithm includes a SIMPLE technique and a VDP technique.

Preferably, the execution result of the application includes at least one of sound, vibration, signal, sensor value, frame buffer, layer (surface), camera output, memory value, instruction set, image, packet, and moving image.

Preferably, the capturing unit determines a time required for capturing by exchanging messages with the application executing unit. The capturing time may be determined by a command input, a random timing, an AI, a signal or a button input, a defined scenario, An event occurrence, a predetermined time unit, and a voice command.

According to an aspect of the present invention, there is provided a method of determining equivalence of a screen, comprising the steps of: (A) executing an identical application in each of different types of devices according to a test scenario, (B) capturing and storing an execution result of the application from each of the devices; and (C) removing an unnecessary portion (an area other than the area of interest) in the execution result of the stored application And (D) comparing the execution results of the two removed or modified applications with each other using a screen equivalence determination algorithm based on a human visual system (HVS).

Preferably, the step (B) continuously communicates with the application execution unit that executes the test scenario, and the execution result of the application through the communication is a sound, a vibration, a signal, a sensor value, a frame buffer, ), A camera output, a memory value, an instruction set, an image, a packet, and a moving image.

Preferably, the screen equivalence determination algorithm includes a SIMPLE technique and a VDP technique. In the SIMPLE technique, the equivalence is compared using the values of the calculated idiosyncrasies. The VDP technique calculates the luminance and chrominance And comparing the equivalence using physical result values.

As described above, the apparatus and method for discriminating the screen equivalence for evaluating the compatibility according to the present invention are based on the human visual system (HVS), and as a result of measuring the degree of recognizing the difference between the two images, which is less susceptible to distortion and is less susceptible to differences in graphics hardware. As such, since the difference is perceived by the person, the screen equivalence can be more effectively discriminated.

1 is a block diagram illustrating a structure of a screen equivalence determining apparatus for evaluating compatibility of a smartphone app according to an embodiment of the present invention;
2 is a flowchart for explaining a screen equivalence determining method for evaluating compatibility of a smartphone app according to an embodiment of the present invention

Other objects, features and advantages of the present invention will become apparent from the detailed description of the embodiments with reference to the accompanying drawings.

Preferred embodiments of a screen equivalence determining apparatus and method for evaluating compatibility according to the present invention will be described with reference to the accompanying drawings. The present invention may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the invention to those skilled in the art. It is provided to let you know. Therefore, the embodiments described in the present specification and the configurations shown in the drawings are merely the most preferred embodiments of the present invention and are not intended to represent all of the technical ideas of the present invention. Therefore, various equivalents It should be understood that water and variations may be present.

1 is a block diagram illustrating a structure of a screen equivalence determining device for evaluating compatibility of a smartphone app according to an embodiment of the present invention.

A screen capturing unit 200, a matching processing unit 300 and an equivalence comparing unit 400, as shown in FIG.

The application execution unit 100 executes an application using a given test scenario in a connected device. This can be done manually by selecting a menu based on the text or index, or by running the scenario executor automatically to run multiple test scenarios. As an example, the game is executed manually, the UI is automatically executed, and the animation is semi-automatic. At this time, the device includes all smart devices such as a smart phone and a tablet.

The screen capture unit 200 captures and stores the execution result of the application from the screen of the device executed through the application execution unit 100. At this time, the execution result of the stored app includes a screen (image), a sound, a vibration, and a sensor operation. In this specification, the description is limited to the execution screen of the application as the execution result of the application. It should be noted, however, that this is for the purpose of illustration only and not for the purpose of limiting the technical idea to a specific embodiment.

The screen capturing unit 200 determines the time required to capture a screen by exchanging messages with the application executing unit 100. [ At this time, at which point the execution result of the application is to be captured and stored can be predefined and executed as described below, which is done through a known technology, and a detailed description thereof will be omitted.

- Enter the command to collect the results.

- Collect the results at random timing.

- Collect results in artificial intelligence.

- Enter the signal or button to collect the results.

- Collect the results according to the scenario.

- When an event occurs asynchronously during testing, it detects and collects the results.

- Collect the results after a certain period of time.

- Collect the results according to voice commands.

The matching processing unit 300 performs a function of removing or modifying a part (an area other than the area of interest) unnecessary for the experiment on the screen captured by the screen capturing unit 200. That is, more precise measurement results can be obtained by removing or modifying the time information of the time at which the screen is captured and the status bar in which the network and various states of the device are displayed. Since all devices under test use the same 38-pixel height status bar, remove or modify the status bar in the same way. The following is a description of how to exclude areas other than the area of interest from the test.

- Image: Change to the same color.

- Image: Change the alpha value to make it transparent.

- Image: Cut the image.

- Image: Adjust resolution (size).

delete

delete

delete

delete

delete

delete

The equivalence comparison unit 400 evaluates the equivalence by using a screen equivalence determination algorithm based on the human visual system (HVS) on the two result screens removed or modified in the matching processing unit 300. At this time, the screen equivalence determination algorithm includes a SIMPLE technique and a VDP technique. The SIMPLE technique outputs the map ratio as an evaluation result. The isometric ratio refers to the ratio of different pixels among the pixels constituting the entire screen in order to quantitatively measure the screen equivalence. That is, a higher value of the ratio means a larger difference between the two images, and a lower ratio means that the two images are equal. The VDP technique is a technique for expressing a human visual response. The VDP technique evaluates the fidelity of a distorted image compared with the original image, and outputs an image indicating a position of a probability of detection. That is, the VDP technique outputs a portion of the screen different from that of the SIMPLE technique as an evaluation result by describing a relative measure, not an absolute measure of image quality. At this time, the equivalence comparing unit 400 is made of a time difference comparing method and an optional space comparing method, and a comparison technique of difference, cognition, and geometry is used as a comparative example. As a comparison, machine learning and comparison of statistics through experimental data are used. CD, MP3, electronic signal, screen, color, print, and flash are used as output methods for judging the difference between the two execution results.

The operation of the screen equivalence determining apparatus for evaluating the compatibility according to the present invention will be described in detail with reference to the accompanying drawings.

2 is a flowchart illustrating a screen equivalence determination method for evaluating compatibility of a smartphone app according to an exemplary embodiment of the present invention.

Referring to FIG. 2, first, the application executing unit 100 executes the same application according to a test scenario in each of different types of devices, that is, a reference device and a target device (S10). At this time, assuming that the application is defined as "ApiDemos" consisting of a list view and that a test scenario capable of automatically searching for it is created by "JUnit", the concrete operation of the test scenario is "ApiDemos" Controls the items of the list view one by one, and stores the execution screen in the screen capture unit 200. [ Therefore, the created test scenario executes all the items of the list view of "ApiDemos" in the same way as above. And the execution of the app that is performed according to the test scenario is displayed on the screen of the device. At this time, the device includes smart devices such as a smart phone and a tablet.

When the execution result of the application is output to the screen according to the above scenario, the screen capture unit 200 reads the contents of the device frame buffer and stores the two image files (S20). At this time, the screen capture unit 200 continuously communicates with the application execution unit 100 to accurately store the execution result screen of the application, and transmits / receives the image as the execution result of the application. The execution result of the app includes not only the image but also sound, vibration, signal, sensor value, frame buffer, layer (surface), camera output, memory value, instruction set, image, packet, moving picture. However, as described above, the execution result of the application is limited to the image of the execution screen of the application, but is not limited thereto.

Then, the two image files thus stored are removed or modified by using the matching processing unit 300 to remove unnecessary parts (areas other than the area of interest), and then, using the equivalence comparing unit 400, Equivalence is evaluated using a screen equivalence discrimination algorithm based on the visual system (HVS). In this case, the screen equivalence determination algorithm includes a SIMPLE technique and a VDP technique.

The equivalence is compared using the value of the idle ratio calculated by the SIMPLE technique and the physical result of luminance and chrominance calculated through the VDP technique (S30). That is, the equivalence comparing unit 400 calculates the equivalence between the two images using the SIMPLE and VDP techniques, outputs the calculated results, selects the best result, , And smart devices like tablets.

At this time, the SIMPLE technique is a method of arithmetically comparing differences between pixel values of two images. That is, a higher value of the ratio means a larger difference between the two images, and a lower ratio means that the two images are equal. The VDP technique includes a calibration step of adjusting an input image, a step of applying a human visual system (HVS) model, and a step of displaying an HVS difference regarding a visual difference. That is, in the case of the VDP technique, the input is a parameter for two images and observation conditions and calibration, while the output is a new image showing the visual difference between the two images.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it is to be understood that the invention is not limited to the disclosed exemplary embodiments. It will be apparent to those skilled in the art that various modifications may be made without departing from the scope of the present invention. Accordingly, the true scope of the present invention should be determined by the technical idea of the appended claims.

Claims (16)

An application execution unit for executing an application based on a given test scenario in a connected device,
A capture unit for capturing and storing an execution result of an application from a device executed through the application execution unit;
A matching processing unit for removing or modifying a part (an area other than the area of interest) unnecessary for the experiment in the execution result of the application captured by the capturing unit;
And an equivalence comparison unit for evaluating equivalence by using a screen equivalence discrimination algorithm based on a human visual system (HVS) for two results removed or modified by the matching processing unit,
At this time, if the matching processing unit is a part unnecessary for the experiment, the image is removed or modified through any one of changing the image to the same color, changing the alpha value, cropping the image, and adjusting the resolution Wherein the screen equivalence discriminating apparatus comprises: The method according to claim 1,
Wherein the device includes a smart device such as a smart phone or a tablet. The method according to claim 1,
Wherein the screen equivalence discrimination algorithm includes a SIMPLE technique and a VDP technique. The method according to claim 1,
Wherein the execution result of the application includes at least one of sound, vibration, signal, sensor value, frame buffer, layer (surface), camera output, memory value, instruction set, image, The apparatus comprising: The method according to claim 1,
Wherein the application execution unit executes a test scenario manually by selecting a menu based on text or an index or automatically executes a test scenario through automatic execution of a scenario execution unit. The apparatus according to claim 1, wherein the capturing unit
Wherein the capturing time is a scene in which a signal or a button input, a capturing time is predefined, a time when a command is input, a random timing, a change of a screen by artificial intelligence, Wherein the test device is one of a generation of an event asynchronously during the test, a predetermined unit of time, and a voice command. delete The method according to claim 1,
Wherein the equivalence comparing unit compares time differences and selective spaces with each other. The method according to claim 1,
Wherein the equivalence comparing unit compares the execution result using the calculated value based on the recognition result and the geometry,
In this case, cognition refers to a cognitive difference modeling a human visual system in the cognitive field, and geometry refers to a distance difference calculated after mapping an image filler to an euclidean space in a geometric field. Screen equality discrimination device. The method according to claim 1,
Wherein the equivalence comparing unit uses a comparison technique of a machine learning (Decision Tree) and a statistical (logistic regression) verification using experimental data as comparison objects. The method according to claim 1,
Wherein the equivalence comparing unit uses any one of a binary file, an electronic signal, a color, a print, and a screen glare as execution results of two apps as a comparison criterion. (A) executing the same application in each of different types of devices according to a test scenario and outputting an execution result of the application from each device;
(B) capturing and storing the execution result of the application from each of the devices,
(C) removing or modifying an unnecessary part (an area other than the area of interest) in the execution result of the stored application;
(D) comparing the execution results of the two removed or modified apps with each other using a screen equivalence determination algorithm based on a human visual system (HVS)
At this time, the step (D) may be performed by removing or modifying a desired region through any one of a desired color change, an alpha value change, an image trimming, and a resolution (size) The method comprising the steps of: 13. The method of claim 12, wherein step (B)
Wherein the execution result of the application is continuously communicated with an application execution unit that executes the test scenario, and the execution result of the application through the communication includes at least one of sound, vibration, signal, sensor value, frame buffer, layer An instruction set, an image, a packet, and a moving picture. delete 13. The method of claim 12,
Wherein the area other than the area of interest in the step (C) includes time information of the time at which the information is captured, a network of the device, and a status bar in which various states are displayed. Identification method. 13. The method of claim 12,
The SIMPLE method and the SIMPLE method compare the equivalence using the calculated idiosyncratic value, and the VDP technique calculates the physical equivalence of the calculated luminance and chrominance And comparing the equivalence by using the resultant value.

Images