CN117076290A - Fine granularity universality problem detection method based on depth page exploration - Google Patents
Fine granularity universality problem detection method based on depth page exploration Download PDFInfo
- Publication number
- CN117076290A CN117076290A CN202310832392.7A CN202310832392A CN117076290A CN 117076290 A CN117076290 A CN 117076290A CN 202310832392 A CN202310832392 A CN 202310832392A CN 117076290 A CN117076290 A CN 117076290A
- Authority
- CN
- China
- Prior art keywords
- page
- universality
- current
- exploration
- app
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000001514 detection method Methods 0.000 title claims abstract description 20
- 238000000034 method Methods 0.000 claims abstract description 12
- 238000012544 monitoring process Methods 0.000 claims abstract description 4
- 238000010586 diagram Methods 0.000 claims description 8
- 238000006243 chemical reaction Methods 0.000 claims description 6
- 238000012360 testing method Methods 0.000 claims description 6
- 230000007704 transition Effects 0.000 claims description 5
- 238000004590 computer program Methods 0.000 claims description 4
- 230000002452 interceptive effect Effects 0.000 claims description 4
- 238000012857 repacking Methods 0.000 claims description 3
- 230000000694 effects Effects 0.000 claims description 2
- 230000003068 static effect Effects 0.000 description 5
- 230000008901 benefit Effects 0.000 description 2
- 238000004891 communication Methods 0.000 description 2
- 230000003993 interaction Effects 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 235000015114 espresso Nutrition 0.000 description 1
- 230000036541 health Effects 0.000 description 1
- 238000007689 inspection Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000008520 organization Effects 0.000 description 1
- 230000008569 process Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 238000013522 software testing Methods 0.000 description 1
- 230000001960 triggered effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/36—Preventing errors by testing or debugging software
- G06F11/3668—Software testing
- G06F11/3672—Test management
- G06F11/3684—Test management for test design, e.g. generating new test cases
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/36—Preventing errors by testing or debugging software
- G06F11/3668—Software testing
- G06F11/3696—Methods or tools to render software testable
Abstract
The invention discloses a fine-granularity universality problem detection method based on deep page exploration, which comprises the steps of firstly comprehensively exploring an input app, transmitting screenshots of a current page into a set monitoring catalog when exploring the page of the app, detecting whether each element accords with accessibility standards one by one, returning problem-related information, and simultaneously carrying out position marking on the transmitted screenshots according to the detected problem information. And then constructing a resource tree with a hierarchical structure from the source file obtained by decompilation and layout file information obtained by running, positioning the detected component containing the universality problem in the resource tree, judging and classifying the authenticity of the problem, and finally generating a detailed problem report. The method and the device can more comprehensively explore more application scenes, acquire high scene coverage rate, simultaneously detect more potential universality problems and reduce missing report problems. Meanwhile, the problems can be further screened and classified by constructing a resource tree, so that false alarm conditions are reduced.
Description
Technical Field
The invention relates to the technical field of mobile phone software testing, in particular to a fine granularity universality problem detection method based on deep page exploration.
Background
The increasing popularity of mobile applications makes them an important element in people's daily lives, and mobile applications have become an important medium for accessing information and services. However, many of these applications are inaccessible to disabled persons, making them difficult to use and enjoy the same benefits as non-disabled users. The World Health Organization (WHO) estimates that about 15% of the world's population suffers from disabilities, which necessitates application developers to ensure that all users have access to their applications, including disabled users. Despite the increasing awareness of unobstructed, many developers still have difficulty creating unobstructed applications due to lack of knowledge or awareness of the unobstructed guidelines. Thus, tools are needed that can help developers detect pervasive problems in their applications.
In recent years, researchers have developed a number of tools to test and detect the ubiquitous problems present in applications and conduct pilot research on these problems. These tools are largely divided into two categories, static and dynamic analysis tools. The static analysis tool is a tool for detecting universality problems by analyzing static information such as codes and resource files of an application program without running the application program. Among them, android link is a widely used static analysis tool that can identify and report many universality problems, such as lack of labels, unlabeled text, inability to zoom in, etc. In addition, tools such as PMD, checkstyle and FindBugs can also perform universality checking on Android application programs. However, conventional rule-based static inspection methods have been found to be ineffective and time consuming in detecting mobile universality problems. And dynamic analysis tools refer to tools that detect pervasive problems by running applications. These tools can simulate user interactions, detect pervasive problems in applications, and generate reports for further analysis. For example, accessibility Scanner is a dynamic analysis tool developed by Google that can automatically detect pervasive problems in applications and provide a solution. In addition, there are other dynamic analysis tools such as UI Automator and Espresso, etc. However, there are also some significant limitations to existing dynamic test tools: 1) When exploring the app, only the initial states of different active pages in the application program are captured, the active related UI scene triggered by the interaction component on the initial UI page is ignored, and the problem of missing report exists. 2) The lack of fine granularity analysis of the problem level has the problem of false alarm.
Disclosure of Invention
The invention aims to overcome the defects of the prior art and provides a fine granularity universality detection method based on deep page exploration.
The invention is realized by the following technical scheme:
a fine granularity universality problem detection method based on depth page exploration comprises the following steps:
step 1, fully exploring an input app
After the mobile phone app file in the original apk format is input, analyzing the current app, including repacking and decompiling operations; installing the repackaged app on an Android virtual machine; the exploration is performed by the following three steps:
step 1.1, obtaining basic properties of ICC from an Android management file or an intent-filters definition in Java codes obtained by decompilation, and randomly generating additional parameter combinations conforming to a specific data structure of the basic properties to correctly generate ICC messages and start a UI page from a console of an Android virtual machine;
step 1.2, after the UI page is successfully started, capturing the layout information of the elements of the current UI page, storing the layout information as a layout file, and then acquiring and positioning all interactable components on the current UI page through the layout file of the current UI page; the interactive component comprises a text field component and an operable component, for the text field component, basic attributes and character string types of the text field component are obtained from a layout file, character strings of corresponding types are randomly generated according to different character string types to fill the text field component, then the interactive component interacts with the positioned operable component in sequence, clicking operation is carried out, a new scene is tried to be entered, and the exploring operation is repeated in the new scene;
step 1.3, if the UI page can not be started successfully through ICC information, analyzing an active conversion diagram of the app, finding a caller of the UI page through conversion relations among different UI pages in the active conversion diagram, and triggering the UI page to be started through the caller; if the caller cannot be directly started, iteratively searching the caller of the previous stage upwards and attempting to start; after the UI page is successfully started in the mode, the UI page is thoroughly explored according to the operation of the step 1.2;
step 2, detecting universality problems
When exploring the UI page of the app, transmitting the screenshot of the current UI page into a set monitoring catalog, detecting whether each element accords with accessibility standards one by one, returning information related to the problem, and simultaneously carrying out position marking on the transmitted screenshot according to the detected problem information;
step 3, analyzing the resource tree
Step 3.1, constructing the source file obtained by decompilation in the step 1 and the layout file information obtained by running into a resource tree with a hierarchical structure, and obtaining a resource tree formed by nodes with rich attributes;
and 3.2, positioning the component containing the universality problem detected in the step 2 in the constructed resource tree, judging and classifying the authenticity of the problem according to the rich attribute of each node, and finally generating a detailed problem report.
In the technical scheme, the automatic testing framework UIAutostat tool is used for capturing the layout information of the current UI page elements.
In the above technical solution, the operable components include a Button, an icon Button ImageButton, and a check box CheckBox.
In the above technical solution, a Story Droid is used to parse an activity transition diagram (ATG) of the app.
In the above technical solution, in step 2, the universality of the current UI page is detected by using a detection interface provided by the ATF of Google, specifically Accessibility Events of the current UI page is grabbed by Accessibility API provided by the ATF, and whether each element meets the accessibility standard defined by the ATF is detected one by taking these shots and Accessibility Events as parameters to enter the detection interface.
In the above technical solution, in step 3.1, a node tree is built according to the hierarchical structure by using the components of the current page according to the layout information of the current UI page, and abundant attributes are given to each node according to all acquired related files.
The present invention may also be devised as a computer readable storage medium storing a computer program, said computer program implementing the steps of the method described hereinabove when executed.
The invention has the advantages and beneficial effects that:
the method and the device can more comprehensively explore more application scenes, acquire high scene coverage rate, simultaneously detect more potential universality problems and reduce missing report problems. Meanwhile, the problems can be further screened and classified by constructing a resource tree, so that false alarm conditions are reduced.
Drawings
FIG. 1 is a flow chart of a fine-grained universality problem detection method based on depth page exploration of the present invention.
Other relevant drawings may be made by those of ordinary skill in the art from the above figures without undue burden.
Detailed Description
In order to make the person skilled in the art better understand the solution of the present invention, the following describes the solution of the present invention with reference to specific embodiments.
Referring to fig. 1, a method for detecting fine-grained universality based on depth page exploration comprises the following steps:
step 1, fully exploring the input app.
After the mobile phone app file in the original apk format is input, analyzing the current app, including repacking and decompiling operations; installing the repackaged app on an Android virtual machine so as to facilitate subsequent dynamic operation; in order to launch more scenes of the app as much as possible, exploration is performed by the following three steps.
In step 1.1, in an Android application, the inter-component communication (InterComponent Communication, ICC) message is essentially an Intent object with many data items, and the Intent object carries information that the Android system uses to determine which UI page to launch, so that by parsing the ICC data, the target UI page can be launched. In this step, basic properties of ICC are obtained from Android management file or content-files definition in Java code obtained by decompilation, and additional parameter combinations conforming to specific data structures are randomly generated to correctly generate ICC messages and start UI pages from a console of an Android virtual machine.
Step 1.2, after the UI page is successfully started, capturing the layout information of the elements of the current UI page by using an automatic testing framework UIAutometer tool, storing the layout information as a layout file, and then acquiring and positioning all interactable components on the current UI page through the layout file of the current UI page; the interactable components comprise two types of text field components (EditText) and operable components (such as buttons Button, icon buttons ImageButton, check boxes and the like), basic attributes of the text field components and character string types thereof are obtained from a layout file for the text field components (EditText), character strings of corresponding types are randomly generated according to different character string types to fill the text field components, such as telephones, dates, mail addresses and texts, and the like, and then the interacted with the positioned operable components in sequence to perform clicking operation to attempt to enter a new scene (the new scene refers to a new state of a current UI page), and the exploring operations are repeated in the new scene.
Step 1.3, if the UI page cannot be started successfully through the ICC message, analyzing an active transition diagram (ATG) of the app by using Story droid ([ S.Chen, L.Fan, C.Chen, T.Su, W.Li, Y.Liu, and L.xu, "Story droid: automated generation of storyboard for Android apps," in 2019IEEE/ACM 41st International Conference on Software Engineering (ICSE) & IEEE,2019, pp.596-607. ]), finding a caller of the UI page through a transition relation between different UI pages in the active transition diagram, and then triggering the UI page to be started through the caller; if the caller cannot be directly started, iteratively searching the caller of the previous stage upwards and attempting to start; after the UI page is successfully started in the mode, the UI page is thoroughly explored according to the operation of the step 1.2.
And 2, detecting the universality problem.
The universality of the current UI page is detected by using a detection interface provided by Google ATF (Accessibility Testing Framework). Specifically, accessing the barrier-free functionality on the Android device through the Accessibility API provided by the ATF, accessibility API can scan the UI page of the app to access the various elements of the current UI page and invoke the detection interface to check if they are accessible.
When the UI page of the app is explored in step 1, a screenshot of the current UI page is transferred into a set monitoring catalog, accessibility Events of the current UI page is grabbed by Accessibility API provided by the ATF, whether each element meets accessibility standards defined by the ATF can be detected one by taking the screenshot and Accessibility Events as parameters and transferring into a detection interface, information related to a problem is returned, and meanwhile, the position of the imported screenshot is marked according to the detected problem information.
And 3, analyzing the resource tree.
And 3.1, constructing a resource tree. And (3) constructing the source file (such as a configuration file, a resource file such as an icon and Java source code) obtained in the decompilation of the step (1) and the layout file and other information obtained by using the UIAutozer in the running process into a resource tree with a hierarchical structure, and obtaining the resource tree formed by nodes with rich attributes. Specifically, the components of the current page are constructed into a node tree according to the hierarchical structure according to the layout information of the current UI page, and rich attributes of each node are endowed according to all acquired related files, such as related codes, corresponding icons, layout information and the like of each node.
And 3.2, analyzing the problems. And (3) positioning the component containing the universality problem detected in the step (2) in the constructed resource tree, and judging and classifying the authenticity of the problem according to the rich attribute of each node. And finally, generating a detailed problem report.
The foregoing has described exemplary embodiments of the invention, it being understood that any simple variations, modifications, or other equivalent arrangements which would not unduly obscure the invention may be made by those skilled in the art without departing from the spirit of the invention.
Claims (7)
1. The fine granularity universality problem detection method based on depth page exploration is characterized by comprising the following steps of:
step 1, fully exploring an input app
After the mobile phone app file in the original apk format is input, analyzing the current app, including repacking and decompiling operations; installing the repackaged app on an Android virtual machine; the exploration is performed by the following three steps:
step 1.1, obtaining basic properties of ICC from an Android management file or an intent-filters definition in Java codes obtained by decompilation, and randomly generating additional parameter combinations conforming to a specific data structure of the basic properties to correctly generate ICC messages and start a UI page from a console of an Android virtual machine;
step 1.2, after the UI page is successfully started, capturing the layout information of the elements of the current UI page, storing the layout information as a layout file, and then acquiring and positioning all interactable components on the current UI page through the layout file of the current UI page; the interactive component comprises a text field component and an operable component, for the text field component, basic attributes and character string types of the text field component are obtained from a layout file, character strings of corresponding types are randomly generated according to different character string types to fill the text field component, then the interactive component interacts with the positioned operable component in sequence, clicking operation is carried out, a new scene is tried to be entered, and the exploring operation is repeated in the new scene;
step 1.3, if the UI page can not be started successfully through ICC information, analyzing an active conversion diagram of the app, finding a caller of the UI page through conversion relations among different UI pages in the active conversion diagram, and triggering the UI page to be started through the caller; if the caller cannot be directly started, iteratively searching the caller of the previous stage upwards and attempting to start; after the UI page is successfully started in the mode, the UI page is thoroughly explored according to the operation of the step 1.2;
step 2, detecting universality problems
When exploring the UI page of the app, transmitting the screenshot of the current UI page into a set monitoring catalog, detecting whether each element accords with accessibility standards one by one, returning information related to the problem, and simultaneously carrying out position marking on the transmitted screenshot according to the detected problem information;
step 3, analyzing the resource tree
Step 3.1, constructing the source file obtained by decompilation in the step 1 and the layout file information obtained by running into a resource tree with a hierarchical structure, and obtaining a resource tree formed by nodes with rich attributes;
and 3.2, positioning the component containing the universality problem detected in the step 2 in the constructed resource tree, judging and classifying the authenticity of the problem according to the rich attribute of each node, and finally generating a detailed problem report.
2. The depth page exploration-based fine-grained universality problem detection method according to claim 1, wherein the method comprises the following steps: the current UI page element layout information is grabbed using an automated test framework UIAutometer tool.
3. The depth page exploration-based fine-grained universality problem detection method according to claim 1, wherein the method comprises the following steps: the operable components include buttons Button, icon buttons ImageButton, check box CheckBox.
4. The depth page exploration-based fine-grained universality problem detection method according to claim 1, wherein the method comprises the following steps: the activity transition diagram of the app is parsed using StoryDroid.
5. The depth page exploration-based fine-grained universality problem detection method according to claim 1, wherein the method comprises the following steps: in step 2, the universality of the current UI page is detected by using a detection interface provided by the ATF of Google, specifically Accessibility Events of the current UI page is grabbed by Accessibility API provided by the ATF, and whether each element meets the accessibility standard defined by the ATF is detected one by taking these shots and Accessibility Events as parameters to enter the detection interface.
6. The depth page exploration-based fine-grained universality problem detection method according to claim 1, wherein the method comprises the following steps: in step 3.1, the components of the current page are constructed into a node tree according to the hierarchical structure according to the layout information of the current UI page, and rich attributes are given to each node according to all acquired related files.
7. A computer readable storage medium, characterized in that a computer program is stored, which computer program, when executed, implements the steps of the method according to any of claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310832392.7A CN117076290B (en) | 2023-07-07 | 2023-07-07 | Fine granularity universality problem detection method based on depth page exploration |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310832392.7A CN117076290B (en) | 2023-07-07 | 2023-07-07 | Fine granularity universality problem detection method based on depth page exploration |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN117076290A true CN117076290A (en) | 2023-11-17 |
| CN117076290B CN117076290B (en) | 2024-03-19 |
Family
ID=88703146
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310832392.7A Active CN117076290B (en) | 2023-07-07 | 2023-07-07 | Fine granularity universality problem detection method based on depth page exploration |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117076290B (en) |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20150339213A1 (en) * | 2014-05-21 | 2015-11-26 | Cgi Technologies And Solutions Inc. | Automated testing of an application system |
| CN109359029A (en) * | 2018-08-27 | 2019-02-19 | 浙江大学 | It is a kind of automate non-intrusion type Android apply accessible support detection method |
| CN111459802A (en) * | 2020-03-09 | 2020-07-28 | 北京邮电大学 | Method, device and equipment for testing WebView page of Android application program based on GUI |
| CN113238952A (en) * | 2021-05-25 | 2021-08-10 | 中国科学院软件研究所 | Intelligent auxiliary guide test method and device based on application program state transition diagram |
| CN115687115A (en) * | 2022-10-31 | 2023-02-03 | 上海计算机软件技术开发中心 | Automatic testing method and system for mobile application program |
| CN115729828A (en) * | 2022-11-28 | 2023-03-03 | 北京航空航天大学 | Android application testing method based on Monte Carlo tree search and deep reinforcement learning |
| CN116383054A (en) * | 2023-03-30 | 2023-07-04 | 南京大学 | Mobile application model construction method combining static analysis and dynamic exploration |
-
2023
- 2023-07-07 CN CN202310832392.7A patent/CN117076290B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20150339213A1 (en) * | 2014-05-21 | 2015-11-26 | Cgi Technologies And Solutions Inc. | Automated testing of an application system |
| CN109359029A (en) * | 2018-08-27 | 2019-02-19 | 浙江大学 | It is a kind of automate non-intrusion type Android apply accessible support detection method |
| CN111459802A (en) * | 2020-03-09 | 2020-07-28 | 北京邮电大学 | Method, device and equipment for testing WebView page of Android application program based on GUI |
| CN113238952A (en) * | 2021-05-25 | 2021-08-10 | 中国科学院软件研究所 | Intelligent auxiliary guide test method and device based on application program state transition diagram |
| CN115687115A (en) * | 2022-10-31 | 2023-02-03 | 上海计算机软件技术开发中心 | Automatic testing method and system for mobile application program |
| CN115729828A (en) * | 2022-11-28 | 2023-03-03 | 北京航空航天大学 | Android application testing method based on Monte Carlo tree search and deep reinforcement learning |
| CN116383054A (en) * | 2023-03-30 | 2023-07-04 | 南京大学 | Mobile application model construction method combining static analysis and dynamic exploration |
Also Published As
| Publication number | Publication date |
|---|---|
| CN117076290B (en) | 2024-03-19 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107562620B (en) | Automatic buried point setting method and device | |
| US8001532B1 (en) | System and method for generating source code-based test cases | |
| Zheng et al. | Automated test input generation for android: Towards getting there in an industrial case | |
| US20130159890A1 (en) | Systems and methods for identifying user interface (ui) elements | |
| CN112187585B (en) | Network protocol testing method and device | |
| CN103777967B (en) | Page returning method, page generation method and device | |
| US20070226241A1 (en) | System and method for generating a custom application | |
| CN109739717A (en) | A kind of method and device of page data acquisition, server | |
| US20070268822A1 (en) | Conformance control module | |
| Li et al. | Peruim: Understanding mobile application privacy with permission-ui mapping | |
| Benford et al. | On lions, impala, and bigraphs: Modelling interactions in physical/virtual spaces | |
| US20060080640A1 (en) | Method, system and program product for retrofitting collaborative components into existing software applications | |
| Cameron et al. | Back to the future: lessons from knowledge engineering methodologies for chatbot design and development | |
| CN107301347A (en) | A kind of method that Hbuilder App calling figures are obtained based on static analysis | |
| Ma et al. | Aladdin: Automating release of deep-link APIs on Android | |
| US11829278B2 (en) | Secure debugging in multitenant cloud environment | |
| CN113778897B (en) | Automatic test method, device and equipment for interface and storage medium | |
| CN111767229A (en) | Performance test method, device and equipment | |
| CN114443905A (en) | Interface document updating method and device, electronic equipment and readable storage medium | |
| CN108595656B (en) | Data processing method and system | |
| Wu et al. | AppCheck: a crowdsourced testing service for android applications | |
| CN117076290B (en) | Fine granularity universality problem detection method based on depth page exploration | |
| Anbunathan et al. | Automatic test generation from UML sequence diagrams for android mobiles | |
| CN111459774A (en) | Method, device and equipment for acquiring flow of application program and storage medium | |
| Kuznetsov et al. | What do all these buttons do? statically mining android user interfaces at scale |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |