CN112231234B - Cross-platform user interface automatic testing method, device, equipment and storage medium - Google Patents
Cross-platform user interface automatic testing method, device, equipment and storage medium Download PDFInfo
- Publication number
- CN112231234B CN112231234B CN202011431464.XA CN202011431464A CN112231234B CN 112231234 B CN112231234 B CN 112231234B CN 202011431464 A CN202011431464 A CN 202011431464A CN 112231234 B CN112231234 B CN 112231234B
- Authority
- CN
- China
- Prior art keywords
- application software
- current page
- target object
- test
- user interface
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/36—Preventing errors by testing or debugging software
- G06F11/3668—Software testing
- G06F11/3672—Test management
- G06F11/3684—Test management for test design, e.g. generating new test cases
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/36—Preventing errors by testing or debugging software
- G06F11/3668—Software testing
- G06F11/3672—Test management
- G06F11/3688—Test management for test execution, e.g. scheduling of test suites
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Computer Hardware Design (AREA)
- Quality & Reliability (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Debugging And Monitoring (AREA)
Abstract
The embodiment of the application provides a cross-platform user interface automatic testing method, a device, equipment and a storage medium, wherein the method comprises the following steps: analyzing a test instruction of a current page of application software displayed on the terminal equipment according to a preset rule to obtain a target object to be tested and test operation performed on the target object; acquiring a screenshot of a current page of application software displayed on terminal equipment, and identifying to obtain position information of a target object in the current page of the application software; testing the target object according to the position information of the target object to obtain a result of the automatic test of the user interface; the test instructions are written according to preset natural language grammars, and the preset natural language grammars for writing the test instructions are different under the condition that the current page of the application software is different in display on terminal equipment of different platforms. The method and the device for testing the user interface can enable the automatic testing of the user interface to have cross-platform capability.
Description
Technical Field
The present application relates to the field of testing technologies, and in particular, to a method and an apparatus for automatically testing a User Interface (UI), an electronic device, and a computer-readable storage medium.
Background
User interface automated testing is primarily based on the functionality of the user interface. The currently widely used user interface automated testing framework and tools mainly include: an Appium automated testing framework, an automated testing tool developed based on an application view tree, and the like.
The currently widely used user interface automated testing framework and tool mainly implement automated testing based on a control tree, because control information of different device platforms is different, such as an Android platform and an iOS platform, user interface automated testing programs need to be written for different installation platforms respectively for the same application software (application software), and because many service logics of the same application software installed on different platforms are the same, a plurality of user interface automated testing programs are written for different installation platforms, the efficiency of writing the user interface automated testing programs is reduced, and the cost of maintaining the user interface automated testing programs is increased.
Disclosure of Invention
The embodiment of the application provides a cross-platform user interface automatic test method, a cross-platform user interface automatic test device, electronic equipment and a computer readable storage medium, which can enable the automatic test of a user interface to have cross-platform capability, improve the efficiency of compiling the user interface automatic test program and effectively reduce the cost of maintaining the user interface automatic test program.
In order to solve the above technical problem, the embodiment of the present application is implemented as follows:
in a first aspect, a cross-platform user interface automated testing method is provided, including:
analyzing a test instruction of a current page of application software displayed on terminal equipment according to a preset rule to obtain a target object to be tested and test operation performed on the target object; the test instruction is written according to preset natural language grammar, and under the condition that the display of the current page of the application software on terminal equipment of different platforms is different, the preset natural language grammar for writing the test instruction is different;
acquiring a screenshot of the current page of the application software displayed on the terminal equipment, and identifying and processing the screenshot of the current page of the application software to obtain position information of the target object in the current page of the application software;
and carrying out the test operation on the target object according to the position information of the target object to obtain the result of the automatic test of the user interface.
In a second aspect, there is provided a cross-platform user interface automated testing apparatus, comprising:
the analysis module is used for analyzing a test instruction of a current page of the application software displayed on the terminal equipment according to a preset rule to obtain a tested target object and test operation performed on the target object; the test instruction is written according to preset natural language grammar, and under the condition that the display of the current page of the application software on terminal equipment of different platforms is different, the preset natural language grammar for writing the test instruction is different;
the identification module is used for acquiring a screenshot of the current page of the application software displayed on the terminal equipment, and identifying the screenshot of the current page of the application software to obtain the position information of the target object in the current page of the application software;
and the processing module is used for carrying out the test operation on the target object according to the position information of the target object to obtain the result of the automatic test of the user interface.
In a third aspect, an embodiment of the present application provides an electronic device, including a processor, a communication interface, a memory, and a communication bus; the processor, the communication interface and the memory complete mutual communication through a bus; the memory is used for storing a computer program; the processor is configured to execute the program stored in the memory, and implement the cross-platform user interface automated testing method according to the first aspect.
In a fourth aspect, the present application provides a computer-readable storage medium, on which a computer program is stored, where the computer program, when executed by a processor, implements the cross-platform user interface automated testing method according to the first aspect.
According to the cross-platform user interface automatic testing method, the cross-platform user interface automatic testing device, the electronic equipment and the computer readable storage medium, the testing instruction of the current page of the application software displayed on the terminal equipment is analyzed according to the preset rule, and the tested target object and the testing operation performed on the target object are obtained; acquiring a screenshot of a current page of application software displayed on terminal equipment, and identifying the screenshot of the current page of the application software to obtain position information of a target object in the current page of the application software; testing the target object according to the position information of the target object to obtain a result of the automatic test of the user interface; the testing instructions are written according to preset natural language grammars, and under the condition that the display of the current page of the application software on terminal equipment of different platforms is different, the preset natural language grammars for writing the testing instructions are different; by utilizing a machine learning technology and a natural language processing technology in the AI technology, the position of a tested target object can be accurately determined in a page by identifying a screenshot of a page of application software displayed on a terminal device, the target object can be accurately operated, a test instruction is compiled through a preset natural language grammar, the programming of the user interface automatic test can be simplified, the strong association relationship between an installation platform of the application software and a test program is decoupled, the user interface test that the same test program is adaptive to different platforms of application software is realized, the test program is maximally reused, the user interface automatic test has a cross-platform capability, the efficiency of compiling the user interface automatic test program is improved, and the cost of maintaining the user interface automatic test program is effectively reduced.
Drawings
In order to more clearly illustrate one or more embodiments or technical solutions in the prior art of the present application, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments described in the present application, and other drawings can be obtained by those skilled in the art without inventive exercise;
FIG. 1 is a flowchart illustrating an implementation manner of a cross-platform user interface automated testing method according to an embodiment of the present application;
FIG. 2 is a schematic flowchart illustrating another implementation manner of a cross-platform user interface automated testing method according to an embodiment of the present application;
FIG. 3 is a flowchart illustrating an implementation manner of parsing a test command according to a preset rule according to an embodiment of the present application;
FIG. 4 is a flowchart illustrating an implementation manner of identifying a screenshot of a page according to an embodiment of the present application;
FIG. 5 is a flowchart illustrating a test program writing method of the cross-platform UI automation test method according to the embodiment of the present application;
FIG. 6 is a schematic diagram illustrating a visual display of a test program of the cross-platform UI automated testing method according to the embodiment of the application;
FIG. 7 is a schematic diagram of a cross-platform UI automation test apparatus according to an embodiment of the application;
fig. 8 is a schematic structural diagram of another component of the cross-platform ui automatic testing device according to an embodiment of the present disclosure.
Detailed Description
In order to make those skilled in the art better understand the technical solutions in one or more embodiments of the present application, the technical solutions in one or more embodiments of the present application will be clearly and completely described below with reference to the drawings in one or more embodiments of the present application, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all embodiments. All other embodiments that can be derived by a person skilled in the art from one or more embodiments of the present application without inventive step shall fall within the scope of protection of this document.
Fig. 1 is a schematic flowchart of an implementation manner of a cross-platform ui automated testing method according to an embodiment of the present disclosure, where the method shown in fig. 1 may be executed by a cross-platform ui automated testing apparatus as an execution main body, and the cross-platform ui automated testing apparatus may be installed in a general terminal device, such as a desktop computer, a notebook computer, a tablet computer, a mobile phone, a wearable device, or the like, or may be installed in a dedicated testing device, which is not limited in this embodiment of the present disclosure, and as shown in fig. 1, the cross-platform ui automated testing method at least includes:
s102, analyzing a test instruction of a current page of application software displayed on the terminal equipment according to a preset rule to obtain a tested target object and test operation performed on the target object; the test instructions are written according to preset natural language grammars, and the preset natural language grammars for writing the test instructions are different under the condition that the current page of the application software is different in display on terminal equipment of different platforms.
In this embodiment of the application, an object of the user interface automated testing may be application software (application software) that can be installed on terminal devices of different platforms, so that the same application software installed on the terminal devices of different platforms is tested by the same user interface automated testing method. For example, the application software may be life utility software, social software, efficiency office software, music video software, shopping software, travel software, etc., the terminal device may be a mobile phone, a tablet computer, a vehicle-mounted computer, a wearable device, etc., and the platform of the terminal device may include one of the following: the Android platform, the iOS platform, the Windows platform, the Linux platform, and the like, and the embodiment of the present application does not limit the types of the application software, the types of the terminal devices, and the types of the platforms of the terminal devices.
In the embodiment of the application, when the same application software installed on the terminal devices of different platforms is tested by the same user interface automatic testing method, the application software installed on the terminal devices is firstly opened, and then the current page of the application software displayed on the terminal devices is tested according to the testing instruction. The test instruction can be written according to a preset natural language grammar; under the condition that the display of the current page of the application software on the terminal equipment of different platforms is different, the preset natural language grammars for writing the test instruction are different; and under the condition that the display of the current page of the application software on the terminal equipment of different platforms is not different, the preset natural language grammars for writing the test instruction are the same. The preset natural language grammar may be all grammars which are preset according to the natural language grammar and support the automatic test of the user interface, for example, a natural language grammar library may be created, and the test instruction may be written by using the natural language grammar library.
In the embodiment of the application, when the current page of the application software displayed on the terminal device is tested according to the test instruction, the test instruction of the current page of the application software displayed on the terminal device can be analyzed according to a preset rule, so that a tested target object and a test operation performed on the target object are obtained. The preset rule can be a rule for splitting, converting and the like the test instruction, which is preset according to the natural language grammar; the target object of the test can be page elements such as characters, images or icons and the like which can be operated in the page of the application software; the test operation performed on the target object may include one of: click operation, slide operation, input operation, check operation, judgment operation, assignment operation, cycle operation and the like. The type of test operation performed on the target object may be determined according to the type of the target object. The type of the target object to be tested and the type of the test operation performed on the target object are not limited in the embodiments of the present application.
And S104, acquiring a screenshot of the current page of the application software displayed on the terminal equipment, and identifying and processing the screenshot of the current page of the application software to obtain the position information of the target object in the current page of the application software.
In the embodiment of the application, after the application software installed on the terminal equipment is opened, the terminal equipment can intercept the screenshot of the current page of the application software, send the screenshot to the cross-platform user interface automatic testing device and acquire the screenshot of the current page of the application software displayed on the terminal equipment; or, the user interface of the terminal device may be displayed on the cross-platform user interface automatic testing device, and after the application software installed on the terminal device is opened, the cross-platform user interface automatic testing device captures a screenshot of the current page of the application software, and obtains the screenshot of the current page of the application software displayed on the terminal device; the embodiment of the application does not limit the implementation manner of obtaining the screenshot of the current page of the application software displayed on the terminal device.
In the embodiment of the application, after the screenshot of the current page of the application software displayed on the terminal device is obtained, the page elements in the screenshot of the current page of the application software can be identified, and the position information of the target object in the current page of the application software is obtained according to the result of identifying the page elements. Optionally, the screenshot of the current page of the application software may be identified through machine learning, for example, the machine learning may be a deep learning model, and an implementation manner of identifying the screenshot of the current page of the application software in the embodiment of the present application is not limited.
And S106, carrying out test operation on the target object according to the position information of the target object to obtain the result of the automatic test of the user interface.
In the embodiment of the application, after the position information of the target object in the current page of the application software is obtained, the target object can be correspondingly tested according to the position information of the target object in the current page, so that the result of the automatic test of the user interface is obtained. Optionally, if only one target object is included in the current page of the application software, the target object is a target object for testing, and at this time, corresponding testing operation may be directly performed on the target object according to the position information of the target object in the current page; if the current page of the application software contains more than one target object, the target object to be tested needs to be determined in the current page of the application software, and then the target object is subjected to corresponding testing operation according to the position information of the determined target object in the current page, for example, the target object to be tested obtained by analyzing the testing instruction may include the ordinal number of the target object in the current page of the application software, and the target object to be tested may be determined in the current page of the application software according to the ordinal number of the target object in the current page of the application software. The embodiment of the present application does not limit the implementation manner of performing the test operation on the target object according to the position information of the target object.
According to the cross-platform user interface automatic testing method, a testing instruction of a current page of application software displayed on the terminal equipment is analyzed according to a preset rule, and a tested target object and testing operation performed on the target object are obtained; acquiring a screenshot of a current page of application software displayed on terminal equipment, and identifying the screenshot of the current page of the application software to obtain position information of a target object in the current page of the application software; testing the target object according to the position information of the target object to obtain a result of the automatic test of the user interface; the testing instructions are written according to preset natural language grammars, and under the condition that the display of the current page of the application software on terminal equipment of different platforms is different, the preset natural language grammars for writing the testing instructions are different; by utilizing a machine learning technology and a natural language processing technology in the AI technology, the position of a tested target object can be accurately determined in a page by identifying a screenshot of a page of application software displayed on a terminal device, the target object can be accurately operated, a test instruction is compiled through a preset natural language grammar, the programming of the user interface automatic test can be simplified, the strong association relationship between an installation platform of the application software and a test program is decoupled, the user interface test that the same test program is adaptive to different platforms of application software is realized, the test program is maximally reused, the user interface automatic test has a cross-platform capability, the efficiency of compiling the user interface automatic test program is improved, and the cost of maintaining the user interface automatic test program is effectively reduced.
The cross-platform user interface automated testing method of the present application will be described in detail below with reference to the embodiments of fig. 2 to 6.
Fig. 2 is a schematic flowchart of another implementation manner of the cross-platform user interface automated testing method according to the embodiment of the present application, where fig. 2 is a cross-platform user interface automated testing method in a case where displays of a current page of application software on terminal devices of different platforms are different, and as shown in fig. 2, the cross-platform automated testing method at least includes:
s202, platform type information of the terminal equipment is obtained, and a test instruction of a current page of the application software displayed on the terminal equipment matched with the platform type information is selected.
In the embodiment of the application, when the current page of the application software displayed on the terminal device is tested according to the test instruction, S204 can be directly executed to obtain the target object to be tested and the test operation performed on the target object under the condition that there is no difference in the display of the current page of the application software on the terminal devices of different platforms; for the situation that the display of the current page of the application software on the terminal devices of different platforms is different, the platform type information of the terminal device may be obtained first, the test instruction of the current page of the application software displayed on the terminal device matched with the platform type information is selected, and then S204 is executed to obtain the target object to be tested and the test operation performed on the target object. When the test instruction is written according to the preset natural language grammar, the test instruction can be written respectively for the pages with different display of the application software on different platforms by adding the platform type judgment logic.
And S204, analyzing the test instruction of the current page of the application software displayed on the terminal equipment according to a preset rule to obtain a tested target object and test operation performed on the target object.
In the embodiment of the present application, the description about S204 may refer to the description about S102 in fig. 1, and thus will not be repeated here.
S206, acquiring a screenshot of the current page of the application software displayed on the terminal equipment, and identifying and processing the screenshot of the current page of the application software to obtain the position information of the target object in the current page of the application software.
In the embodiment of the present application, the description about S206 may refer to the description about S104 in fig. 1, and thus will not be repeated here.
And S208, carrying out test operation on the target object according to the position information of the target object to obtain a result of the automatic test of the user interface.
In the embodiment of the present application, the description about S208 may refer to the description about S106 in fig. 1, and thus will not be repeated here.
In this embodiment, under the condition that the current page of the application software is displayed differently on the terminal devices of different platforms, the current page of the application software displayed on the terminal device is tested by respectively setting test instructions for the pages of the application software which are displayed differently on the different platforms in the same test program, acquiring the platform type information of the terminal device, selecting the test instruction matched with the platform type information, and switching the test instruction according to the platform type, so that the test of the user interface of the same application software on the different platforms through one test program is realized.
Fig. 3 is a schematic flowchart of an implementation manner of performing parsing processing on a test instruction according to a preset rule in the embodiment of the present application, and as shown in fig. 3, the parsing processing is performed on the test instruction of a current page of application software displayed on a terminal device according to the preset rule to obtain a target object to be tested and a test operation performed on the target object, where the parsing processing at least includes:
s302, according to a preset natural language grammar rule, splitting a test instruction of a current page of application software displayed on the terminal equipment to obtain a test target object and test operation performed on the target object.
In this embodiment, a natural language processing engine may be constructed, and a test instruction of a current page of application software displayed on a terminal device may be split according to a preset natural language grammar rule to obtain a test target object and a test operation performed on the target object.
S304, converting the tested target object and the test operation performed on the target object into a machine executable language according to a preset natural language and machine language comparison table.
In this embodiment, a natural language to machine language converter may be constructed, and a test target object obtained by splitting a test instruction and a test operation performed on the test target object may be converted into a machine executable language according to a preset natural language to machine language comparison table.
According to the embodiment, the test instruction of the current page of the application software displayed on the terminal equipment can be split and converted through the preset natural language grammar rule and the natural language and machine language comparison table, so that the test instruction compiled according to the preset natural language grammar is converted into the machine executable language through the natural language processing technology, the test equipment can test the current page of the application software displayed on the terminal equipment according to the machine executable language corresponding to the test instruction, the program design of the user interface automatic test can be simplified, and a feasible basis is provided for cross-platform application of the user interface automatic test program.
Fig. 4 is a schematic flowchart of an implementation manner of identifying and processing a screenshot of a page in an embodiment of the present application, and as shown in fig. 4, acquiring the screenshot of a current page of application software displayed on a terminal device, and identifying and processing the screenshot of the current page of the application software to obtain location information of a target object in the current page of the application software at least includes:
s402, capturing a screenshot of the current page of the application software displayed on the terminal equipment.
In this embodiment, the screenshot of the current page of the application software displayed on the terminal device is the screenshot of the current page of the application software captured by the cross-platform user interface automatic testing device after the user interface of the terminal device is displayed on the cross-platform user interface automatic testing device and the application software installed on the terminal device is opened.
And S404, performing character recognition processing and/or image recognition processing on the screenshot of the current page of the application software.
In this embodiment, the text in the screenshot of the current page of the application software may be subjected to text Recognition processing, for example, the text Recognition processing may be performed by Optical Character Recognition (OCR); the image recognition processing may be performed on the icon and the image in the screenshot of the current page of the application software, for example, the image recognition processing may be performed on the icon in the screenshot of the current page through general icon recognition, and the image recognition processing may be performed on the image in the screenshot of the current page through custom image matching. In some optional examples, the screenshots of the current page of the application software can be respectively subjected to character recognition processing, general icon recognition in image recognition processing and custom image matching through three deep learning models; in other optional examples, the method may include performing character detection, icon detection, and image detection on a screenshot of a current page of the application software through a deep learning model, marking out a character region, an icon region, and an image region in the screenshot of the current page of the application software, and performing character recognition processing and image recognition processing on the character region, the icon region, and the image region marked in the screenshot of the current page of the application software through three deep learning models, respectively; the embodiment of the application does not limit the implementation mode of performing character recognition processing and image recognition processing on the screenshot of the current page of the application software.
S406, determining the position information of the target object in the current page of the application software according to the result of the character recognition processing and/or the result of the image recognition processing.
In this embodiment, the position information of the target object in the current page of the application software may be determined according to a result of performing text recognition processing and/or image recognition processing on the screenshot of the current page of the application software. The current page of the application software can comprise one target object or a plurality of target objects; when the current page of the application software comprises a target object, according to the result of the character recognition processing and/or the result of the image recognition processing, determining the position information of the target object in the current page of the application software, namely the position information of the tested target object in the current page of the application software; when the current page of the application software comprises a plurality of target objects, according to the result of the character recognition processing and/or the result of the image recognition processing, the position information of the determined target objects in the current page of the application software is a plurality of, and the position information of the tested target objects in the current page of the application software needs to be determined.
In the embodiment, the character recognition processing and/or the image recognition processing are/is performed on the screenshot of the current page of the application software, then the position information of the target object in the current page of the application software is determined according to the result of the character recognition processing and/or the image recognition processing, the character recognition processing and/or the image recognition processing can be performed on the screenshot of the page of different application software of different terminal equipment through the same character recognition processing and/or image recognition processing model, and the universality of the cross-platform user interface automatic test method can be ensured.
Fig. 5 is a schematic flowchart of a test program writing method of the cross-platform user interface automated testing method according to the embodiment of the present application, and fig. 6 is a schematic diagram of a visual display of a test program of the cross-platform user interface automated testing method according to the embodiment of the present application.
As shown in fig. 5, the test program writing method at least includes: s502, judging whether the display of the current page of the application software on the terminal equipment of different platforms is different; if the display of the current page of the application software on the terminal equipment of different platforms is different, executing S504 to select the platform suitable for the terminal equipment; after selecting a platform suitable for the terminal equipment, executing S506 to write a test instruction according to a preset natural language grammar; if there is no difference in the display of the current page of the application software on the terminal devices of different platforms, S506 is directly executed to write a test instruction according to the preset natural language grammar.
The steps S504 and S506 are sequentially executed under the condition that the current page of the application software is displayed differently on the terminal devices of different platforms, that is, the platforms of different terminal devices installed by the application software, such as an Android platform and an iOS platform, are respectively selected, and then the test instructions are respectively written according to the preset natural language grammar after the platform of the terminal device is selected. As shown in fig. 6, 06 to 11 indicate that the pages of the application software have different displays on the terminal devices of different platforms, and the pages are respectively selected from the Android platform and the iOS platform to write test instructions according to the preset natural language syntax.
And directly executing S506 under the condition that the display of the current page of the application software on the terminal equipment of different platforms is not different, and compiling the test instruction according to the preset natural language grammar without distinguishing the platforms of different terminal equipment for compiling the test instruction. As shown in fig. 6, 03 to 05 and 12 indicate that there is no difference in displaying of the pages of the application software on the terminal devices of different platforms, and test instructions can be uniformly written according to preset natural language syntax for the Android platform and the iOS platform.
Corresponding to the cross-platform user interface automated testing method described in fig. 1 to fig. 6, based on the same technical concept, an embodiment of the present application further provides a cross-platform user interface automated testing apparatus, where the cross-platform user interface automated testing apparatus may be installed in a general terminal device, such as a desktop computer, a notebook computer, a tablet computer, a mobile phone, a wearable device, and the like, or may be installed in a dedicated testing device, which is not limited in this application embodiment, and fig. 7 is a schematic diagram of a composition structure of the cross-platform user interface automated testing apparatus according to the embodiment of the present application, where the cross-platform user interface automated testing apparatus is configured to execute the cross-platform user interface automated testing method described in fig. 1 to fig. 6, and as shown in fig. 7, the cross-platform user interface automated testing apparatus at least includes: the analysis module 710, the recognition module 720 and the processing module 730, and the analysis module 710, the recognition module 720 and the processing module 730 are connected in sequence. Wherein the content of the first and second substances,
the analysis module 710 is configured to analyze a test instruction of a current page of application software displayed on the terminal device according to a preset rule to obtain a target object to be tested and a test operation performed on the target object; the test instructions are written according to preset natural language grammars, and the preset natural language grammars for writing the test instructions are different under the condition that the current page of the application software is different in display on terminal equipment of different platforms.
In the embodiment of the present application, the description of the parsing module 710 can refer to the description of S102 in fig. 1, and therefore, the description is not repeated here.
The identifying module 720 is configured to obtain a screenshot of a current page of the application software displayed on the terminal device, and identify the screenshot of the current page of the application software to obtain location information of a target object in the current page of the application software.
In the embodiment of the present application, the description of the identification module 720 can refer to the description of S104 in fig. 1, and therefore, the description will not be repeated here.
The processing module 730 is configured to perform a test operation on the target object according to the position information of the target object, so as to obtain a result of the automated test on the user interface.
In the embodiment of the present application, the description of the processing module 730 can refer to the description of S106 in fig. 1, and therefore, the description is not repeated here.
According to the cross-platform user interface automatic testing device, a testing instruction of a current page of application software displayed on the terminal equipment is analyzed according to a preset rule, and a tested target object and a testing operation performed on the target object are obtained; acquiring a screenshot of a current page of application software displayed on terminal equipment, and identifying the screenshot of the current page of the application software to obtain position information of a target object in the current page of the application software; testing the target object according to the position information of the target object to obtain a result of the automatic test of the user interface; the testing instructions are written according to preset natural language grammars, and under the condition that the display of the current page of the application software on terminal equipment of different platforms is different, the preset natural language grammars for writing the testing instructions are different; by utilizing a machine learning technology and a natural language processing technology in the AI technology, the position of a tested target object can be accurately determined in a page by identifying a screenshot of a page of application software displayed on a terminal device, the target object can be accurately operated, a test instruction is compiled through a preset natural language grammar, the programming of the user interface automatic test can be simplified, the strong association relationship between an installation platform of the application software and a test program is decoupled, the user interface test that the same test program is adaptive to different platforms of application software is realized, the test program is maximally reused, the user interface automatic test has a cross-platform capability, the efficiency of compiling the user interface automatic test program is improved, and the cost of maintaining the user interface automatic test program is effectively reduced.
Alternatively, as shown in fig. 8, fig. 8 is a schematic structural diagram of another component of the cross-platform ui automatic testing device according to the embodiment of the present application, the cross-platform user interface automatic testing device comprises a matching module 740 besides a parsing module 710, a recognition module 720 and a processing module 730, wherein the matching module 740 is connected with the parsing module 710, in the case of differences in the display of the current page of the application software on terminal devices of different platforms, analyzing the test instruction of the current page of the application software displayed on the terminal equipment according to a preset rule, acquiring platform type information of the terminal equipment before a tested target object and a test operation performed on the target object are obtained, and selecting the test instruction of the current page of the application software displayed on the terminal equipment matched with the platform type information.
Optionally, the parsing module 710 is configured to split a test instruction of a current page of application software displayed on the terminal device according to a preset natural language grammar rule to obtain a target object to be tested and a test operation performed on the target object; and converting the tested target object and the test operation performed on the target object into a machine executable language according to a preset comparison table of the natural language and the machine language.
Optionally, the identifying module 720 is configured to intercept a screenshot of a current page of the application software displayed on the terminal device; performing character recognition processing and/or image recognition processing on the screenshot of the current page of the application software; and determining the position information of the target object in the current page of the application software according to the result of the character recognition processing and/or the result of the image recognition processing.
Optionally, the recognition module 720 is configured to perform optical character recognition processing on the text in the screenshot of the current page of the application software; and/or carrying out general icon identification processing on the icon in the screenshot of the current page of the application software; and/or performing self-defined image matching processing on the image in the screenshot of the current page of the application software.
Optionally, the test operation performed on the target object includes one of: click operation, slide operation, input operation, check operation, judgment operation, assignment operation and cycle operation.
Optionally, the platform of the terminal device includes one of: android platform, iOS platform, Windows platform and Linux platform.
Corresponding to the cross-platform user interface automated testing method described in fig. 1 to fig. 6, based on the same technical concept, an embodiment of the present application further provides an electronic device, where the electronic device includes: a processor, a communication interface, a memory, and a communication bus; the processor, the communication interface and the memory complete mutual communication through a bus; a memory for storing a computer program; a processor for executing the program stored in the memory, implementing the method of:
analyzing a test instruction of a current page of application software displayed on the terminal equipment according to a preset rule to obtain a target object to be tested and test operation performed on the target object; the testing instructions are written according to preset natural language grammars, and under the condition that the display of the current page of the application software on terminal equipment of different platforms is different, the preset natural language grammars for writing the testing instructions are different;
acquiring a screenshot of a current page of application software displayed on terminal equipment, and identifying the screenshot of the current page of the application software to obtain position information of a target object in the current page of the application software;
and carrying out test operation on the target object according to the position information of the target object to obtain a result of the automatic test of the user interface.
According to the electronic equipment, a test target object and test operation performed on the target object are obtained by analyzing a test instruction of a current page of application software displayed on terminal equipment according to a preset rule; acquiring a screenshot of a current page of application software displayed on terminal equipment, and identifying the screenshot of the current page of the application software to obtain position information of a target object in the current page of the application software; testing the target object according to the position information of the target object to obtain a result of the automatic test of the user interface; the testing instructions are written according to preset natural language grammars, and under the condition that the display of the current page of the application software on terminal equipment of different platforms is different, the preset natural language grammars for writing the testing instructions are different; by utilizing a machine learning technology and a natural language processing technology in the AI technology, the position of a tested target object can be accurately determined in a page by identifying a screenshot of a page of application software displayed on a terminal device, the target object can be accurately operated, a test instruction is compiled through a preset natural language grammar, the programming of the user interface automatic test can be simplified, the strong association relationship between an installation platform of the application software and a test program is decoupled, the user interface test that the same test program is adaptive to different platforms of application software is realized, the test program is maximally reused, the user interface automatic test has a cross-platform capability, the efficiency of compiling the user interface automatic test program is improved, and the cost of maintaining the user interface automatic test program is effectively reduced.
Based on the same technical concept, the embodiment of the present application further provides a computer-readable storage medium storing a computer program, where the computer program, when executed by a processor, implements the following processes:
analyzing a test instruction of a current page of application software displayed on the terminal equipment according to a preset rule to obtain a target object to be tested and test operation performed on the target object; the testing instructions are written according to preset natural language grammars, and under the condition that the display of the current page of the application software on terminal equipment of different platforms is different, the preset natural language grammars for writing the testing instructions are different;
acquiring a screenshot of a current page of application software displayed on terminal equipment, and identifying the screenshot of the current page of the application software to obtain position information of a target object in the current page of the application software;
and carrying out test operation on the target object according to the position information of the target object to obtain a result of the automatic test of the user interface.
According to the computer-readable storage medium of the embodiment of the application, a test instruction of a current page of application software displayed on terminal equipment is analyzed according to a preset rule, so that a tested target object and test operation performed on the target object are obtained; acquiring a screenshot of a current page of application software displayed on terminal equipment, and identifying the screenshot of the current page of the application software to obtain position information of a target object in the current page of the application software; testing the target object according to the position information of the target object to obtain a result of the automatic test of the user interface; the testing instructions are written according to preset natural language grammars, and under the condition that the display of the current page of the application software on terminal equipment of different platforms is different, the preset natural language grammars for writing the testing instructions are different; by utilizing a machine learning technology and a natural language processing technology in the AI technology, the position of a tested target object can be accurately determined in a page by identifying a screenshot of a page of application software displayed on a terminal device, the target object can be accurately operated, a test instruction is compiled through a preset natural language grammar, the programming of the user interface automatic test can be simplified, the strong association relationship between an installation platform of the application software and a test program is decoupled, the user interface test that the same test program is adaptive to different platforms of application software is realized, the test program is maximally reused, the user interface automatic test has a cross-platform capability, the efficiency of compiling the user interface automatic test program is improved, and the cost of maintaining the user interface automatic test program is effectively reduced.
It should also be noted that the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.
One or more embodiments of the present application may be described in the general context of computer-executable instructions, such as program modules, being executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform particular tasks or implement particular abstract data types. One or more embodiments of the present application may also be practiced in distributed computing environments where tasks are performed by remote processing devices that are linked through a communications network. In a distributed computing environment, program modules may be located in both local and remote computer storage media including memory storage devices.
The embodiments in the present application are described in a progressive manner, and the same and similar parts among the embodiments can be referred to each other, and each embodiment focuses on the differences from the other embodiments. In particular, for the system embodiment, since it is substantially similar to the method embodiment, the description is simple, and for the relevant points, reference may be made to the partial description of the method embodiment.
The above description is only an example of this document and is not intended to limit this document. Various modifications and changes may occur to those skilled in the art from this document. Any modifications, equivalents, improvements, etc. which come within the spirit and principle of the disclosure are intended to be included within the scope of the claims of this document.
Claims (10)
1. A cross-platform user interface automated testing method is characterized by comprising the following steps:
splitting a test instruction of a current page of application software displayed on terminal equipment according to a preset natural language grammar rule to obtain a tested target object and test operation performed on the target object; the test instruction is written according to preset natural language grammar, and under the condition that the display of the current page of the application software on terminal equipment of different platforms is different, the preset natural language grammar for writing the test instruction is different;
converting the test operation performed on the target object into a machine executable language according to a preset natural language and machine language comparison table;
acquiring a screenshot of the current page of the application software displayed on the terminal equipment, and identifying and processing the screenshot of the current page of the application software to obtain position information of the target object in the current page of the application software;
and testing the target object according to the position information of the target object and the test operation converted into the machine executable language to obtain a result of the automatic test of the user interface.
2. The automated testing method according to claim 1, wherein when the current page of the application software is displayed differently on the terminal devices of different platforms, the analyzing process is performed on the test instruction of the current page of the application software displayed on the terminal device according to a preset rule, and before obtaining a target object to be tested and performing a test operation on the target object, the method further comprises:
and acquiring the platform type information of the terminal equipment, and selecting a test instruction of the current page of the application software displayed on the terminal equipment matched with the platform type information.
3. The automated testing method of claim 2, further comprising:
and converting the tested target object into a machine executable language according to a preset comparison table of the natural language and the machine language.
4. The automated testing method according to claim 3, wherein the obtaining of the screenshot of the current page of the application software displayed on the terminal device, and the identifying of the screenshot of the current page of the application software to obtain the location information of the target object in the current page of the application software comprises:
intercepting a screenshot of a current page of the application software displayed on the terminal equipment;
performing character recognition processing and/or image recognition processing on the screenshot of the current page of the application software;
and determining the position information of the target object in the current page of the application software according to the result of the character recognition processing and/or the result of the image recognition processing.
5. The automated testing method of claim 4, wherein said performing a text recognition process and/or an image recognition process on the screenshot of the current page of the application software comprises:
carrying out optical character recognition processing on characters in the screenshot of the current page of the application software; and/or carrying out general icon identification processing on the icon in the screenshot of the current page of the application software; and/or performing self-defined image matching processing on the image in the screenshot of the current page of the application software.
6. The automated testing method of any of claims 1 to 5, wherein the testing operation performed on the target object comprises one of: click operation, slide operation, input operation, check operation, judgment operation, assignment operation and cycle operation.
7. The automated testing method of claim 6, wherein the platform of the terminal device comprises one of: android platform, iOS platform, Windows platform and Linux platform.
8. A cross-platform user interface automated testing apparatus, comprising:
the analysis module is used for splitting a test instruction of a current page of the application software displayed on the terminal equipment according to a preset natural language grammar rule to obtain a tested target object and test operation performed on the target object; the test instruction is written according to preset natural language grammar, and under the condition that the display of the current page of the application software on terminal equipment of different platforms is different, the preset natural language grammar for writing the test instruction is different;
the conversion module is used for converting the test operation performed on the target object into a machine executable language according to a preset natural language and machine language comparison table;
the identification module is used for acquiring a screenshot of the current page of the application software displayed on the terminal equipment, and identifying the screenshot of the current page of the application software to obtain the position information of the target object in the current page of the application software;
and the processing module is used for testing the target object according to the position information of the target object and the test operation converted into the machine executable language to obtain a result of the automatic test of the user interface.
9. An electronic device comprising a processor, a communication interface, a memory, and a communication bus; the processor, the communication interface and the memory complete mutual communication through a bus; the memory is used for storing a computer program; the processor is used for executing the program stored in the memory to realize the cross-platform user interface automatic testing method according to any one of claims 1 to 7.
10. A computer-readable storage medium, having stored thereon a computer program which, when executed by a processor, implements a cross-platform user interface automated testing method according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011431464.XA CN112231234B (en) | 2020-12-10 | 2020-12-10 | Cross-platform user interface automatic testing method, device, equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011431464.XA CN112231234B (en) | 2020-12-10 | 2020-12-10 | Cross-platform user interface automatic testing method, device, equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112231234A CN112231234A (en) | 2021-01-15 |
| CN112231234B true CN112231234B (en) | 2021-03-19 |
Family
ID=74124685
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011431464.XA Active CN112231234B (en) | 2020-12-10 | 2020-12-10 | Cross-platform user interface automatic testing method, device, equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112231234B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112506778A (en) * | 2020-12-10 | 2021-03-16 | 北京云测信息技术有限公司 | WEB user interface automatic test method, device, equipment and storage medium |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103150255A (en) * | 2013-03-29 | 2013-06-12 | 北京经纬恒润科技有限公司 | Method and device for testing script |
| CN108845931A (en) * | 2018-05-24 | 2018-11-20 | 吉旗（成都）科技有限公司 | A kind of automated test frame based on natural language |
| CN109144857A (en) * | 2018-08-01 | 2019-01-04 | 武汉普利商用机器有限公司 | Cross-platform test of heuristics method and system |
| CN111290955A (en) * | 2020-02-18 | 2020-06-16 | 卡斯柯信号有限公司 | Non-invasive automatic test system for subway signal system test software |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102855184B (en) * | 2012-07-24 | 2015-06-10 | 播思通讯技术（北京）有限公司 | Android automatic test cross application device and method |
| US9135150B2 (en) * | 2013-02-27 | 2015-09-15 | International Business Machines Corporation | Automated execution of functional test scripts on a remote system within a unit testing framework |
-
2020
- 2020-12-10 CN CN202011431464.XA patent/CN112231234B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103150255A (en) * | 2013-03-29 | 2013-06-12 | 北京经纬恒润科技有限公司 | Method and device for testing script |
| CN108845931A (en) * | 2018-05-24 | 2018-11-20 | 吉旗（成都）科技有限公司 | A kind of automated test frame based on natural language |
| CN109144857A (en) * | 2018-08-01 | 2019-01-04 | 武汉普利商用机器有限公司 | Cross-platform test of heuristics method and system |
| CN111290955A (en) * | 2020-02-18 | 2020-06-16 | 卡斯柯信号有限公司 | Non-invasive automatic test system for subway signal system test software |
Non-Patent Citations (1)
| Title |
|---|
| "AI在自动化测试领域的应用";fin测试开发;《https://www.cnblogs.com/finer/p/12313917.html》;20191229;第1-5页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112231234A (en) | 2021-01-15 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109542399B (en) | Software development method and device, terminal equipment and computer readable storage medium | |
| US9846634B2 (en) | Visual graphical user interface verification | |
| CN110765013B (en) | Automatic execution method and system for flow | |
| US20130117855A1 (en) | Apparatus for automatically inspecting security of applications and method thereof | |
| CN104881364A (en) | Automated testing method and system applied to Android system | |
| CN108415826B (en) | Application testing method, terminal device and computer readable storage medium | |
| CN106294134A (en) | The collapse localization method of code and device | |
| CN109828906B (en) | UI (user interface) automatic testing method and device, electronic equipment and storage medium | |
| CN103312850A (en) | Mobile phone automation testing system and work method | |
| CN112506778A (en) | WEB user interface automatic test method, device, equipment and storage medium | |
| CN111797026A (en) | Test case generation method and device, computer equipment and storage medium | |
| CN111367802B (en) | Web automatic test method, device and computer readable storage medium | |
| CN112231234B (en) | Cross-platform user interface automatic testing method, device, equipment and storage medium | |
| CN108733555A (en) | A kind of application testing method and device | |
| CN112231206A (en) | Script editing method for application program test, computer readable storage medium and test platform | |
| CN111475237A (en) | Menu processing method and device, electronic equipment and storage medium | |
| CN113887442A (en) | OCR training data generation method, device, equipment and medium | |
| CN107783867B (en) | Gesture testing method and device, computer equipment and storage medium | |
| CN113705224A (en) | Voice recognition scheduling service voice interaction method and system | |
| CN112363919A (en) | Automatic test method, device, equipment and storage medium for user interface AI | |
| US11954507B2 (en) | GUI component recognition apparatus, method and program | |
| CN116719736A (en) | Test case generation method and device for testing software interface | |
| CN107766228A (en) | It is a kind of based on multilingual automated testing method and device | |
| CN108132799B (en) | Inter-process static program analysis information extraction method, device and equipment | |
| CN110727436A (en) | Operation interface script execution method and device, terminal equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |