CN112214397A - User interface testing method and device, computer equipment and storage medium - Google Patents
User interface testing method and device, computer equipment and storage medium Download PDFInfo
- Publication number
- CN112214397A CN112214397A CN202010939537.XA CN202010939537A CN112214397A CN 112214397 A CN112214397 A CN 112214397A CN 202010939537 A CN202010939537 A CN 202010939537A CN 112214397 A CN112214397 A CN 112214397A
- Authority
- CN
- China
- Prior art keywords
- test
- user interface
- definition
- testing
- acquiring
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/36—Preventing errors by testing or debugging software
- G06F11/3668—Software testing
- G06F11/3672—Test management
- G06F11/3688—Test management for test execution, e.g. scheduling of test suites
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Computer Hardware Design (AREA)
- Quality & Reliability (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- User Interface Of Digital Computer (AREA)
Abstract
The application relates to a user interface testing method, a user interface testing device, computer equipment and a storage medium. The method comprises the following steps: acquiring test identification information and user interface information uploaded by a test terminal aiming at a tested user interface; acquiring a test step definition of a test step to be executed according to the test identification information; identifying user interface information according to the test step definition, and acquiring operation object information corresponding to the test step when the condition of entering the test step is determined to be met; and returning the operation object information corresponding to the testing step to the testing terminal, and executing the testing operation on the operation object in the user interface by the testing terminal according to the operation object information. The method can achieve the purposes of horizontal expansion, massive user support, high concurrency and different applications without developing a set of automation tools for different services respectively.
Description
Technical Field
The present application relates to the field of testing technologies, and in particular, to a method and an apparatus for testing a user interface, a computer device, and a storage medium.
Background
With the development of intelligent technology, people can not leave application programs more and more. For an application, in the testing phase, a user interface test needs to be performed. The main contents of the user interface test include: whether the layout of the functional modules of the user interface is reasonable or not, whether the overall style is consistent or not and whether the placement positions of the controls accord with the use habits of customers or not are tested, and more importantly, whether the operation is convenient or fast, the navigation is simple and understandable, whether characters in the interface are correct or not, whether the naming is uniform or not, whether the page is attractive or not, whether the combination of the characters and the pictures is perfect or not and the like are met.
In order to improve the testing effect, the automated testing of the user interface is a development direction. One way to implement automated testing is to use a test development kit (SDK): some general functions are abstracted and packed to form a function library or a development kit, and a user can flexibly call corresponding functions to complete test work. The form sdk indeed makes it more flexible and faster to develop scripts for test development on the client side, but on the other hand, the invocation of related functions is still focused on a stand-alone environment. When we want to perform a large-scale test, such as performing a synchronous test on the mobile phones of TOP 300, and each mobile phone runs 10 different apps, a test script that needs to support 3000 devices is executed, and once a problem is encountered in the script, it is difficult to locate the mobile phone and it is impossible to monitor it remotely. In fact, the automated testing that relies on the single-machine script execution cannot support the concurrent deployment and debugging of thousands of mobile phones, and either only very simple processes (pull-up app, close) can be performed, or only batch-wise completed.
Therefore, the traditional user interface automatic test has the technical problem that the high concurrency requirement cannot be met.
Disclosure of Invention
In view of the above, there is a need to provide a user interface testing method, apparatus, computer device and storage medium capable of meeting high concurrency requirements.
A user interface testing method, the method comprising:
acquiring test identification information and user interface information uploaded by a test terminal aiming at a tested user interface;
acquiring a test step definition of a test step to be executed according to the test identification information;
identifying the user interface information according to the test step definition, and acquiring the operation object information corresponding to the test step when determining that the condition for entering the test step is met;
and returning the operation object information corresponding to the test step to the test terminal, and executing test operation on the operation object in the user interface by the test terminal according to the operation object information.
A user interface testing apparatus, the apparatus comprising:
the information acquisition module is used for acquiring test identification information and user interface information uploaded by the test terminal aiming at a tested user interface;
the definition acquisition module is used for acquiring a test step definition of a test step to be executed according to the test identification information;
the analysis module is used for identifying the user interface information according to the test step definition, and acquiring the operation object information corresponding to the test step when the condition of entering the test step is determined to be met;
and the feedback module is used for returning the operation object information corresponding to the testing step to the testing terminal, and the testing terminal executes the testing operation on the operation object in the user interface according to the operation object information.
A computer device comprising a memory and a processor, the memory storing a computer program, the processor implementing the following steps when executing the computer program:
acquiring test identification information and user interface information uploaded by a test terminal aiming at a tested user interface;
acquiring a test step definition of a test step to be executed according to the test identification information;
identifying the user interface information according to the test step definition, and acquiring the operation object information corresponding to the test step when determining that the condition for entering the test step is met;
and returning the operation object information corresponding to the test step to the test terminal, and executing test operation on the operation object in the user interface by the test terminal according to the operation object information.
A computer-readable storage medium, having stored thereon a computer program which, when executed by a processor, performs the steps of:
acquiring test identification information and user interface information uploaded by a test terminal aiming at a tested user interface;
acquiring a test step definition of a test step to be executed according to the test identification information;
identifying the user interface information according to the test step definition, and acquiring the operation object information corresponding to the test step when determining that the condition for entering the test step is met;
and returning the operation object information corresponding to the test step to the test terminal, and executing test operation on the operation object in the user interface by the test terminal according to the operation object information.
According to the user interface testing method, the user interface testing device, the computer equipment and the storage medium, when the UI automation test is executed, the testing terminal only needs to execute lightweight operation, the lightweight operation comprises uploading of test identification information and user interface information, testing operation is carried out according to operation object information returned by the server, and logic judgment processing is achieved by the back-end server. Because the business logic and judgment are completed by the back end, the front end is only responsible for receiving instructions and operations, and does not perform business logic related judgment, the UI automation service can bear the automation requirements of multiple concurrent and different business scenes, and a set of automation tools do not need to be developed for different businesses respectively.
Drawings
FIG. 1 is a diagram of an application environment of a user interface testing method in one embodiment;
FIG. 2 is a flowchart illustrating a method for testing a user interface according to one embodiment;
FIG. 3 is an interaction diagram of user interface testing in one embodiment;
FIG. 4 is an interface diagram of a marking tool in accordance with another embodiment;
FIG. 5 is a flow diagram illustrating a preview of a test process in one embodiment;
FIG. 6 is an architectural diagram illustrating user interface testing in one embodiment;
FIG. 7 is a block diagram of the user interface test apparatus in one embodiment;
FIG. 8 is a diagram illustrating an internal structure of a computer device according to an embodiment.
Detailed Description
In order to make the objects, technical solutions and advantages of the present application more apparent, the present application is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the present application and are not intended to limit the present application.
The user interface testing method provided by the application can be applied to the application environment shown in fig. 1. Wherein the test terminal 102 communicates with the server 104 over a network. Firstly, a client side edits the test content of a user interface by using an automatic marking tool to generate a test definition of a test flow of a product, and stores the test definition of the test flow of the product in a database. In the testing process, the testing terminal uploads the testing identification information and the user interface information to the server, and the server acquires the testing identification information and the user interface information uploaded by the testing terminal aiming at the user interface under testing; acquiring a test step definition of a test step to be executed according to the test identification information; when the user interface information is identified to meet the condition of entering the testing step according to the testing step definition, acquiring operation object information corresponding to the testing step; and returning the operation object information in the testing step to the testing terminal, and executing the testing operation on the operation object in the user interface by the testing terminal according to the operation object information.
The server may be an independent physical server, a server cluster or a distributed system formed by a plurality of physical servers, or a cloud server providing basic cloud computing services such as cloud service, a cloud database, cloud computing, cloud function, cloud storage, network service, cloud communication, middleware service, domain name service, security service, CDN, big data and artificial intelligence platform. The test terminal 102 may be, but is not limited to, a smart phone, a tablet computer, a laptop computer, a desktop computer, a smart speaker, a smart watch, and the like. The terminal and the server may be directly or indirectly connected through wired or wireless communication, and the application is not limited herein.
Cloud computing (cloud computing) is a computing model that distributes computing tasks over a pool of resources formed by a large number of computers, enabling various application systems to obtain computing power, storage space, and information services as needed. The network that provides the resources is referred to as the "cloud". Resources in the "cloud" appear to the user as being infinitely expandable and available at any time, available on demand, expandable at any time, and paid for on-demand.
As a basic capability provider of cloud computing, a cloud computing resource pool (called as an IaaS (Infrastructure as a Service) platform for short is established, and multiple types of virtual resources are deployed in the resource pool and are selectively used by external clients.
In one embodiment, as shown in fig. 2 and 3, a user interface testing method is provided, which is described by taking the method as an example applied to the server in fig. 1, and includes the following steps:
s202, test identification information and user interface information uploaded by the test terminal aiming at the user interface under test are obtained.
The test identification information refers to information that can be used to distinguish a test object, and generally includes a product identifier and a test procedure identifier. The product identification is used for distinguishing the test products, and the test flow identification is used for distinguishing different test flows of one test product. The product identifier may be a product ID, and the test procedure identifier may be a test procedure ID. And determining the test flow of the test product corresponding to the test task executed by the test terminal according to the test identification information. It is understood that a test product has multiple test flows.
The user interface information is information of a user interface to be tested, and interface elements can be acquired according to the user interface information. A user interface test is a test performed on an interface element.
Specifically, the test terminal runs a test product and performs a test according to a test flow of the test product. In the testing process, aiming at the user interface under test, the test identification information and the user interface information are sent to the server, the server identifies the user interface and returns the operation object which is implemented on the user interface according to the testing process, so that the testing terminal can implement the testing operation on the operation object.
And S204, acquiring a test step definition of the test step to be executed according to the test identification information.
In which a product has multiple test flows, and each test flow may have multiple test steps. For each test flow of the product, the test terminal marks the test flow of one user interface by using a marking tool for marking the UI automatic test flow. And after the marking is finished, generating a test flow definition of the test flow of the product.
The test terminal utilizes a marking tool of the UI automatic test flow to mark the test flow of a user interface, then generates the test definition of the test flow, submits the test flow information, the marked user interface information and the test definition to the server, and the server stores and establishes the corresponding relation of the test flow, the test definition and the marked user interface information of the product. And when the test identification information and the user interface information submitted by the test terminal are acquired, acquiring the pre-stored corresponding test definition and labeled user interface information according to the test identification information.
The test flow definition comprises a test path, test description condition judgment, test description circulation logic, test operation logic and the like. The test definition is data automatically generated according to the labeled content of the automatic test process labeled by the labeling tool, and the data format is the JSON format. A test flow definition includes the test step definitions of each test step constituting the test flow, and the test step definitions are the test contents of a test step, the judgment of test description conditions, the test description loop logic, the test operation logic, and the like.
And S206, identifying the user interface information according to the test step definition, and acquiring the operation object information corresponding to the test step when the condition of entering the test step is determined to be met.
Specifically, the test step definition includes a test condition that defines the test step in advance, and the test condition is used to determine whether a condition for entering the test step is satisfied. The test definition has the attribute of the test condition, identifies the key words of the attribute of the test condition and obtains the test condition.
Specifically, the test condition is used to determine whether there is an interface element in the user interface information that is required to perform the test step. Taking the user interface information as an image for example, the test conditions may include the following attributes: original image source, matching type and matching parameters. The original image source is the marked user interface picture in the testing step, namely the marked image. The matching parameters may further define attributes such as location, color, etc. of the interface elements. The matching type may be image matching, specifically, the matching degree between the user interface picture and the labeled user interface picture in the test step. Accordingly, the matching parameter may further define a threshold matching degree. The matching parameters may further define a matching waiting time, i.e. a longest time for waiting for the user interface picture to match the marked user interface picture of the testing step, avoiding an excessive waiting, and returning a result of the mismatch if the longest time is exceeded.
For example, one test condition defines the original image source, the match type, and the match parameters. Wherein, the original image source is an annotated original image. The matching type is image matching, the matching parameters comprise a matching image to be matched, a matching degree threshold value of 0.7, the matching waiting time is 60 seconds, the frame position of a matching element on a local picture (calculated according to the height and width ratio of the picture), the matching degree of an original image, namely a marked image and the image to be matched is firstly evaluated according to the matching parameters, and if the matching degree threshold value of 0.7 is met, whether an interface element required by testing exists at the corresponding position of the image to be matched is further judged. If yes, matching is successful, and it is determined that the user interface information meets the test conditions predefined in the test definition.
And if so, acquiring the operation object information from the test step definition.
The operation object information comprises a test operation object and operation contents for the user interface. Specifically, after the operation object and the operation content in the user interface are marked by the marking tool, the generated test definition includes the test operation logic. The test operation logic is specifically used for carrying out test operation on a test object of the user interface. The operand information is obtained by parsing the test operation logic from the test step definition.
The operation object is an interface element in the user interface, such as a button, a navigation bar, and the like. The operation content is the content of test behavior which the test terminal needs to implement on the object at the time of testing. The operation content may be an operation action or data processing. The operation action includes, but is not limited to, clicking on a specified element, entering an input at a specified element position, commanding the client to temporarily not operate, and requiring the client to exit the program. Data processing includes, but is not limited to, performing numerical comparisons, performing alphanumeric recognition on pictures, calling sub-processes, storing data in the form of key-values, performing numerical subtraction, performing numerical addition, performing numerical multiplication, and performing numerical division.
And S208, returning the operation object information corresponding to the test step to the test terminal, and executing the test operation on the operation object in the user interface by the test terminal according to the operation object information.
In this embodiment, the test terminal sends the user interface information and the test identification information to the server for the user interface to be tested, and the server obtains the operation object information after parsing and identifying and returns the operation object information to the test terminal. And the test terminal executes test operation on the operation object in the user interface according to the operation object information. In other words, in this embodiment, when the UI automation test is executed, the test terminal only needs to execute a lightweight operation, including uploading test identification information and user interface information, and the operation object information returned by the server performs the test operation, that is, the logical judgment processing is implemented by the back-end server, and by adopting the lightweight client execution + distributed back-end system, the purposes of horizontal extension, support of a large number of users, high concurrency, and different applications are achieved. The front end is separated from the back end, the client side realizes operation according to self requirements, the back end is only responsible for realizing automatic logic, therefore, the back end automatic logic realizes state description and switching based on a finite automaton, the back end service is stateless, context information interaction is carried out by depending on an external data storage system, all service logic and judgment are completed by the back end, and the front end is only responsible for receiving instructions and operation and does not carry out service logic related judgment.
According to the user interface testing method, when the UI automatic test is executed, the testing terminal only needs to execute the lightweight operation, the lightweight operation comprises uploading the test identification information and the user interface information, the test operation is carried out on the object information according to the operation returned by the server, and the logic judgment processing is realized by the back-end server. Because the business logic and judgment are completed by the back end, the front end is only responsible for receiving instructions and operations, and does not perform business logic related judgment, the UI automation service can bear the automation requirements of multiple concurrent and different business scenes, and a set of automation tools do not need to be developed for different businesses respectively.
The method comprises the following steps of defining and identifying user interface information according to a test step, and acquiring operation object information corresponding to the test step when determining that conditions for entering the test step are met, wherein the method comprises the following steps: acquiring test conditions which are predefined in the test step definition and enter the test step; identifying whether the user interface information satisfies the test condition; and if so, acquiring the operation object information corresponding to the test step.
In the definition of the test step, the condition which needs to be met when entering a certain test step is designed by defining the attribute of the test condition, thereby realizing the judgment of the switching condition of the test step. By defining the logical operation attribute of the server, simple logical judgment and data access of the back end are realized, and mathematical operation and judgment of multiple steps can be established, so that more complicated condition judgment can be realized. Through condition judgment and operation information acquisition, logic judgment of the test step can be realized, and when the condition of entering the test step is met, the operation object information of the test step is acquired.
Specifically, the user interface information includes a user interface picture, and the identifying whether the user interface information satisfies the test condition includes: and identifying the user interface picture according to the test condition, and identifying whether the user interface picture is matched with the test condition.
Specifically, the test step definition includes a predefined test condition, and the test condition is used to determine whether a condition for entering the test step is satisfied. The attribute of the test condition exists in the test step definition, and the test condition is obtained by identifying the key words of the attribute of the test condition.
Specifically, the test condition is used to determine whether there is an interface element in the user interface information that is required to perform the test step. Taking the user interface information as an image for example, the test conditions may include the following attributes: original image source, matching type and matching parameters. The matching parameters can further define attributes such as positions, colors and the like of the interface elements. The obtained test condition is specifically an interface element which is defined by the obtained test condition.
Specifically, identifying the user interface picture according to the test condition, and identifying whether the user interface picture is matched with the test condition includes: identifying whether the user interface picture is consistent with the corresponding marked picture in the testing step; and if so, identifying whether the user interface picture has interface elements appointed by the test conditions, and if so, determining that the user interface picture is matched with the predefined test conditions.
And judging whether the user interface picture is consistent with the marked picture in the step, identifying by using an identification model which uses deep learning training in advance, and detecting the picture target by using a traditional visual algorithm. After the test flow is marked by the marking tool, the marking picture is uploaded to a third-party service, and the deep learning training recognition model provided by the third-party service is used.
The method comprises the steps of obtaining interface elements defined by predefined test conditions, and identifying whether the defined interface elements exist in a user interface picture. Specifically, a test condition defines a matching parameter for positioning a test interface element required by the annotation picture. The interface elements may be buttons, navigation bars, pictures or texts in the user interface, and the like.
In another embodiment, the user interface information includes a control attribute of the user interface, and the identifying whether the user interface information satisfies the test condition includes: and judging whether the control attribute of the user interface has the control specified by the test condition.
Specifically, the attribute of the interface element defined by the predefined test condition is obtained, whether the control attribute of the user interface meets the attribute of the interface element defined by the test condition is judged, if yes, the control attribute of the user interface is consistent with the attribute of the interface element marked with the test requirement, and the user interface picture is determined to be matched with the predefined test condition.
In another embodiment, after obtaining the operation object information corresponding to the testing step, the method further includes: marking the identification mark of the next round of step to be tested according to the testing path of the testing process;
obtaining a test step definition of a test step to be executed according to the test identification information, wherein the test step definition comprises the following steps: acquiring a test flow definition of the current test flow according to the test identification information; and acquiring a test step definition corresponding to the identification mark of the step to be tested of the previous round of test mark according to the test path defined by the test flow.
Specifically, a test flow is composed of a plurality of test steps, a similar linked list is defined in the test flow, the similar linked list comprises an identification (such as an ID) of a current step, defined related test content and an identification of a next step, and if the identification of the next step is undefined, the test flow is ended. The flow definition can be firstly stored in the cache after the initial reading, so that the reading at the next time is convenient.
The test step definition also includes a field of the next round of step to be tested, and the identification mark of the next step of the current step is obtained and marked by identifying a keyword (such as expectedStates) corresponding to the field. Meanwhile, the test definition also comprises a quit field and a waiting time field. And the waiting time field is used for marking whether the process needs to be exited after the state is reached and entering the next state to judge the required waiting time.
Specifically, the test flow defines a complex UI flow including condition judgment and loop control. The UI process is defined as a Finite State Machine (Finite State Machine), and the data description in the JSON format generates a structure in the form of a directed graph, so that the process recovery of the State Machine can be completed.
The test flow definition comprises a test step definition of a plurality of test steps, a unique ID of each small test step is defined, and a test list (a plurality of test steps can be provided) of the next test step defines a complete Finite automaton (Finite State Machine), so that the definition of any linear path can be met.
And marking the identification mark of the next round of step to be tested according to the testing path of the testing process after the operation object information corresponding to the testing step is obtained. When the test of the next step is started, acquiring the test flow definition of the current test flow according to the test identification information; and acquiring a test step definition corresponding to the identification mark of the step to be tested of the previous round of test mark according to the test path defined by the test flow.
In this embodiment, through the attributes such as condition judgment and test path, the cyclic logic of the test flow can be effectively realized.
In one embodiment, the user interface testing method further includes a labeling process, specifically, the labeling process includes: acquiring the labeling content of a user interface of each testing step in the product testing process; the marked content is marked by a marking tool through a test terminal; generating a test definition of a test flow of the product according to the marked content; a test definition of a test flow of the product is stored.
The automatic labeling tool is only responsible for the interface labeling function, and the labeling interface of one labeling tool is shown in fig. 4 and comprises a plurality of panels, such as an equipment display and equipment operation window, a scene (UIState) management window, a scene relation diagram display window, a scene editing window, a scene data object display (a labeled object and an operation object), a target scene attribute editing window (part of scene attributes), an expected scene display window of a target scene and the like. By importing the user interface picture into a marking tool, setting attributes such as an operation object identifier, an operation type, an execution sequence and the like in a target scene attribute editing window and a target scene object attribute editing window.
After the labeling is finished, the test flow can be pre-reviewed in a scene relation diagram display window of a labeling interface, one test flow is shown in fig. 5, and the test flow shown in fig. 5 is a test flow entering a game fighting interface in a game product. The test flow comprises three test steps, and each test step identifies one test interface. The preview of the whole process does not need to execute UI automation, but only needs to analyze the JSON data file generated above and draw by using a tool.
And after the labeling is finished, acquiring the labeling content, generating the definition of the product testing process according to the labeling content, and correspondingly storing the testing process definition and the testing identification information.
Taking the test flow of fig. 5 as an example, a user interface test method is described, first obtaining a product ID, a test flow ID and a user interface picture uploaded by a test terminal, obtaining a test flow definition according to the product ID and the test flow ID, and obtaining a test step definition of a test step to be executed according to the test flow definition.
Specifically, a first test step definition is obtained. The first test step defines the test conditions entering the test step as: the interface has a test start button, text for the test start button, and location parameters for the start button. And identifying whether a test start button exists in the user interface picture, whether the character of the test start button is correct, whether the position parameter is correct, if so, meeting the test condition, acquiring an operation object corresponding to the test step, namely the test start button in the user interface, and clicking the test start button to open the access page according to the test content.
Obtaining a second test step definition according to the test flow definition, wherein the test conditions entering the test step in the second test step are as follows: the picture entering the page is a preset picture, and the entering interface is provided with a button 1 and characters and position parameters of the button 1. And identifying whether the picture of the user interface is a preset picture, whether a button 1 is arranged in the picture, and whether the characters and position parameters of the button 1 are correct, if so, meeting the test conditions, acquiring an operation object corresponding to the test step, namely the button 1 in the user interface, and clicking the button 1 to open a battle page according to the test content.
Obtaining a third testing step definition according to the testing flow definition, wherein the testing conditions entering the testing step in the third testing step are as follows: the pictures on the battle page are preset pictures, and the battle interface is provided with an end button and the characters and position parameters of the end button. And identifying whether the user interface picture is a preset picture, whether an end button exists in the picture, whether the character and position parameters of the end button are correct, if so, meeting the test conditions, acquiring an operation object corresponding to the test step, namely the end button in the user interface, and clicking the end button to enter a return page.
And according to the definition of the test flow, ending the test flow.
In another embodiment, for the problem that the conventional test script cannot be reused, the flow definition of the called sub-flow may be obtained, and the operation object information may be obtained. For some common automatic processes such as social software logging, account login and the like, a common callable module can be made, and other automatic processes only need to designate the ID of a common sub-process in the self-process.
The application also provides an application scenario applying the user interface testing method. Specifically, the application of the user interface test method in the application scenario is as follows:
as shown in the overall architecture of the user interface test scheme of fig. 6, two phases are included, namely a labeling phase and an automated execution phase of the user interface test.
In the labeling stage, as shown in the left part of fig. 6, an automatic labeling tool is used to label the user interface test flow of the product to be tested, and then the user interface test flow is stored in the database through the labeling server.
The automatic labeling tool is only responsible for the user interface labeling function, and generates a test flow definition (in a JSON data format) of the user interface test flow according to the labeling of the user interface. And then the data is stored in a database through a marking server. The following key data need to be stored: product ID, flow ID (a unique number for the automation flow), flow definition (i.e., JSON data generated previously), picture set (a picture data set generated to identify the labeled UI element).
After receiving the data, the annotation server stores the related data into a database and a file system, and submits the annotated image to an image recognition training service, and recognition model training is performed by using a deep learning technology. After the model training is finished, the model is deployed to the identification model service, and at the moment, the UI automation service can be used online.
In the automated execution phase of the user interface test, as in the right part of fig. 6, the test terminal uploads the test identification information and the user interface information. The test identification information comprises a product ID and a test flow ID. The user interface information is a user interface picture.
And after receiving the test identification information and the user interface information, the server searches the ID of the step to be executed through the data service and acquires the step flow definition of the step. The process definition is similar to the linked list and comprises the current step ID, the related definition and the ID of the next step, and if the ID of the next step is undefined, the process is ended. The flow definition can be firstly stored in the cache after the initial reading, so that the next reading is facilitated.
And identifying whether the user interface information meets the conditions of the testing step or not according to the definition of the flow steps. Specifically, the target identification is carried out on whether the operation object marked on the user interface picture exists or not, the parameters are transmitted to the target detection service, the target detection service returns the identification result, and the automatic service returns the operation object information according to the step definition and the identification result.
And if the correct operation object is not found in the identification result, the test terminal is required to retransmit the current cut-off image. And after the client finishes the return operation, the client indicates that the step is finished, sends a finishing instruction to the back end, and the back end marks the corresponding step in the conversation in the cache as finished. After waiting for a period of time, the client returns to the first step for execution. Note that the waiting time may be considered as being agreed upon in the flow definition, and the waiting time after each step is executed is different.
The step flow definition is to use JSON format to describe complex UI flow including condition judgment and circulation control. The UI process is defined as a Finite State Machine (Finite State Machine), and the data description in the JSON format generates a structure in the form of a directed graph, so that the process recovery of the State Machine can be completed. The core link is as follows: how to describe the basic linear path, how to describe the condition judgment, how to describe the loop logic, and how to invoke the universally operable module.
Aiming at the problems, the JSON format data description is utilized, and four attributes are designed:
by defining a unique ID for each UI State, and a list of possible IDs for the next UI State (there may be multiple), a complete Finite automaton (Finite State Machine) is defined, which can satisfy the definition of any linear path.
By defining the identity attribute, the condition which must be met when entering a certain state is designed, thereby realizing the condition judgment of the finite automaton state switching.
Action realizes simple logic judgment and data access of a back end by defining the server logic operation attribute server, and can establish mathematical operation and judgment of multiple steps, thereby realizing more complicated condition judgment
And through the attributes such as condition judgment, a state list and the like, the circulation logic can be effectively realized.
By setting the subflow attribute, the unique identifier of the universal flow can be defined, so that the module sharing is realized.
Note that the state attribute names are used herein for reference only, and the attribute names in each state are not necessarily identical to the names used herein.
Wherein the description of the linear path is determined by the properties of each test step.
In one embodiment, the code for the test flow is as follows:
each test step comprises a plurality of attributes, each meaning:
id is the only identification of the state, and the identification is a character string type;
identification, namely an identification condition for judging whether to enter the state, which is defined by other data (described later), and is identified by an identification condition id;
expectedStates, there may be multiple expected subsequent states, represented by a list, where the element is an identifier of the state;
ResultOnExit whether it is necessary to exit the process after the state is reached
IntervalTime, after reaching the state, entering the next state to judge the required waiting time
actions-actions that are performed to reach the next state after entering that state. The operation is not limited to one single action, and may be plural.
For example, the test condition identity defines an image recognition object s1_ o1, which is matched with the recognition element according to the acquired picture. The code is as follows:
the attributes contained in the identified matching definition of the upper graph include:
screenimage original image Source
Type of unitType match, designated herein as image match.
The identification ByImage establishes a local image (pic \ s1_ o1.jpg) needing matching, a matching degree threshold value of 0.7, matching waiting time of 60 seconds, and the border position of a matching element on the local image (calculated according to the ratio of the height to the width of the image)
The attribute of the operation object may be an operation of the operation object or data processing.
The operation action refers to an operation which needs to be completed by the client, and includes:
click-clicking on a specified element
input: entering at a specified element location
sleep: commanding client to temporarily not operate
And (3) exiting: requiring a client to exit a program
The data processing comprises the following steps: the operation required to be completed by the client comprises the following steps:
click-clicking on a specified element
input: entering at a specified element location
sleep: commanding client to temporarily not operate
And (3) exiting: requiring a client to exit a program
For example, one step is data processing of an operation object, and the code thereof is as follows:
specifically, a server _ save action is called in the start _ state, two key values left _ key1 and right _ key1 are stored respectively (redis can be used in practical application), and then the state _0 is entered. A server _ subtrect action is first called at state _0, two values stored in the previous state are read, and the subtraction is performed, and the result is stored in result _ key 1. Next, a second action, server. assert, is performed in the same state, and result _ key1 is read to determine if its value matches the expected value expectedValue. If the action execution is successful, check if the next expectedState, state _1, can be entered.
The user interface testing method based on the framework has the greatest benefit that the UI automation service can bear the automation requirements of multiple concurrent and different service scenes, and a set of automation tools do not need to be developed for different services respectively. At present, the service is applied to a plurality of projects such as game adaptation test, cloud game automation, recharging test and the like, the load QPS exceeds 2000, and the input CPU resource does not exceed 100.
It should be understood that, although the steps in the flowchart of fig. 2 are shown in order as indicated by the arrows, the steps are not necessarily performed in order as indicated by the arrows. The steps are not performed in a strict order unless explicitly stated herein, and may be performed in other orders. Moreover, at least a portion of the steps in fig. 2 may include multiple steps or multiple stages, which are not necessarily performed at the same time, but may be performed at different times, and the order of performing the steps or stages is not necessarily sequential, but may be performed alternately or in alternation with other steps or at least a portion of the steps or stages in other steps.
In one embodiment, as shown in fig. 7, there is provided a user interface testing apparatus, which may be a part of a computer device using a software module or a hardware module, or a combination of the two, the apparatus specifically includes: information acquisition module, definition acquisition module, analysis module and feedback module, wherein:
an information obtaining module 702, configured to obtain test identification information and user interface information uploaded by a test terminal for a user interface to be tested;
a definition obtaining module 704, configured to obtain a test step definition of a test step to be executed according to the test identification information;
the analysis module 706 is configured to identify user interface information according to the test step definition, and when it is determined that the condition for entering the test step is met, obtain operation object information corresponding to the test step;
and the feedback module 708 is configured to return the operation object information corresponding to the test step to the test terminal, and the terminal performs a test operation on the operation object in the user interface according to the operation object information.
When the user interface testing device executes the UI automation test, the testing terminal only needs to execute light-weight operation, the light-weight operation comprises uploading test identification information and user interface information, and carrying out testing operation according to operation object information returned by the server, and the logic judgment processing is realized by the back-end server. Because the business logic and the judgment are completed by the back end, the front end is only responsible for receiving instructions and operations, and does not perform business logic related judgment, the UI automation service can bear the automation requirements of multiple concurrent and different business scenes, and a set of automation tools do not need to be developed for different businesses respectively.
In another embodiment, a parsing module includes:
a condition obtaining module, configured to obtain a test condition for entering the testing step, where the test condition is predefined in the testing step definition;
the judging module is used for identifying whether the user interface information meets the test condition;
and the content analysis module is used for acquiring the operation object information corresponding to the test step when the condition meeting the test condition is identified.
In another embodiment, the test system further comprises a step identification module for marking the identification of the next round to be tested according to the test path of the test flow.
A definition acquisition module comprising:
and the searching module is used for acquiring the test flow definition of the current test flow according to the test identification information.
And the matching module is used for acquiring the test step definition corresponding to the identification mark of the step to be tested of the previous round of test mark according to the test path defined by the test flow.
In another embodiment, the user interface information includes a user interface picture.
And the judging module is used for identifying the user interface picture according to the test condition and identifying whether the user interface picture is matched with the test condition.
The identification module is used for identifying whether the user interface picture corresponds to the marked picture in the test step; and if so, identifying whether the user interface picture has interface elements specified by the test conditions.
In one embodiment, the user interface information includes control properties of the user interface;
and the judging module is used for judging whether the control attribute of the user interface has the control specified by the test condition.
In another embodiment, the method further comprises:
the annotated content acquisition module is used for acquiring the annotated content of the user interface of each test step in the product test flow; the marked content is marked by a marking tool through a test terminal;
the data conversion module is used for generating a test flow definition of a test flow of the product according to the marked content;
and the storage module is used for storing the test definition of the test flow of the product.
For the specific definition of the user interface testing device, reference may be made to the above definition of the user interface testing method, which is not described herein again. The modules in the user interface testing device can be wholly or partially implemented by software, hardware and a combination thereof. The modules can be embedded in a hardware form or independent from a processor in the computer device, and can also be stored in a memory in the computer device in a software form, so that the processor can call and execute operations corresponding to the modules.
In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as shown in fig. 8. The computer device includes a processor, a memory, and a network interface connected by a system bus. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device comprises a nonvolatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of an operating system and computer programs in the non-volatile storage medium. The database of the computer device is used for storing test definition data. The network interface of the computer device is used for communicating with an external terminal through a network connection. The computer program is executed by a processor to implement a user interface testing method.
Those skilled in the art will appreciate that the architecture shown in fig. 8 is merely a block diagram of some of the structures associated with the disclosed aspects and is not intended to limit the computing devices to which the disclosed aspects apply, as particular computing devices may include more or less components than those shown, or may combine certain components, or have a different arrangement of components.
In an embodiment, there is further provided a computer device comprising a memory and a processor, the memory having stored therein a computer program, the processor implementing the steps of the above method embodiments when executing the computer program.
In an embodiment, a computer-readable storage medium is provided, in which a computer program is stored which, when being executed by a processor, carries out the steps of the above-mentioned method embodiments.
In one embodiment, a computer program product or computer program is provided that includes computer instructions stored in a computer-readable storage medium. The computer instructions are read by a processor of a computer device from a computer-readable storage medium, and the computer instructions are executed by the processor to cause the computer device to perform the steps in the above-mentioned method embodiments.
It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by hardware instructions of a computer program, which can be stored in a non-volatile computer-readable storage medium, and when executed, can include the processes of the embodiments of the methods described above. Any reference to memory, storage, database or other medium used in the embodiments provided herein can include at least one of non-volatile and volatile memory. Non-volatile Memory may include Read-Only Memory (ROM), magnetic tape, floppy disk, flash Memory, optical Memory, or the like. Volatile Memory can include Random Access Memory (RAM) or external cache Memory. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM), for example.
The technical features of the above embodiments can be arbitrarily combined, and for the sake of brevity, all possible combinations of the technical features in the above embodiments are not described, but should be considered as the scope of the present specification as long as there is no contradiction between the combinations of the technical features.
The above-mentioned embodiments only express several embodiments of the present application, and the description thereof is more specific and detailed, but not construed as limiting the scope of the invention. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the concept of the present application, which falls within the scope of protection of the present application. Therefore, the protection scope of the present patent application shall be subject to the appended claims.
Claims (10)
1. A method for user interface testing, the method comprising:
acquiring test identification information and user interface information uploaded by a test terminal aiming at a tested user interface;
acquiring a test step definition of a test step to be executed according to the test identification information;
identifying the user interface information according to the test step definition, and acquiring the operation object information corresponding to the test step when determining that the condition for entering the test step is met;
and returning the operation object information corresponding to the test step to the test terminal, and executing test operation on the operation object in the user interface by the test terminal according to the operation object information.
2. The method of claim 1, wherein identifying the user interface information according to the test step definition, and acquiring the operation object information corresponding to the test step when determining that the condition for entering the test step is satisfied comprises:
acquiring test conditions which are predefined in the test step definition and enter the test step;
identifying whether the user interface information satisfies the test condition;
and if so, acquiring the operation object information corresponding to the test step from the test step definition.
3. The method according to claim 1, further comprising, after obtaining the operation object information corresponding to the test procedure from the test procedure definition:
marking the identification mark of the next round of step to be tested according to the testing path of the testing process;
obtaining a test step definition of a test step to be executed according to the test identification information, wherein the test step definition comprises the following steps:
acquiring a test flow definition of the current test flow according to the test identification information;
and acquiring a test step definition corresponding to the identification mark of the step to be tested of the previous round of test mark according to the test path defined by the test flow.
4. The method of claim 2, wherein the user interface information comprises a user interface picture;
identifying whether the user interface information satisfies the test condition, including:
and identifying the user interface picture according to the test condition, and identifying whether the user interface picture is matched with the test condition.
5. The method of claim 4, wherein identifying the user interface picture according to the test condition and identifying whether the user interface picture matches the test condition comprises:
identifying whether the user interface picture is consistent with the marked picture corresponding to the testing step;
and if so, identifying whether the user interface picture has interface elements specified by the test conditions.
6. The method of claim 2, wherein the user interface information includes control properties of the user interface;
identifying whether the user interface information satisfies the test condition, including:
and judging whether the control attribute of the user interface is consistent with the attribute of the control specified by the test condition.
7. The method of claim 1, further comprising:
acquiring the labeling content of a user interface of each testing step in the product testing process; the marked content is marked by a marking tool through a test terminal;
generating a test flow definition of the test flow of the product according to the labeled content;
storing a test definition of the test flow for the product.
8. A user interface testing apparatus, the apparatus comprising:
the information acquisition module is used for acquiring test identification information and user interface information uploaded by a test terminal aiming at a tested user interface;
the definition acquisition module is used for acquiring a test step definition of a test step to be executed according to the test identification information;
the analysis module is used for identifying the user interface information according to the test step definition, and acquiring the operation object information corresponding to the test step when the condition of entering the test step is determined to be met;
and the feedback module is used for returning the operation object information corresponding to the testing step to the testing terminal, and the testing terminal executes the testing operation on the operation object in the user interface according to the operation object information.
9. A computer device comprising a memory and a processor, the memory storing a computer program, characterized in that the processor, when executing the computer program, implements the steps of the method of any of claims 1 to 7.
10. A computer-readable storage medium, in which a computer program is stored which, when being executed by a processor, carries out the steps of the method according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010939537.XA CN112214397B (en) | 2020-09-09 | 2020-09-09 | User interface test method and device, computer equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010939537.XA CN112214397B (en) | 2020-09-09 | 2020-09-09 | User interface test method and device, computer equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112214397A true CN112214397A (en) | 2021-01-12 |
| CN112214397B CN112214397B (en) | 2022-05-20 |
Family
ID=74049271
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010939537.XA Active CN112214397B (en) | 2020-09-09 | 2020-09-09 | User interface test method and device, computer equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112214397B (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113660535A (en) * | 2021-08-18 | 2021-11-16 | 海看网络科技(山东)股份有限公司 | System and method for monitoring content change of EPG column of IPTV service |
| CN113791974A (en) * | 2021-08-25 | 2021-12-14 | 微梦创科网络科技（中国）有限公司 | User interface UI element explicit waiting method and device |
Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104035867A (en) * | 2014-06-04 | 2014-09-10 | 重庆泰凯科技有限公司 | Android-based application function traverse test method |
| CN104484273A (en) * | 2014-12-19 | 2015-04-01 | 广州博冠信息科技有限公司 | Test method for application program, device and system |
| CN104699616A (en) * | 2015-03-31 | 2015-06-10 | 北京奇虎科技有限公司 | Method, device and system for testing application |
| CN105607993A (en) * | 2015-09-30 | 2016-05-25 | 北京奇虎科技有限公司 | Method and device for testing user interfaces UI of applications |
| CN107391383A (en) * | 2017-08-11 | 2017-11-24 | 上海巍擎信息技术有限责任公司 | The method of testing and test system of a kind of Application Program Interface |
| CN107577607A (en) * | 2017-09-07 | 2018-01-12 | 携程旅游信息技术（上海）有限公司 | User interface automated testing method, device, electronic equipment, storage medium |
| CN108509333A (en) * | 2017-02-28 | 2018-09-07 | 腾讯科技（深圳）有限公司 | Adjustment method and device |
| CN109783388A (en) * | 2019-01-16 | 2019-05-21 | 北京金山云网络技术有限公司 | UI automated testing method, device and electronic equipment |
| CN110221969A (en) * | 2019-05-21 | 2019-09-10 | 深圳壹账通智能科技有限公司 | A kind of page function test method and relevant apparatus |
| CN110321284A (en) * | 2019-06-03 | 2019-10-11 | 平安科技（深圳）有限公司 | Test data input method, device, computer equipment and storage medium |
| US10628291B2 (en) * | 2017-01-23 | 2020-04-21 | International Businees Machines Corporation | Programming assistance to identify suboptimal performing code and suggesting alternatives |
-
2020
- 2020-09-09 CN CN202010939537.XA patent/CN112214397B/en active Active
Patent Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104035867A (en) * | 2014-06-04 | 2014-09-10 | 重庆泰凯科技有限公司 | Android-based application function traverse test method |
| CN104484273A (en) * | 2014-12-19 | 2015-04-01 | 广州博冠信息科技有限公司 | Test method for application program, device and system |
| CN104699616A (en) * | 2015-03-31 | 2015-06-10 | 北京奇虎科技有限公司 | Method, device and system for testing application |
| CN105607993A (en) * | 2015-09-30 | 2016-05-25 | 北京奇虎科技有限公司 | Method and device for testing user interfaces UI of applications |
| US10628291B2 (en) * | 2017-01-23 | 2020-04-21 | International Businees Machines Corporation | Programming assistance to identify suboptimal performing code and suggesting alternatives |
| CN108509333A (en) * | 2017-02-28 | 2018-09-07 | 腾讯科技（深圳）有限公司 | Adjustment method and device |
| CN107391383A (en) * | 2017-08-11 | 2017-11-24 | 上海巍擎信息技术有限责任公司 | The method of testing and test system of a kind of Application Program Interface |
| CN107577607A (en) * | 2017-09-07 | 2018-01-12 | 携程旅游信息技术（上海）有限公司 | User interface automated testing method, device, electronic equipment, storage medium |
| CN109783388A (en) * | 2019-01-16 | 2019-05-21 | 北京金山云网络技术有限公司 | UI automated testing method, device and electronic equipment |
| CN110221969A (en) * | 2019-05-21 | 2019-09-10 | 深圳壹账通智能科技有限公司 | A kind of page function test method and relevant apparatus |
| CN110321284A (en) * | 2019-06-03 | 2019-10-11 | 平安科技（深圳）有限公司 | Test data input method, device, computer equipment and storage medium |
Non-Patent Citations (1)
| Title |
|---|
| 张文烨: ""基于图像识别的移动端应用控件检测方法"", 《计算机应用》 * |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113660535A (en) * | 2021-08-18 | 2021-11-16 | 海看网络科技(山东)股份有限公司 | System and method for monitoring content change of EPG column of IPTV service |
| CN113791974A (en) * | 2021-08-25 | 2021-12-14 | 微梦创科网络科技（中国）有限公司 | User interface UI element explicit waiting method and device |
| CN113791974B (en) * | 2021-08-25 | 2024-05-17 | 微梦创科网络科技（中国）有限公司 | Explicit waiting method and device for user interface UI element |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112214397B (en) | 2022-05-20 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106716934B (en) | Chat interaction method and device and electronic equipment thereof | |
| CA2915619C (en) | Method and apparatus for customized software development kit (sdk) generation | |
| CA2912852C (en) | Method and apparatus for code virtualization and remote process call generation | |
| US11256484B2 (en) | Utilizing natural language understanding and machine learning to generate an application | |
| CN108776787A (en) | Image processing method and device, electronic equipment, storage medium | |
| US10169005B2 (en) | Consolidating and reusing portal information | |
| CN112214397B (en) | User interface test method and device, computer equipment and storage medium | |
| CN105393226A (en) | Automated Generation Of Scripted And Manual Test Cases | |
| Bernstein | Crowd-powered systems | |
| CN116933362A (en) | Automatic generation method and device for indoor design scheme, computer equipment and medium | |
| CN116450202A (en) | Page configuration method, page configuration device, computer equipment and computer readable storage medium | |
| CN111367501A (en) | Software online development method, device, equipment and storage medium | |
| US10289788B1 (en) | System and method for suggesting components associated with an electronic design | |
| CN112423024A (en) | Video transcoding method and device, computer equipment and storage medium | |
| US11188548B2 (en) | Profile data store automation via bots | |
| CN115756692A (en) | Method for automatically combining and displaying pages based on style attributes and related equipment thereof | |
| CN111124386B (en) | Animation event processing method, device, equipment and storage medium based on Unity | |
| JP2007157128A (en) | Device and method for setting device, device and method for acquiring information, recording medium and program | |
| KR101987183B1 (en) | Apparatus, method and server for app development supporting front-end development using app development utility | |
| CN110942306A (en) | Data processing method and device and electronic equipment | |
| CN113806596B (en) | Operation data management method and related device | |
| CN110908745A (en) | Data analysis display method and system, storage medium, server and client | |
| US11245795B2 (en) | System and method of running an agent guide script-flow in an employee desktop web client | |
| CN117971378A (en) | Workflow execution method, workflow execution device, electronic equipment and storage medium | |
| CN112162742A (en) | Code deployment method, device and equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |