CN110347608B

CN110347608B - Interactive application testing method and related device

Info

Publication number: CN110347608B
Application number: CN201910647927.7A
Authority: CN
Inventors: 韩尹波; 孙大伟; 方亮; 荆彦青; 贺国睿; 曹斌; 李林申
Original assignee: Tencent Technology Shenzhen Co Ltd
Current assignee: Tencent Technology Shenzhen Co Ltd
Priority date: 2019-07-17
Filing date: 2019-07-17
Publication date: 2024-03-22
Anticipated expiration: 2039-07-17
Also published as: CN110347608A

Abstract

The application discloses a testing method of an interactive application, which is used for improving the testing efficiency of the interactive application in the testing process. The method comprises the following steps: acquiring a first image, wherein the first image is a screenshot in the running process of the interactive application; acquiring a target point set of the first image through a detection model, wherein the target point set comprises one or more target coordinate points; generating an operation instruction set according to the target point set, wherein the operation instruction set comprises one or more operation instructions corresponding to the target coordinate points; and obtaining a test result according to the operation instruction set.

Description

Interactive application testing method and related device

Technical Field

The present disclosure relates to the field of computer technologies, and in particular, to a method and an apparatus for testing an interactive application.

Background

With the development of mobile communication technology, mobile terminals have become an indispensable tool. People can communicate through the mobile terminal, and can also use the mobile terminal to surf the internet, work, entertain and the like. Because the mobile terminal has portability, the game on the mobile terminal can be played at any time and any place, so the game on the mobile terminal has a trend of replacing the traditional personal computer terminal game, and the game becomes the first choice of leisure and entertainment for people.

In the development process of a mobile terminal game, various aspects of testing are generally required to be performed on the mobile terminal game so as to test whether an abnormality exists in the rerun process of the mobile terminal game. At present, the test scheme of the mobile terminal game is to test based on a Monkey test tool, namely, operations of a user are simulated by randomly generating clicking, sliding and the like in the running process of the mobile terminal game, so that the test of the mobile terminal game is realized.

However, the scheme based on the Monkey testing tool for testing has pure randomness, and in most cases, random operation generated by the Monkey testing tool is invalid operation, namely, corresponding operation response events cannot be triggered, so that the operation efficiency is low, and the testing efficiency is low.

Disclosure of Invention

The embodiment of the application provides a testing method and a related device for an interactive application, which are characterized in that an interface screenshot in the running process of the interactive application is obtained, the screenshot is processed through a pre-trained detection model, an effective point set capable of triggering an operation response event on the screenshot is obtained, and then the corresponding clicking operation is carried out on the interactive application based on the effective point set, so that the effective triggering of the operation response event in the testing process of the interactive application can be ensured, and the testing efficiency is improved.

A first aspect of an embodiment of the present application provides a method for testing an interactive application, including:

acquiring a first image, wherein the first image is a screenshot in the running process of the interactive application;

acquiring a target point set of the first image through a detection model, wherein the target point set comprises one or more target coordinate points;

generating an operation instruction set according to the target point set, wherein the operation instruction set comprises one or more operation instructions corresponding to the target coordinate points;

and obtaining a test result according to the operation instruction set.

A second aspect of the embodiments of the present application provides a testing apparatus for an interactive application, including:

the acquisition unit is used for acquiring a first image by an application, wherein the first image is a screenshot in the running process of the interactive application;

the acquisition unit is also used for acquiring a target point set of the screenshot through the detection model, wherein the target point set comprises one or more target coordinate points;

the generating unit is used for generating an operation instruction set according to the target point set, wherein the operation instruction set comprises one or more operation instructions corresponding to the target coordinate points;

the acquisition unit is also used for acquiring a test result according to the operation instruction set.

In one possible design, in an implementation manner of the second aspect of the embodiments of the present application, the method further includes a joining unit;

The acquisition unit is also used for acquiring the category corresponding to one or more target coordinate points;

the adding unit is used for adding one or more target coordinate points into the first processing queue according to the categories corresponding to the one or more target coordinate points;

the generating unit is further configured to generate an operation instruction set according to the order of one or more target coordinate points in the first processing queue, where the operation instructions in the operation instruction set have a corresponding operation order.

In one possible design, in a first implementation manner of the second aspect of the embodiments of the present application,

the acquisition unit is also used for acquiring the position information of one or more target coordinate points in the screenshot;

the adding unit is further used for adding one or more target coordinate points to the second processing queue according to the position information of the one or more target coordinate points in the screenshot;

the generating unit is further configured to generate an operation instruction set according to the order of the one or more target coordinate points in the second processing queue, where the operation instructions in the operation instruction set have a corresponding operation order.

The acquisition unit is also used for acquiring the display types of the screenshot, wherein the display types comprise horizontal screen display and vertical screen display;

and the adding unit is also used for adding the one or more target coordinate points into the second processing queue according to the position information of the one or more target coordinate points in the screenshot and the display type of the screenshot.

In a possible design, in a first implementation manner of the second aspect of the embodiments of the present application, the method further includes a replacing unit;

the generation unit is further used for generating a random point set according to the first image, wherein the random point set comprises N random coordinate points, and the distance value between the N random coordinate points is larger than a preset threshold value;

a replacing unit, configured to replace M random coordinate points in the random point set with target coordinate points in the target point set, to obtain an updated random point set, where the target point set includes M target coordinate points;

and the generating unit is also used for generating an operation instruction set according to the updated random point set.

In a possible design, in a first implementation manner of the second aspect of the embodiments of the present application, the method further includes an operation unit;

the acquisition unit is further used for acquiring the first interface control information if a target page exists on the running interface of the interactive application, wherein the target page is a sub page independent of the main interface of the interactive application;

The generating unit is further used for generating a target page canceling operation instruction if the first interface control information is matched with a preset regular expression;

and the operation unit is also used for operating the target page according to the target page cancel operation instruction.

the acquisition unit is further used for acquiring second interface control information if the running interface of the interactive application is a login interface;

the acquisition unit is further used for acquiring login information if the second interface control information is matched with preset login interface control information;

and the operation unit is also used for carrying out login operation on the login interface according to the login information.

A third aspect of the embodiments of the present application provides a server, including:

memory, transceiver, processor, and bus system;

wherein the memory is used for storing programs;

the processor is used for executing the program in the memory, and comprises the following steps:

acquiring a target point set of the screenshot through a detection model, wherein the target point set comprises one or more target coordinate points;

obtaining a test result according to the operation instruction set;

the bus system is used to connect the memory and the processor to communicate the memory and the processor.

A fourth aspect of the embodiments provides a computer readable storage medium having instructions stored therein which, when run on a computer, cause the computer to perform the method of any of the above aspects.

A fifth aspect of the embodiments of the present application provides a computer program product comprising instructions which, when run on a computer or processor, cause the computer or processor to perform the method of any of the above aspects.

From the above technical solutions, the embodiments of the present application have the following advantages:

Drawings

FIG. 1 is a schematic diagram of an architecture of a test system for an interactive application according to an embodiment of the present application;

FIG. 2 is an example diagram of a page of an interactive application to be tested provided in an embodiment of the present application;

FIG. 3 is an exemplary diagram of a testing method for an interactive application according to an embodiment of the present application;

FIG. 4 is an exemplary diagram of a game interface screenshot provided in this embodiment;

FIG. 5 is an exemplary diagram of labeling and categorizing the game interface shots of FIG. 4;

FIG. 6 is a diagram illustrating a location priority of a screenshot provided in an embodiment of the present application;

FIG. 7 is a diagram illustrating a position priority of a vertical screen shot according to an embodiment of the present application;

FIG. 8 is a schematic diagram of a test flow of an interactive application according to an embodiment of the present application;

FIG. 9 is a schematic flow chart of generating a point set according to an embodiment of the present application;

FIG. 10 is a schematic diagram of an embodiment of a testing apparatus for an interactive application according to an embodiment of the present application;

fig. 11 is a schematic diagram of a server structure according to an embodiment of the present application.

Detailed Description

The embodiment of the application provides a testing method and a related device for an interactive application, which are used for improving the testing efficiency of the interactive application in the testing process.

The terms "first," "second," "third," "fourth" and the like in the description and in the claims of this application and in the above-described figures, if any, are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used may be interchanged where appropriate such that embodiments of the present application described herein may be capable of operation in sequences other than those illustrated or described herein, for example. Furthermore, the terms "comprises," "comprising," and "includes" and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed or inherent to such process, method, article, or apparatus.

It should be appreciated that the embodiments of the present application are applicable to machine learning-based application testing scenarios, and in particular, to scenarios in which applications such as instant messaging programs, information browsing programs, or interactive applications are tested. Among them, machine Learning (ML) is a multi-domain interdisciplinary, and involves multiple disciplines such as probability theory, statistics, approximation theory, convex analysis, algorithm complexity theory, etc. It is specially studied how a computer simulates or implements learning behavior of a human to acquire new knowledge or skills, and reorganizes existing knowledge structures to continuously improve own performance. Machine learning is the core of artificial intelligence, a fundamental approach to letting computers have intelligence, which is applied throughout various areas of artificial intelligence. Machine learning and deep learning typically include techniques such as artificial neural networks, confidence networks, reinforcement learning, transfer learning, induction learning, teaching learning, and the like.

For example, when testing an instant messaging program, capturing a screenshot of a main interface, a chat window interface or a chat information input interface in the instant messaging program, identifying the screenshot based on a pre-trained detection model (such as the artificial neural network described above), obtaining an effective operation point set in the screenshot, for example, obtaining a point set corresponding to the positions of a chat window, an address book and the like in the main interface screenshot, or obtaining a point set corresponding to the positions of a text input, a picture input, a sending information and the like in the chat window interface screenshot, and then operating the instant messaging program interface based on the effective operation point set to trigger a corresponding operation event to enter a corresponding scene page. For another example, when the game is tested, the screen capture of the prop purchase page, the combat page or the array adjustment page in the game is intercepted, then the screen capture is identified based on a pre-trained detection model, an effective operation point set in the screen capture is obtained, for example, the point set corresponding to the positions of gold coin purchase, diamond purchase and the like in the prop purchase page is obtained, and then the game page is operated based on the effective operation point sets, so that corresponding operation events are triggered, and the game page enters into the corresponding scene page. For convenience of description, the following will take a game playing test as an example, and describe in detail the method for testing an interactive application provided in the embodiments of the present application.

At present, a scheme for testing a game is generally based on a Monkey testing tool, namely, operations of a user are simulated by randomly generating clicking, sliding and the like in the running process of the game, so that the game is tested. Here, monkey refers to a person who has no test experience and even does not know about the computer at all (just like a Monkey), he does not need to know any knowledge about the user interaction of the program, and if he is given a program, he will operate on his interface, which is purposeless and randomly pressed, in other words, the Monkey test tool can be understood as a tool for performing random clicking operation on the game interface.

For the scheme based on the Monkey test tool, assuming that there are M pixels in the game interface, where N pixels are valid click areas, then the number of valid click expenses is M/N. The effective clicking is to click a certain position in the interface, the interface generates a response (for example, enters another scene interface or pops up a new response interface), the effective clicking area is an area which can generate effective clicking after clicking, the effective clicking time is the average clicking time of one interface to generate one effective clicking, and the effective area occupation ratio is the proportion of the area occupation area of the effective clicking area in one interface. Generally, for most game interfaces, the effective area of the game interface is 1% to 10%, that is, the effective click-through is 100 when the effective area is 1%. Furthermore, during testing, a certain time interval (e.g., 1 second or 2 seconds) is often required between clicking operations in order for the screen interface to jump normally, and in this case, traversing a game interface may take 100 seconds, which is extremely inefficient.

In view of this, the embodiment of the application provides a testing method and related device for an interactive application, by acquiring an interface screenshot in the running process of the interactive application and processing the screenshot through a pre-trained detection model, since the detection model is obtained by training a large number of samples marked with effective clicking areas, an effective point set on the screenshot can be obtained, and then corresponding clicking operation is performed on the interactive application based on the effective point set, so that the effective triggering of the operation response event in the testing process of the interactive application can be ensured, and the testing efficiency is improved.

Referring to fig. 1, fig. 1 is a schematic diagram of an architecture of an interactive application testing system in an embodiment of the present application, as shown in the fig. 1, a terminal establishes communication connection with a server through a network, a user needing to test the interactive application may upload the interactive application to be tested to the server through the terminal, the server executes the interactive application testing method provided by the embodiment of the present application, tests the interactive application, and sends a testing result to the terminal after the testing is completed.

As shown in fig. 2, fig. 2 is an exemplary diagram of a page of an interactive application to be tested according to an embodiment of the present application. The user can open a corresponding application uploading interface on the terminal, upload the interactive application to be tested on the uploading interface, select a system or a machine type running the interactive application, such as an android system or an ios system, and the like, then upload the interactive application to the server by clicking a confirmation button, and finally execute the test on the interactive application by the server.

It should be noted that, in the embodiment of the present application, in addition to the testing of the interactive application uploaded by the user through the server, the testing of the interactive application may also be performed on a terminal where a software program corresponding to the method for testing an interactive application mentioned in the embodiment of the present application is deployed, which is not limited herein specifically.

The foregoing is a description of the testing system of the interactive application provided in the embodiment of the present application, please refer to fig. 3, and fig. 3 is an exemplary diagram of a testing method of the interactive application provided in the embodiment of the present application.

An embodiment of a method for testing an interactive application provided in an embodiment of the present application includes:

301. Acquiring a first image, wherein the first image is a screenshot in the running process of the interactive application;

in this embodiment, during testing of an interactive application (such as a game, which will be described below as an example of a game), an interface screenshot during the running of the game may be obtained, so that the interface screenshot is used as an interface to be operated. The screenshot of the interface can be any screenshot of an interface in the running process of the game, such as a screenshot of a game main interface, a screenshot of a combat interface or a screenshot of a prop purchasing interface, and the like.

Specifically, referring to fig. 4, fig. 4 is an exemplary diagram of a game interface screenshot provided in the present embodiment. In fig. 4, a prop purchase interface in a game is shown, where a plurality of purchase buttons marked with different amounts are arranged below the prop purchase interface, the purchase buttons are all rectangular patterns, corresponding purchasable props (i.e. diamonds) are respectively arranged above the purchase buttons, and rectangular return buttons are arranged at the upper right corner of the interface. Obviously, the purchase button and the return button in the interface are effective click areas, namely after clicking the purchase button, a purchase instruction can be triggered to purchase corresponding props; clicking the close button can trigger a close operation instruction to close the current prop purchase page and return to the previous interface. In addition to the purchase button and the close button, other areas are invalid click areas, i.e., effective interface response cannot be obtained after clicking.

302. Acquiring a target point set of the first image through a detection model, wherein the target point set comprises one or more target coordinate points;

in this embodiment, the detection model is a pre-trained model, and can identify an effective click region in the game interface screenshot, so as to predict and obtain the effective click region in the game interface screenshot, and a target coordinate point capable of triggering effective click operation can be determined based on the effective click region in the game interface screenshot.

In particular, the detection model may be a model capable of detecting the generic game element, i.e. the detection model is trained using samples labeled with the generic game element. The method mainly comprises the following two steps of: data labeling and model training.

Firstly, the specific flow of data annotation is as follows: and collecting a certain number of game screenshots in advance, wherein the game screenshots can be screenshots in different types of games, can also be screenshots of different interfaces in the same game, and can also comprise screenshots in different types of games at the same time, and each same game has a plurality of screenshots of different interfaces so as to ensure the diversity of the game screenshots. After the game screen shots are collected, game elements with strong universality in the game screen shots can be marked and classified. As shown in FIG. 5, FIG. 5 is an exemplary diagram of labeling and categorizing the game interface shots of FIG. 4. Wherein, a plurality of rectangular buttons below the interface in fig. 5 are all labeled, and are labeled as K1, K1 represents a rectangular type (i.e., rectangular buttons), that is, rectangular buttons for clicking on purchasing props are labeled and classified as rectangular categories; the close button in the upper right hand corner of the interface is labeled and labeled K2, K2 indicating the close category, i.e., the button for closing the page is labeled and categorized as the close category.

In the embodiment of the present application, game elements with strong versatility refer to elements that can trigger an effective operation instruction after clicking, and generally include, but are not limited to, background frames such as rectangular frames, circular frames, rounded rectangular frames, and the like. It will be appreciated that for most games, in order to ensure the visual experience of the user, the effective click region in the game interface is often formed by adding corresponding text to a shaped background frame, so that if a specific background frame can be identified, a specific effective click region can be identified.

Secondly, the specific flow of model training is as follows: selecting an initial model to be trained, inputting marked sample data (namely a game screenshot) into the initial model, and carrying out iterative training on the initial model until a loss function in the model converges to a stable value, so as to obtain a trained detection model.

It will be appreciated that the detection model may specifically be a Machine Learning (ML) model, and that after a certain sample data is given, machine Learning (i.e. model training) may be performed based on the given sample data. That is, after training the machine learning model by using sample data marked with common game elements, the model can simulate human learning based on the specific sample data, so as to learn to distinguish the common game elements in the samples, and thus, after obtaining the game interface screenshot, the trained model can identify the common game elements in the game interface screenshot, so as to obtain the common game elements in the game interface screenshot. In this embodiment, the detection model may specifically include, but is not limited to, the following models: convolutional neural network (convolutional neural network, CNN), full convolutional network (fully convolutional networks, FCN).

Specifically, when the first image is detected by the detection model, the detection model is trained from a large number of sample data to which the common game element is labeled in advance, so that the detection model can detect the common game element in the first image. Since the detected background frame in the first image is mostly a rectangular frame, a circular frame or a rounded rectangular frame, clicking any point in the operation background frame can trigger a valid operation instruction for the background frame. Thus, for a background frame (i.e., a game element), a target coordinate point corresponding to the background frame may be generated, for example, taking the center point of the background frame as the target coordinate point, that is, the target point set obtained based on the first image, the number of target coordinate points being identical to the number of detected common elements.

303. Generating an operation instruction set according to the target point set, wherein the operation instruction set comprises one or more operation instructions corresponding to the target coordinate points;

in this embodiment, since the target point set generally includes one or more target coordinate points, and the purpose of testing the game is to enable entering into more game scenes by clicking on the interface in the game, after obtaining the target coordinate points, an operation instruction set for clicking on the target coordinate points may be generated based on the target coordinate points, where the operation instruction set includes an operation instruction corresponding to each target coordinate point, and these operation instructions specifically refer to an instruction of clicking on the target coordinate point corresponding to the operation.

304. And obtaining a test result according to the operation instruction set.

In this embodiment, after the operation instruction set is obtained, the game may be operated based on one operation instruction in the operation instruction set, so that the game progress jumps from the current one game interface to the other game interface. Specifically, after the operation instruction set including one or more operation instructions is obtained, the operation instructions in the operation instruction set can be triggered one by one, and when one of the operation instructions triggers and causes the game interface to jump, the test result of the current game interface, such as game performance data, game log and other test data corresponding to the current game interface, is correspondingly obtained. In other words, after the target point set including one or more target coordinate points is obtained, clicking operation may be performed on the target coordinate points in the target point set one by one, so long as clicking one of the target points triggers an effective operation instruction, so that the game enters into another new game interface to achieve the purpose, and each target coordinate point in the target point set is not required to be effective. Obviously, because the target coordinate points in the target point set are obtained based on the target area obtained by detection of the detection model, the probability that the target coordinate points in the target point set can trigger an effective operation instruction is far greater than that of the coordinate points clicked randomly, so that the high efficiency of the test process is ensured.

Finally, after entering a new game interface, a corresponding screenshot image can be obtained based on the new game interface, then the screenshot image is detected through a detection model to obtain a target point set, and finally the new screenshot image is operated based on the target point set to ensure that the test process can be continuously developed. That is, in the whole game test process, for each game interface in the game, the steps 301 to 304 may be executed, where the steps 301 to 304 are executed to more efficiently enter a new game interface and obtain corresponding game test data, and finally, when the game test is finished (for example, a predetermined game test time is reached), the game test data obtained by entering each new game interface is summarized, so as to obtain a game test result corresponding to the game.

In the embodiment, the screenshot is processed through the pre-trained detection model to obtain the effective point set capable of triggering the operation response event on the screenshot, and then the corresponding clicking operation is performed on the interactive application based on the effective point set, so that the operation response event can be effectively triggered in the testing process of the interactive application, and the testing efficiency is improved.

Optionally, on the basis of the first embodiment corresponding to fig. 3, in an optional embodiment of the method for testing an interactive application provided in the embodiment of the present application, before generating the operation instruction set according to the target point set, the method further includes:

acquiring categories corresponding to one or more target coordinate points;

adding one or more target coordinate points into a first processing queue according to the category corresponding to the one or more target coordinate points;

generating an operation instruction set according to the target point set, wherein the operation instruction set comprises:

and generating an operation instruction set according to the sequence of one or more target coordinate points in the first processing queue, wherein the operation instructions in the operation instruction set have corresponding operation sequences.

In this embodiment, since the target coordinate point is obtained based on the target area obtained by detecting the detection model, and the sample data for training the detection model is labeled with the category corresponding to each sample, the target area obtained by detecting the detection model also has a corresponding category, in other words, the target coordinate point corresponding to the target area also has a corresponding category. It will be appreciated that, in the same game interface, after clicking on different target coordinate points, the number of game interfaces that can be traversed finally is different, for example, after clicking on target coordinate point a1 in game interface a, the game interface B1 can be accessed, and after clicking on target coordinate point B in game interface B1, the game interface C is opened, so that it is continued, possibly the game interfaces can be accessed; after clicking the target coordinate point a2 in the game interface a, the game interface B2 may be entered, and for the game interface B2, clicking any one coordinate point in the game interface B2 may not enter a new game interface (i.e. only able to return to the game interface a), so that, for the same game interface, if each time, clicking the target coordinate point corresponding to more game interfaces can be preferentially entered, it can be ensured that more game interfaces can be traversed in the game test process, and the test effectiveness is improved.

Therefore, in this embodiment, the execution sequence of the operation instruction corresponding to each target coordinate point may be ordered based on the category of the target coordinate point, that is, the order of clicking the target coordinate point is ordered, so that when clicking the target coordinate point, the target coordinate point corresponding to more game interfaces can be clicked in a subsequent step. Specifically, in the actual application process, the categories of the target coordinate points corresponding to more game interfaces in different games are different, so that the corresponding categories can be selected as targets of preferential clicking according to specific games.

In addition, for most game interfaces, there are often buttons (e.g., rectangular buttons, circular buttons, etc.) on the game interface that can be accessed to a new game interface after clicking, and there are return buttons to the previous game interface; for some specific game interfaces, there may be closing buttons, where most of the game interfaces exist in the form of popup windows, such as advertisement interfaces, prop bar interfaces, server selection interfaces, activity bulletin interfaces, etc., where the game interfaces often have corresponding closing buttons, after clicking the closing buttons, the corresponding popup windows may be closed, and clicking other buttons in the game interfaces may be difficult to enter into an effective new game interface. For example, for an advertisement page, clicking a button other than the close button in the advertisement page may generate a link, which causes the terminal to open the browser and enter the web page corresponding to the link; for a server selection page, there may be a server selection list on the server page, where the list includes a plurality of alternative servers, and clicking a button on the page other than the close button changes the selected server, and cannot enter into a new interface. Therefore, for the above game pages, the close button on the game page can be clicked preferentially, that is, the current game page is closed, so as to avoid wasting excessive clicking operation time on the page.

acquiring position information of one or more target coordinate points in a screenshot;

according to the position information of one or more target coordinate points in the screenshot, adding the one or more target coordinate points into a second processing queue;

and generating an operation instruction set according to the sequence of one or more target coordinate points in the second processing queue, wherein the operation instructions in the operation instruction set have corresponding operation sequences.

It will be appreciated that since the game is running on the mobile terminal, most users playing the game on the mobile terminal operate the screen of the mobile terminal with the thumb while holding the mobile terminal with one or both hands, thereby achieving the game play. However, for most mobile terminals on the market, the screen is larger, and while holding the mobile terminal, it is often difficult for the user to touch all the areas on the screen of the mobile terminal effectively, that is, when the user holds the mobile terminal, the user can only touch the partial areas on the screen of the mobile terminal conveniently, so that the user often clicks the areas that he easily touches, and the areas that he cannot touch conveniently are less touched. Considering the clicking habit of users, most games usually place buttons that need to be frequently clicked (or buttons that can trigger a new interface) in an area that is easy to touch by users when designing interfaces.

In view of this, in this embodiment, after the target point set is obtained, the position information of each target coordinate point in the entire screenshot may be obtained according to the coordinate information of each target coordinate point, that is, the position information of each target coordinate point on the mobile terminal screen is obtained, then the target coordinate points are ordered based on the position information of the target coordinate points, and for the target coordinate points located in the area that is easy to touch by the user, the target coordinate points may be clicked preferentially, that is, the click priority is higher; for a target coordinate point located in an area that is more difficult for the user to reach, then the click may be delayed, i.e., the click priority is lower. In this way, the clicking order of each target coordinate point can be arranged according to the position information of the target coordinate point, that is, the operation instructions corresponding to each target coordinate point can be ordered. Therefore, the effective rate of clicking each time can be improved, the fact that the clicked target coordinate point is the coordinate point corresponding to the subsequent multi-layer game interface is guaranteed, and the traversing quantity of the game interface is improved.

It should be noted that, for different types of mobile terminals, the screen sizes of the mobile terminals are often different, that is, for different mobile terminals, the locations of the areas that are easily touched by the user may also be different, so in this embodiment, specific screen information of the mobile terminal running the game to be tested may also be obtained in advance (for example, the screen information of the mobile terminal is determined according to the type of the mobile terminal), and then the priority of the target coordinate point may be determined together based on the screen information of the mobile terminal and the location information of the target coordinate point. In addition, for different types of mobile terminals (such as smartphones and tablet computers), the difference in screen size may be larger, so that the type corresponding to the mobile terminal running the game to be tested may be obtained in advance, then the model of the mobile terminal belonging to the type is obtained, and finally the screen size information of the mobile terminal is obtained.

Optionally, on the basis of the first embodiment corresponding to fig. 3, in an optional embodiment of the method for testing an interactive application provided in the embodiment of the present application, before obtaining position information of one or more target coordinate points in the screenshot, the method further includes:

acquiring a display type of the screenshot, wherein the display type comprises a horizontal screen display and a vertical screen display;

adding the one or more target coordinate points to a second processing queue according to the position information of the one or more target coordinate points in the screenshot, wherein the method comprises the following steps:

and adding the one or more target coordinate points into a second processing queue according to the position information of the one or more target coordinate points in the screenshot and the display type of the screenshot.

It will be appreciated that the location information of the user's touch may be different for mobile terminals of different screen sizes, and the location information of the user's touch may be different for the same mobile terminal when the user operates the game interface displayed on the vertical screen than when the user operates the game interface displayed on the horizontal screen.

In particular, referring to fig. 6 and 7, fig. 6 is a position priority exemplary diagram of a screenshot provided in an embodiment of the present application; fig. 7 is a position priority exemplary diagram of a vertical screen shot according to an embodiment of the present application. As shown in fig. 6, fig. 6 (a) shows a mobile terminal placed with two hands holding a vertical screen, it can be seen that in this case, the thumb of the user can easily touch the position below the screen of the mobile terminal, the thumb of the user needs to move a certain distance to touch the middle position of the screen of the mobile terminal, and for the position above the screen of the mobile terminal, the thumb of the user needs to have a longer distance to touch, in other words, for the lower position, the middle position and the upper position of the screen of the mobile terminal, the difficulty of touching by the user is respectively: easy, general and difficult; based on this, it can be seen that the graph (b) in fig. 6, where the graph (b) in fig. 6 shows the click operation priorities corresponding to different positions on the screen of the mobile terminal when the mobile terminal is displayed on the vertical screen, and it can be seen that when the target coordinate point is located at a position below the screen of the mobile terminal, the click operation priority of the target coordinate point is the first; when the target coordinate point is positioned at the middle position of the mobile terminal screen, the click operation priority of the target coordinate point is second; when the target coordinate point is located at a position above the mobile terminal screen, the click operation priority of the target coordinate point is third, that is, the priority of the target coordinate point can be set according to the difficulty level of the user touching different positions on the mobile terminal screen.

As shown in fig. 7, fig. 7 (a) shows a mobile terminal placed with both hands holding a horizontal screen, it can be seen that in this case, the thumb of the user can easily touch the middle position under the screen of the mobile terminal, and the thumb of the user needs to be curled to a certain extent to touch the positions on both sides of the screen of the mobile terminal, while for the position above the screen of the mobile terminal, the thumb of the user needs to have a longer distance to touch the positions; based on this, it can be seen that fig. 7 (b), where fig. 7 (b) shows the click operation priorities corresponding to different positions on the screen of the mobile terminal when the mobile terminal is displayed on the horizontal screen, and it can be seen that when the target coordinate point is located at a position below the screen of the mobile terminal and is offset from the middle, the click operation priority of the target coordinate point is the first; when the target coordinate point is positioned in the middle of the mobile terminal screen and is in a position close to the middle, the click operation priority of the target coordinate point is second; when the target coordinate point is positioned at the edge positions of the two sides below the mobile terminal screen, the click operation priority of the target coordinate point is third; when the target coordinate point is positioned at other positions of the mobile terminal screen, the point set operation priority of the target coordinate point is fourth; obviously, when the mobile terminal performs horizontal screen display, the priority of the target coordinate point can be set according to the difficulty level of the user touching different positions on the screen of the mobile terminal.

generating a random point set according to the first image, wherein the random point set comprises N random coordinate points, and the distance value between the N random coordinate points is larger than a preset threshold value;

replacing M random coordinate points in the random point set with target coordinate points in the target point set to obtain an updated random point set, wherein the target point set comprises M target coordinate points;

and generating an operation instruction set according to the updated random point set.

It can be understood that under some special situations, the target coordinate points detected by the detection model may not trigger an effective operation instruction, that is, after clicking the target coordinate points, the target coordinate points may not jump to a new game page, so in order to ensure that the test can be continuously performed, a certain alternative coordinate point may be preset, and thus, when the target coordinate points cannot trigger an effective operation instruction, the target coordinate points may also be triggered by clicking the alternative coordinate points.

Specifically, in this embodiment, a random point set may be generated when the first image is obtained, that is, a plurality of coordinate points in the first image are randomly selected, for example, N coordinate points are randomly selected, where N may be a numerical value (for example, 10) selected according to the actual game situation, and the distance between the N random coordinate points is greater than a preset threshold value, so as to ensure that the N random coordinate points can be dispersed in each position in the first image as much as possible; then, all the target coordinate points in the target point set are replaced by the first M random coordinate points in the random point set (i.e., the target point set is assumed to have M target coordinate points), so as to obtain an updated random point set (i.e., the first M random coordinate points are replaced by the random point set after the target coordinate points). And finally triggering clicking operation based on the updated random point set.

It should be noted that, in the case where N is greater than M, that is, the number of random coordinate points in the random point set is greater than the number of target coordinate points in the target point set, the M target coordinate points may be directly replaced by the first M random coordinate points; in the case where N is less than or equal to M, that is, the number of random coordinate points in the random point set is less than or equal to the number of target coordinate points in the target point set, the random point set may be directly replaced with the target point set.

Optionally, on the basis of the first embodiment corresponding to fig. 3, in an optional embodiment of the method for testing an interactive application provided in the embodiment of the present application, before acquiring the first image, the method further includes:

if a target page exists on an operation interface of the interactive application, acquiring first interface control information, wherein the target page is a sub page independent of a main interface of the interactive application;

if the first interface control information is matched with a preset regular expression, generating a target page cancel operation instruction;

and operating the target page according to the target page cancel operation instruction.

It will be appreciated that during game play, there may be some reasons why the interface displayed on the mobile terminal is not an in-game interface, e.g., the game interface on the mobile terminal is overlaid by a popup interface, or the game interface is overlaid by a notification bar interface, etc. In such cases, it is often necessary to process interfaces outside of these games, such as a pop-up interface or a notification bar interface.

In this embodiment, if it is detected that a sub-page (e.g., a popup interface) independent of the game main interface exists on the game running interface, current interface control information may be acquired by calling a corresponding application programming interface (Application Programming Interface, API) in the system at this time, if it is found that the acquired interface control information matches with a preset regular expression, it may be considered that a sub-page independent of the game main interface exists on the game running interface, so that a target page cancel operation instruction may be generated for the sub-page (i.e., the popup page) at this time, and a cancel popup page may be clicked according to the target page cancel operation instruction, that is, when an interface control matching with the preset popup regular expression is detected, the popup page may be cancelled by clicking a cancel button on the popup, thereby ensuring normal running of the game test.

if the running interface of the interactive application is a login interface, acquiring second interface control information;

if the second interface control information is matched with the preset login interface control information, login information is obtained;

and carrying out login operation on the login interface according to the login information.

In this embodiment, if it is found that the game enters the login interface, the login of the game may be completed by automatically inputting information such as an account password. Specifically, the interface control information of the current interface can be obtained by calling a corresponding API in the system, if the interface control information is matched with the preset login interface control information, the current interface is represented as a login interface, and at the moment, information (such as a login account number, a login password and the like) required by login can be obtained. Then, a login operation is automatically performed on the login interface based on the login information, for example, a login account and a login password are automatically input, and a login button is clicked, thereby completing the login operation.

For ease of understanding, the following describes in detail the testing method of the interactive application provided in the embodiment of the present application with reference to fig. 8 and 9. Referring to fig. 8, fig. 8 is a schematic diagram of a test flow of an interactive application according to an embodiment of the present application; fig. 9 is a schematic flow chart of generating a point set according to an embodiment of the present application.

In fig. 8, the test flow of the interactive application is specifically as follows:

801. starting a game;

802. judging whether the interface currently displayed by the mobile terminal is an interface in a game or not, specifically, judging by calling an API responded by a system to acquire current interface control information;

803. if the interface currently displayed by the mobile terminal is not an interface in the game, for example, the game interface is covered by a popup interface, is in a game login interface or is an interface of a desktop or a third party application program entering the mobile terminal, specifically, the processing of the current interface, for example, canceling the popup interface, automatically logging in the game or re-pulling up the game, returning to the game interface, etc., can be performed by calling a corresponding API of the system, and after the processing of the interface is completed, the process goes to the continuing execution step 807;

804. if the interface currently displayed by the mobile terminal is an interface in a game, acquiring a screenshot of the interface displayed on the screen of the current mobile terminal, namely acquiring a first image;

805. Generating a candidate point set according to the first image, namely generating a point set of operation to be clicked;

806. operating coordinate points one by one according to the sequence of coordinate points in the candidate point set based on the candidate point set, and jumping to a new game interface without operating each coordinate point once if an effective clicking operation is triggered after one coordinate point is operated;

807. judging whether the game test is finished, for example, judging whether the current test time reaches the preset test time (for example, 5 minutes), if so, ending the game test, otherwise, turning to a continuous execution step 802;

808. and ending the game test.

In addition, fig. 9 shows a process of how the candidate point set is specifically generated in step 805 in fig. 8.

In fig. 9, the process of generating the candidate point set is specifically as follows:

901. generating a random point set, specifically, for a first image with resolution ratio of w×h, randomly selecting N random coordinate points with a distance value between them greater than a preset threshold value S to form the random point set;

902. acquiring target coordinate points, specifically, detecting a first image through a detection model to obtain M target coordinate points;

903. converting the coordinate system, namely converting M target coordinate points from the image coordinate system into a screen coordinate system, obtaining M converted target coordinate points, and forming a target point set;

904. Updating the random point set, specifically, replacing the first M random coordinate points in the random point set with the target coordinate points in the target point set to obtain an updated random point set;

905. and obtaining a candidate point set, wherein the candidate point set is the updated random point set.

Specifically, referring to table 1, table 1 shows a comparison of the number of total scenes traversed by a test solution provided in the embodiment of the present application and a comparison of the number of clicks attempted by a login interface (a part of games do not need to be logged in) under the condition that the test duration is 5 minutes. The scene traversal number represents the depth of game test, and the login attempt times represent the effective rate of test execution operation.

TABLE 1

As can be seen from table 1, the solution provided in the embodiment of the present application improves the test depth of the operation effectiveness compared with the Monkey solution under the condition of no test script writing cost.

Referring to fig. 10, fig. 10 is a schematic diagram illustrating an embodiment of a testing device for an interactive application in an embodiment of the present application, where the testing device 100 for an interactive application in an embodiment of the present application includes:

The acquiring unit 1001 acquires a first image by an application, the first image being a screenshot in the running process of the interactive application;

the obtaining unit 1001 is further configured to obtain, by using the detection model, a target point set of the screenshot, where the target point set includes one or more target coordinate points;

a generating unit 1002, configured to generate an operation instruction set according to the target point set, where the operation instruction set includes one or more operation instructions corresponding to the target coordinate points;

the obtaining unit 1001 is further configured to obtain a test result according to the operation instruction set.

Optionally, on the basis of the embodiment corresponding to fig. 10, another embodiment of the testing device 100 for an interactive application provided in the embodiment of the present application further includes a joining unit 1003;

the obtaining unit 1001 is further configured to obtain a category corresponding to one or more target coordinate points;

a joining unit 1003, configured to join the one or more target coordinate points to the first processing queue according to the categories corresponding to the one or more target coordinate points;

the generating unit 1002 is further configured to generate an operation instruction set according to an order of one or more target coordinate points in the first processing queue, where operation instructions in the operation instruction set have a corresponding operation order.

Alternatively, in another embodiment of the interactive application testing apparatus 100 provided in the embodiment of the present application based on the embodiment corresponding to fig. 10,

the obtaining unit 1001 is further configured to obtain position information of one or more target coordinate points in the screenshot;

a joining unit 1003 further configured to join the one or more target coordinate points to the second processing queue according to the position information of the one or more target coordinate points in the screenshot;

the generating unit 1002 is further configured to generate an operation instruction set according to the order of the one or more target coordinate points in the second processing queue, where the operation instructions in the operation instruction set have a corresponding operation order.

the obtaining unit 1001 is further configured to obtain a display type of the screenshot, where the display type includes a horizontal screen display and a vertical screen display;

the adding unit 1003 is further configured to add the one or more target coordinate points to the second processing queue according to the location information of the one or more target coordinate points in the screenshot and the display type of the screenshot.

Optionally, on the basis of the embodiment corresponding to fig. 10, another embodiment of the testing device 100 for an interactive application provided in the embodiment of the present application further includes a replacing unit 1004;

the generating unit 1002 is further configured to generate a random point set according to the first image, where the random point set includes N random coordinate points, and a distance value between the N random coordinate points is greater than a preset threshold;

a replacing unit 1004, configured to replace M random coordinate points in the random point set with target coordinate points in the target point set, to obtain an updated random point set, where the target point set includes M target coordinate points;

the generating unit 1002 is further configured to generate an operation instruction set according to the updated random point set.

Optionally, on the basis of the embodiment corresponding to fig. 10, another embodiment of the testing device 100 for an interactive application provided in the embodiment of the present application further includes an operation unit 1005;

the obtaining unit 1001 is further configured to obtain first interface control information if a target page exists on an operation interface of the interactive application, where the target page is a sub page independent of a main interface of the interactive application;

the generating unit 1002 is further configured to generate a target page cancel operation instruction if the first interface control information matches with a preset regular expression;

The operation unit 1005 is further configured to operate on the target page according to the target page cancel operation instruction.

the obtaining unit 1001 is further configured to obtain second interface control information if the running interface of the interactive application is a login interface;

the obtaining unit 1001 is further configured to obtain login information if the second interface control information matches with preset login interface control information;

the operation unit 1005 is further configured to perform a login operation on the login interface according to the login information.

Fig. 11 is a schematic diagram of a server structure provided in an embodiment of the present application, where the server 1100 may vary considerably in configuration or performance, and may include one or more central processing units (central processing units, CPU) 1122 (e.g., one or more processors) and memory 1132, one or more storage media 1130 (e.g., one or more mass storage devices) storing applications 1142 or data 1144. Wherein the memory 1132 and the storage medium 1130 may be transitory or persistent. The program stored on the storage medium 1130 may include one or more modules (not shown), each of which may include a series of instruction operations on a server. Still further, the central processor 1122 may be provided in communication with a storage medium 1130, executing a series of instruction operations in the storage medium 1130 on the server 1100.

The server 1100 may also include one or more power supplies 1126, one or more wired or wireless network interfaces 1150, one or more input-output interfaces 1158, and/or one or more operating systems 1141, such as Windows Server, mac OS XTM, unixTM, linuxTM, freeBSDTM, etc.

In the present embodiment, the server 1300 includes CPU 1322 with the following functions:

and obtaining a test result according to the operation instruction set.

It will be clear to those skilled in the art that, for convenience and brevity of description, specific working procedures of the above-described systems, apparatuses and units may refer to corresponding procedures in the foregoing method embodiments, which are not repeated herein.

In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods may be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative, e.g., the division of the units is merely a logical function division, and there may be additional divisions when actually implemented, e.g., multiple units or components may be combined or integrated into another system, or some features may be omitted or not performed. Alternatively, the coupling or direct coupling or communication connection shown or discussed with each other may be an indirect coupling or communication connection via some interfaces, devices or units, which may be in electrical, mechanical or other form.

The units described as separate units may or may not be physically separate, and units shown as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

In addition, each functional unit in each embodiment of the present application may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit. The integrated units may be implemented in hardware or in software functional units.

The integrated units, if implemented in the form of software functional units and sold or used as stand-alone products, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present application may be embodied in essence or a part contributing to the prior art or all or part of the technical solution in the form of a software product stored in a storage medium, including several instructions to cause a computer device (which may be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in the embodiments of the present application. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a read-only memory (ROM), a random access memory (random access memory, RAM), a magnetic disk, or an optical disk, or other various media capable of storing program codes.

The above embodiments are merely for illustrating the technical solution of the present application, and not for limiting the same; although the present application has been described in detail with reference to the foregoing embodiments, it should be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit and scope of the corresponding technical solutions.

Claims

1. A method for testing an interactive application, comprising:

if a target page exists on an operation interface of the interactive application, a corresponding application programming interface in a system is called to acquire first interface control information, wherein the target page is a sub page independent of a main interface of the interactive application;

if the first interface control information is matched with a preset regular expression, determining that a sub-page independent of the main interface exists on the operation interface, and generating a target page cancelling operation instruction aiming at the target page;

operating the target page according to the target page cancel operation instruction so as to cancel the target page;

detecting a background frame through a detection model to obtain an effective click region so as to obtain a target point set of the first image, wherein the target point set comprises one or more target coordinate points, the effective click region is composed of the background frame and corresponding elements, one background frame generates one target coordinate point corresponding to the background frame, the target coordinate point corresponding to the background frame forms the target point set, the detection model is trained through sample data marked with the background frame, so that the background frame in the interface screenshot is identified through the detection model obtained through training, the background frame in the interface screenshot is obtained, and the priority of the target coordinate point is determined based on screen information of a terminal and position information of the target coordinate point;

and acquiring a test result according to the operation instruction set, wherein the operation instructions in the operation instruction set are triggered one by one, and when one of the operation instructions is triggered and the operation interface of the interactive application is caused to jump, the test result of the current operation interface is correspondingly acquired.

2. The method of claim 1, wherein prior to generating the set of operation instructions from the set of target points, the method further comprises:

acquiring categories corresponding to the one or more target coordinate points;

adding the one or more target coordinate points to a first processing queue according to the categories corresponding to the one or more target coordinate points;

the generating an operation instruction set according to the target point set comprises the following steps:

and generating an operation instruction set according to the sequence of the one or more target coordinate points in the first processing queue, wherein the operation instructions in the operation instruction set have corresponding operation sequences.

3. The method of claim 1, wherein prior to generating the set of operation instructions from the set of target points, the method further comprises:

acquiring position information of the one or more target coordinate points in the screenshot;

adding the one or more target coordinate points to a second processing queue according to the position information of the one or more target coordinate points in the screenshot;

And generating an operation instruction set according to the sequence of the one or more target coordinate points in the second processing queue, wherein the operation instructions in the operation instruction set have corresponding operation sequences.

4. The method of testing an interactive application according to claim 3, wherein prior to said obtaining location information of said one or more target coordinate points in said screenshot, said method further comprises:

the adding the one or more target coordinate points to a second processing queue according to the position information of the one or more target coordinate points in the screenshot, including:

and adding the one or more target coordinate points to the second processing queue according to the position information of the one or more target coordinate points in the screenshot and the display type of the screenshot.

5. The method of testing an interactive application according to claim 1, wherein prior to said generating a set of operational instructions from said set of target points, said method further comprises:

6. The method of testing an interactive application according to any one of claims 1 to 5, wherein prior to said capturing the first image, the method further comprises:

if the running interface of the interactive application is a login interface, second interface control information is acquired;

7. A test device for an interactive application, comprising:

the system comprises an acquisition unit, a control unit and a control unit, wherein the acquisition unit is used for calling a corresponding application programming interface in a system to acquire first interface control information if a target page exists on an operation interface of the interactive application, wherein the target page is a sub page independent of a main interface of the interactive application;

The generating unit is used for generating a target page cancelling operation instruction aiming at the target page if the first interface control information is matched with a preset regular expression and the fact that the sub page independent of the main interface exists on the operation interface is determined;

the operation unit is used for operating the target page according to the target page cancel operation instruction so as to cancel the target page;

the acquisition unit is also used for acquiring a first image, wherein the first image is a screenshot in the running process of the interactive application;

the acquisition unit is further configured to acquire an effective click region by detecting a background frame through a detection model to acquire a target point set of the screenshot, where the target point set includes one or more target coordinate points, the effective click region is formed by the background frame and corresponding elements, one background frame generates one target coordinate point corresponding to the background frame, the target coordinate point corresponding to the background frame forms the target point set, and training the detection model by using sample data marked with the background frame, so that the detection model obtained through training identifies the background frame in the interface screenshot, and the background frame in the interface screenshot is obtained, and the priority of the target coordinate point is determined based on screen information of the terminal and position information of the target coordinate point;

The generating unit is further configured to generate an operation instruction set according to the target point set, where the operation instruction set includes one or more operation instructions corresponding to the target coordinate point;

the obtaining unit is further configured to obtain a test result according to the operation instruction set, where operation instructions in the operation instruction set are triggered one by one, and when one of the operation instructions triggers and causes the running interface of the interactive application to jump, the test result of the current running interface is obtained correspondingly.

8. The interactive application testing apparatus according to claim 7, wherein the apparatus further comprises a joining unit;

the obtaining unit is further configured to obtain a category corresponding to the one or more target coordinate points;

the adding unit is used for adding the one or more target coordinate points to a first processing queue according to the categories corresponding to the one or more target coordinate points;

the generating unit is further configured to generate an operation instruction set according to the order of the one or more target coordinate points in the first processing queue, where the operation instructions in the operation instruction set have a corresponding operation order.

9. The interactive application testing apparatus according to claim 7, wherein the apparatus further comprises a joining unit;

the acquisition unit is further used for acquiring the position information of the one or more target coordinate points in the screenshot;

the adding unit is used for adding the one or more target coordinate points to a second processing queue according to the position information of the one or more target coordinate points in the screenshot;

10. The interactive application testing device according to claim 9, wherein the obtaining unit is further configured to obtain a presentation type of the screenshot, where the presentation type includes a horizontal screen presentation and a vertical screen presentation;

the adding unit is further configured to add the one or more target coordinate points to the second processing queue according to the position information of the one or more target coordinate points in the screenshot and the display type of the screenshot.

11. The interactive application testing apparatus according to claim 7, wherein the apparatus further comprises a replacement unit;

the generation unit is used for generating a random point set according to the first image, wherein the random point set comprises N random coordinate points, and the distance value between the N random coordinate points is larger than a preset threshold value;

the replacing unit is configured to replace M random coordinate points in the random point set with target coordinate points in the target point set, to obtain an updated random point set, where the target point set includes M target coordinate points;

the generating unit is further configured to generate an operation instruction set according to the updated random point set.

12. The device for testing an interactive application according to any one of claims 7 to 11, wherein the obtaining unit is further configured to obtain second interface control information if the running interface of the interactive application is a login interface; if the second interface control information is matched with the preset login interface control information, login information is obtained;

the operation unit is further used for performing login operation on the login interface according to the login information.

13. A server, comprising: memory, transceiver, processor, and bus system;

wherein the memory is used for storing programs;

operating the target page according to the target page cancel operation instruction so as to cancel the target page; acquiring a first image, wherein the first image is a screenshot in the running process of the interactive application;

detecting a background frame through a detection model to obtain an effective click region so as to obtain a target point set of the screenshot, wherein the target point set comprises one or more target coordinate points, the effective click region is composed of the background frame and corresponding elements, one background frame generates one target coordinate point corresponding to the background frame, the target coordinate point corresponding to the background frame forms the target point set, the detection model is trained through sample data marked with the background frame, so that the background frame in the interface screenshot is identified through the detection model obtained through training, the background frame in the interface screenshot is obtained, and the priority of the target coordinate point is determined based on screen information of a terminal and position information of the target coordinate point;

acquiring a test result according to the operation instruction set, wherein the operation instructions in the operation instruction set are triggered one by one, and when one of the operation instructions is triggered and the operation interface of the interactive application is caused to jump, the test result of the current operation interface is correspondingly acquired;

the bus system is used for connecting the memory and the processor so as to enable the memory and the processor to communicate.

14. A computer readable storage medium comprising instructions which, when run on a computer, cause the computer to perform the method of any one of claims 1 to 6.