CN111857719A

CN111857719A - Data processing method, device, electronic equipment and medium

Info

Publication number: CN111857719A
Application number: CN202010762084.8A
Authority: CN
Inventors: 蔡天勤; 张钊; 丁光磊
Original assignee: Beijing ByteDance Network Technology Co Ltd
Current assignee: Beijing ByteDance Network Technology Co Ltd
Priority date: 2020-07-31
Filing date: 2020-07-31
Publication date: 2020-10-30
Anticipated expiration: 2040-07-31
Also published as: CN111857719B

Abstract

The embodiment of the disclosure provides a data processing method, a data processing device, electronic equipment and a medium. The method comprises the following steps: acquiring page text information of a sample application program, wherein the page text information comprises page layout description information and corresponding page action operation information; and analyzing and generating control layout information and corresponding action position information of each of a plurality of continuous pages of the sample application program according to the page text information of the sample application program to serve as training samples, wherein the training samples are used for training test models, and the test models are used for carrying out automatic testing on the application program to be tested. The embodiment of the disclosure improves the processing efficiency of the model and reduces the time consumption for training the model.

Description

Data processing method, device, electronic equipment and medium

Technical Field

The present disclosure relates to the field of computer technologies, and in particular, to a data processing method, an apparatus, an electronic device, and a medium.

Background

With the continuous development of computer technology, various Applications (APPs) are increasing, and the APPs gradually become a tool widely used in daily life. In the using process of the APP, the training sample can be used for training the neural network model, and the trained neural network model is used for testing the stability of the APP.

In the prior art, a plurality of display pages of an App can be acquired, and for each display page, on one hand, each control contained on the display page is identified through an image identification algorithm to obtain a control layout; and on the other hand, manually marking the action position information for operating the display page. And forming a training sample by using the control layout diagrams of the plurality of display pages and the corresponding action position information, and training the neural network model by using the training sample.

When the training sample is used for training the neural network model, because the training sample comprises a plurality of control layout diagrams, under normal conditions, the number of the control layout diagrams is tens of thousands, even hundreds of thousands or millions, and the processing speed of the neural network on the pictures is relatively slow, the processing speed of the neural network can be greatly reduced when the neural network is trained by using the control layout diagrams, so that the time consumption of the process of training the neural network is long.

Disclosure of Invention

The present disclosure provides a data processing method, an apparatus, an electronic device, and a medium, which can solve the technical problem in the prior art that a long time is consumed when a neural network is trained.

In a first aspect, a data processing method is provided, and the method includes:

acquiring page text information of a sample application program, wherein the page text information comprises page layout description information and corresponding page action operation information;

and analyzing and generating control layout information and corresponding action position information of each of a plurality of continuous pages of the sample application program according to the page text information of the sample application program to serve as training samples, wherein the training samples are used for training test models, and the test models are used for carrying out automatic testing on the application program to be tested.

In a second aspect, there is provided a data processing apparatus, the apparatus comprising:

the acquisition module is used for acquiring page text information of the sample application program, wherein the page text information comprises page layout description information and corresponding page action operation information;

and the generating module is used for analyzing and generating control layout information and corresponding action position information of each of a plurality of continuous pages of the sample application program according to the page text information of the sample application program, and the control layout information and the corresponding action position information are used as training samples, the training samples are used for training a test model, and the test model is used for carrying out automatic test on the application program to be tested.

In a third aspect, an electronic device is provided, which includes:

an electronic device, comprising:

one or more processors;

a memory;

one or more application programs, wherein the one or more application programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs configured to: the corresponding operations of the data processing method according to the embodiment of the method are performed.

In a fourth aspect, a computer-readable medium is provided, on which a computer program is stored which, when being executed by a processor, carries out the data processing method shown in the method embodiments.

The technical scheme provided by the disclosure has the following beneficial effects:

compared with the prior art, the method and the device for processing the data, the electronic equipment and the medium have the advantages that page text information of a sample application program is obtained, the page text information comprises page layout description information and corresponding page action operation information, the control layout information and the corresponding action position information of a plurality of continuous pages of the sample application program are analyzed and generated according to the page text information of the sample application program and serve as training samples, the training samples are used for training test models, and the test models are used for carrying out automatic testing on the application program to be tested. Because the control layout information and the action position information are extracted from the page text information and are both text information, the processing speed of the network on the text information is relatively high, and therefore, the processing efficiency of the model can be greatly improved by using the training sample training model formed by the control layout information and the action position information, and the time consumption of the training model is reduced.

Drawings

The above and other features, advantages and aspects of various embodiments of the present disclosure will become more apparent by referring to the following detailed description when taken in conjunction with the accompanying drawings. Throughout the drawings, the same or similar reference numbers refer to the same or similar elements. It should be understood that the drawings are schematic and that elements and features are not necessarily drawn to scale.

Fig. 1 is a schematic flow chart of a data processing method according to an embodiment of the present disclosure;

fig. 2 is a schematic flowchart of a method for testing an application according to an embodiment of the present disclosure;

fig. 3 is a schematic diagram of layout information of a control provided in an embodiment of the present disclosure;

fig. 4 is a schematic diagram of a control layout and action position diagram according to an embodiment of the present disclosure;

fig. 5 is a schematic structural diagram of a data processing apparatus according to an embodiment of the present disclosure;

fig. 6 is a schematic structural diagram of an electronic device according to an embodiment of the present disclosure.

Detailed Description

Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. While certain embodiments of the present disclosure are shown in the drawings, it is to be understood that the present disclosure may be embodied in various forms and should not be construed as limited to the embodiments set forth herein, but rather are provided for a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the disclosure are for illustration purposes only and are not intended to limit the scope of the disclosure.

It should be understood that the various steps recited in the method embodiments of the present disclosure may be performed in a different order, and/or performed in parallel. Moreover, method embodiments may include additional steps and/or omit performing the illustrated steps. The scope of the present disclosure is not limited in this respect.

The term "include" and variations thereof as used herein are open-ended, i.e., "including but not limited to". The term "based on" is "based, at least in part, on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". Relevant definitions for other terms will be given in the following description.

It should be noted that the terms "first", "second", and the like in the present disclosure are only used for distinguishing the devices, modules or units, and are not used for limiting the devices, modules or units to be different devices, modules or units, and also for limiting the sequence or interdependence relationship of the functions executed by the devices, modules or units.

It is noted that references to "a", "an", and "the" modifications in this disclosure are intended to be illustrative rather than limiting, and that those skilled in the art will recognize that "one or more" may be used unless the context clearly dictates otherwise.

The names of messages or information exchanged between devices in the embodiments of the present disclosure are for illustrative purposes only, and are not intended to limit the scope of the messages or information.

The following describes the technical solutions of the present disclosure and how to solve the above technical problems in specific embodiments. The following several specific embodiments may be combined with each other, and details of the same or similar concepts or processes may not be repeated in some embodiments. Embodiments of the present disclosure will be described below with reference to the accompanying drawings.

The embodiment of the present disclosure provides a data processing method, which may be executed by an electronic device, where the electronic device may be a terminal device or a server, and is not limited herein. As shown in fig. 1, the method includes step S101 and step S102.

Step S101, page text information of a sample application program is obtained, and the page text information comprises page layout description information and corresponding page action operation information.

Wherein the at least one sample application includes or does not include an application under test.

For each page displayed in the historical running process of at least one sample application program, the control layout description information of the page and the page action operation information of the page can be respectively obtained by using a tool.

The historical operation process can be a test process and/or an actual use process.

Step S102, analyzing and generating control layout information and corresponding action position information of a plurality of continuous pages of the sample application program according to the page text information of the sample application program, wherein the control layout information and the corresponding action position information are used as training samples, the training samples are used for training test models, and the test models are used for carrying out automatic test on the application program to be tested.

In the embodiment of the present disclosure, the test model may be a neural network model, and the neural network model may be a reinforcement learning model, and the reinforcement learning model may be a Proximal Policy Optimization (PPO) model, or a PPO2 model, etc., which is not limited herein.

Compared with the prior art, the data processing method includes the steps of obtaining page text information of a sample application program, wherein the page text information comprises page layout description information and corresponding page action operation information, analyzing and generating control layout information and corresponding action position information of a plurality of continuous pages of the sample application program according to the page text information of the sample application program, and using the control layout information and the corresponding action position information as training samples, wherein the training samples are used for training test models, and the test models are used for carrying out automatic testing on the application program to be tested. Because the control layout information and the action position information are extracted from the page text information and are both text information, the processing speed of the network on the text information is relatively high, and therefore, the processing efficiency of the model can be greatly improved by using the training sample training model formed by the control layout information and the action position information, and the time consumption of the training model is reduced.

In another possible implementation manner of the embodiment of the present disclosure, in step S101, acquiring page text information of a sample application program specifically includes:

and acquiring text information of an extensible markup language XML (extensive markup language) class of the sample application program or text information of a JS object numbered musical notation Json class as page text information.

In the embodiment of the present disclosure, the page text information may be text information of an Extensible markup language (XML) class, or may be text information of a JavaScript object notation (Json) class.

The page text information comprises page layout description information and corresponding page action operation information, so that the page layout description information can be XML-type text information or Json-type text information; the page action operation information may be text information of an XML class or text information of a Json class.

Another possible implementation manner of the embodiment of the present disclosure, step S102, that is, analyzing and generating control layout information and corresponding action position information of each of a plurality of continuous pages of a sample application according to page text information of the sample application, as a training sample, may specifically include steps S1021 to S1024.

And S1021, extracting a text control, a picture control and other visual controls of the page as three-channel information of the page according to the page layout description information of the sample application program, and generating control layout information of the page based on the three-channel information of the page.

The text type control is a TextView type control, the picture type control is an ImageView type control, and the other visual type controls are other View type controls except the TextView type control and the ImageView type control.

The View type control is common and has obvious characteristics, the page can be well described, the View type control is divided into a TextView type control, an ImageView type control and other View type controls except the TextView type control and the ImageView type control, and the page layout can be well embodied.

By adopting the three types of controls, the important structural features of the page can be kept as much as possible, and when a test model is trained by using training samples containing the three types of controls, the prediction accuracy of the model can be improved; meanwhile, by adopting the three types of controls, the non-important structural features of the page can be ignored, and when the test model is trained by using the training samples containing the three types of controls, the calculation load of the model can be reduced.

It should be noted that the control types in the page may include, in addition to the View class control, a Layout (Layout) class control, a Frame (Frame) class control, and a Button (Button) class control, which is not limited in the embodiment of the present disclosure.

It is understood that the channel information of the page may not be limited to three, and may be more or less according to actual requirements. For example, the channel information may include not only the three types of controls, but also at least one of a Layout type control, a Frame type control, and a Button type control; or the channel information does not contain one or more of the three types of controls, or the three types of controls can be integrated into two types of controls, for example, the TextView type control and the ImageView type control are classified into one type, and the View type controls other than the TextView type control and the ImageView type control are classified into one type.

Further, control layout information for the page may be generated based on the three channel information for the page.

Specifically, in step S1021, extracting a text control, a picture control, and other visual controls of the page as three-channel information of the page according to the page layout description information of the sample application program, and generating control layout information of the page based on the three-channel information of the page, which may specifically include:

drawing a blank picture according to a preset resolution; traversing the controls in the page layout description information by adopting a tree subsequent traversal algorithm, and extracting text controls, picture controls and other visual controls of the page as three-channel information of the page, wherein the three-channel information of the page comprises controls corresponding to leaf nodes and/or controls corresponding to non-leaf nodes; and filling the three-channel information of the page into the blank picture according to a preset resolution ratio to obtain the control layout information of the page.

In the embodiment of the disclosure, the page layout description information has a tree structure, the tree structure is used for describing a parent-child relationship between the controls in the page layout description information, and each node in the tree structure is a control in the page layout description information.

The parent-child relationship among the controls is the inclusion relationship among the controls, the outer controls comprise inner controls, the outer controls serve as parent nodes, and the inner controls serve as child nodes. For example, if control 1 includes control 2, control 1 is an outer-layer control, control 2 is an inner-layer control, control 1 is a parent node, and control 2 is a child node.

And traversing each node in the tree structure, and taking the control corresponding to the traversed current node as a control to be selected. A tree's sequential traversal algorithm may be employed to sequentially traverse each node in the tree structure. The tree's subsequent traversal algorithm follows the idea of first traversing left, then right and then root, i.e. first traversing the left sub-tree, then traversing the right sub-tree, and finally accessing the root node. And when the current node is traversed, taking the control corresponding to the current node as the control to be selected.

Further, if the control to be selected is determined to be one type of control in the three-channel information of the page, the control to be selected is taken as a target control, and the position information and the executable operation type of the target control are extracted from the code information corresponding to the target control in the page layout description information.

For example, the page layout description information (denoted as example 1) includes code information corresponding to the target control as shown below:

"node":{

"@index":"1",

"@ text": fuel,

"@resource-id":"com.ss.android.article.local:id/d4_",

"@class":"android.widget.TextView",

"@clickable":"true",

"@long-clickable":"false",

"@bounds":"[585,387][777,452]"

}

it should be noted that the above code information is only used as an example, and in an actual application process, the code information corresponding to the target control may be completely the same as the above code information, or completely different from the above code information, or partially the same as the above code information. The code information corresponding to the target control may include not only the above-mentioned code information but also other code information, for example, may include package code information "@ package" @ com.s.android.local "and the like.

For code information corresponding to the target control in example 1, it may be determined that the operation type executable by the target control includes a click type according to "@ clickable": true ", and it may be determined that the operation type executable by the target control does not include a long-press type according to" @ long-clickable ": false"; the position information of the target control can be determined according to "@ bounds": 585,387] [777,452] ", namely the target control is a rectangle, and the coordinates of two opposite corner points of the rectangle are [585,387] and [777,452], respectively. Therefore, the executable operation type of the target control and the position information of the target control can be extracted from the code information corresponding to the target control.

It should be noted that each node in the tree structure corresponds to a control in the page layout description information, and the tree structure includes a leaf node (that is, there is no corresponding child node) and a non-leaf node (that is, there is a corresponding child node), so each node in the tree structure may be a leaf node or a non-leaf node. And because the control corresponding to each node in the tree structure is the control to be selected, and at least one target control can be selected from each control to be selected, the at least one target control can include a control corresponding to a leaf node and/or a control corresponding to a non-leaf node, and therefore the three-channel information of the page includes the control corresponding to the leaf node and/or the control corresponding to the non-leaf node.

In the embodiment of the disclosure, when the target control is a control corresponding to a non-leaf node, it indicates that the target control includes at least one control, and the stacking condition between the controls can be better shown, so that the stacking condition of the pages is shown, and the information of the pages can be retained as much as possible. The training samples contain control layout information of the controls corresponding to the non-leaf nodes, so that the test model constructed by the training samples can predict the operation behaviors on the page more accurately.

In the embodiment of the present disclosure, the determining that the control to be selected is a type of control in three-channel information of the page includes:

and if the code information of the control to be selected meets the first condition, determining that the control to be selected is one type of control in the three-channel information of the page.

The first condition is that the code information of the to-be-selected control contains description information of a control type to which the to-be-selected control belongs, and the control type to which the to-be-selected control belongs is one of three channels of information of a page.

In the embodiment of the present disclosure, the description information of the control type to which the control to be selected belongs is class information, and for example, the code information in example 1 includes "@ class": android.

Recording one control type in three channels of information of a page as a control type a, if the control type to which the control to be selected belongs is the control type a, the description information of the control type to which the control to be selected belongs is the description information of the control type a, and the code information of the control to be selected contains the description information of the control type a.

For example, the three channels of information of the page include a TextView class control, and the code information of the control to be selected includes description information "" @ class "" of the TextView class control, and "android.

In the actual use process, the control type to which the control to be selected belongs, which is determined by the description information of the control type to which the control to be selected belongs, may be any one of a TextView type control, an ImageView type control, other View type controls except the TextView type control and the ImageView type control, a Layout type control, a Frame type control, and a Button type control.

In the embodiment of the disclosure, since the target control is a type of control in the three-channel information of the page, further, the executable operation type and the position information of each target control are extracted according to the above manner, and the executable operation type and the position information of each target control are utilized to fill in the blank picture according to the preset resolution, so that the control layout information of the page can be obtained, that is, the three-channel information of the page is filled in the blank picture according to the preset resolution to obtain the control layout information of the page.

Specifically, the position information of each target control can be utilized to fill each target control into the blank picture according to a preset resolution, so as to obtain the control layout information of the page. For example, the preset resolution may be 288 × 166.

In step S1022, the action position information for the page is extracted according to the page action operation information of the sample application.

The page action operation information comprises code information corresponding to an operation position for operating the page and code information corresponding to an operation type for operating the page.

In the embodiment of the present disclosure, the operation position of operating the page may be extracted from the code information corresponding to the operation position of operating the page, the operation type of operating the page may be extracted from the code information corresponding to the operation type of operating the page, and the operation position information for the page and the operation type of operating the page may be used to configure the operation position information for the page.

For example, the page action operation information of the page includes "touch _ y": 21"," touch _ x ": 67", "actionType": clock ", wherein," touch _ y ": 21", "touch _ x": 67 "" represents code information corresponding to an operation position for operating the page, specifically, "touch _ y": 21 "" represents that the ordinate of the operation position is 21, and "touch _ x": 67 "" represents that the abscissa of the operation position is 67, that is, the operation position is (67, 21); "actionType": clock "" represents code information corresponding to an operation type of operating a page, and specifically, "actionType": clock "" represents that the operation type is clock, that is, the operation type is a CLICK type; the operation position information is constituted by the operation position (67,21) and the operation type CLICK.

The page operation information of the page may include code information corresponding to an operation position of the operation on the page and code information corresponding to an operation type of the operation on the page, and may further include other code information, such as code information of a control corresponding to the page operation on the page.

For example, the page action operation information of the page further includes "element _ width": 88"," element _ height ": 88", and "indicates that the width and height of the control corresponding to the page operation are both 88, that is, the size of the control corresponding to the page operation is 88.

And step S1023, integrating the control layout information and the corresponding action position information of the single page to generate a control layout and action position graph of the single page.

Specifically, in step S1023, the control layout information and the corresponding action position information of a single page are integrated to generate a control layout and action position map of the single page, which specifically includes:

and aiming at a single page, marking action position information corresponding to the page in the control layout information of the page to obtain a control layout and action position diagram of the single page.

In the embodiment of the disclosure, the action position information can be marked in the control layout information of the page according to the action position information of the page, so as to obtain the control layout and the action position diagram of the page. As shown in fig. 4, the page pointed by the reference numeral 41 is a control layout and action position diagram, the control layout and action position diagram is marked with the area pointed by the reference numeral 45, and the area pointed by the reference numeral 45 is action position information corresponding to the page.

And step S1024, taking the control layout and the action position diagram of a plurality of continuous pages as a training sample.

In the embodiment of the present disclosure, in a plurality of continuous pages, the specific numerical values of the plurality of pages are not limited, and may be two or more.

A preferred implementation manner, using a control layout and an action position diagram of a plurality of continuous pages as a training sample, may specifically include: and taking the control layout and action position graphs of the four continuous pages as training samples.

For example, operation 1 on page 1 may change to page 2, operation 2 on page 2 may change to page 3, operation 3 on page 3 may change to page 4, operation 4 on page 4 may change to page 5.

For example, in fig. 4, the page indicated by the reference numeral 41 is a control layout and action position diagram of the page 1 labeled with the action position information of the operation 1, where the action position information of the operation 1 is the region indicated by the reference numeral 45. Similarly, the page indicated by reference numeral 42 is the control layout and action position map of the page 2 labeled with the action position information of the operation 2, the page indicated by reference numeral 43 is the control layout and action position map of the page 3 labeled with the action position information of the operation 3, and the page indicated by reference numeral 44 is the control layout and action position map of the page 4 labeled with the action position information of the operation 4. The test model may be trained using the pages pointed to by reference numerals 41-44 in fig. 4.

In another possible implementation manner of the embodiment of the present disclosure, one channel information of a page may correspond to one color information. And filling the three-channel information of the page into the blank picture according to the corresponding color information and the preset resolution to obtain the control layout information of the page.

For example, the TextView class control may be set to red, the ImageView class control may be set to green, and the other View class controls other than the TextView class control and the ImageView class control may be set to blue. Filling the textView type control into a blank picture according to red and a preset resolution; filling the ImageView control into a blank picture according to green and a preset resolution; filling other View type controls except the TextView type control and the ImageView type control into a blank picture according to the blue and the preset resolution ratio to obtain the control layout information of the page.

When at least one type of control in the three-channel information of the page has control overlap, filling the three-channel information of the page into the blank picture according to a preset resolution to obtain control layout information of the page, which may specifically include:

respectively determining the maximum overlapping quantity corresponding to the text type control, the picture type control and other visual type controls; determining the color information corresponding to the text type control, the picture type control and other visual type controls according to the maximum overlapping quantity corresponding to the text type control, the picture type control and the other visual type controls; and filling the three-channel information of the page into the blank picture according to the corresponding color information and the preset resolution to obtain the control layout information of the page.

In the embodiment of the disclosure, the color information of the type of control is determined according to the maximum overlapping number of the type of control, so that the control overlapping state of the type of control can be reflected in the control layout information of the rendered page through the brightness degree of the color, and the page characteristics can be captured as carefully as possible.

For example, if the maximum overlapping number of TextView class controls in the same position in the page is 5, it may be determined that the color information of the TextView class controls is (51, 0, 0), in the control layout information of the page obtained by rendering (51, 0, 0), the position corresponding to the maximum overlapping number is rendered 5 times, and the color at the position corresponding to the maximum overlapping number is (255, 0, 0), that is, red, through the color superposition of the 5 times of rendering, and the position corresponding to the maximum overlapping number is brighter than the color at the position corresponding to the non-maximum overlapping number.

As shown in fig. 3, in the control layout information of the page, the area indicated by 3b is brighter than the area indicated by 3a, and therefore, the number of controls overlapping in the area indicated by 3b is greater than the number of controls overlapping in the area indicated by 3 a.

In the embodiment of the present disclosure, the three-channel information of the page may only include a control corresponding to a leaf node, may also only include a control corresponding to a non-leaf node, and may also include a control corresponding to a leaf node and a control corresponding to a non-leaf node. Because the control corresponding to the non-leaf node comprises the control corresponding to the leaf node, under the condition that the non-leaf node is considered, the overlapping condition of a certain type of control may occur in the control layout information of the page, and under the condition that the non-leaf node is not considered, the overlapping condition of a certain type of control cannot occur in the control layout information of the page.

In the embodiment of the disclosure, the color information of a certain type of control is determined according to the maximum overlapping number of the control, so that the control overlapping state of the page can be reflected by the brightness degree of the color in the page layout information of the rendered page.

In another possible implementation manner of the embodiment of the present disclosure, as shown in fig. 2, the method further includes steps S201 to S203.

Step S201, acquiring control layout information of four continuous pages and action position information of the first three pages in the application program to be tested, as input information of the test model.

Actually, when an application program to be tested is tested, there are two cases of input of the test model, which are respectively denoted as case B1 and case B2.

In case B1, the control layout information of the current page may be input to the test model as input information of the test model.

For example, the current page is the 5 th page, and the input of the test model may be the control layout information of the 5 th page.

In case B2, the control layout information of the current page, the control layout information of at least one preceding page, and the action position information of each preceding page may be input to the test model as input information of the test model.

The previous page is any page except the current page in the process of changing from the preset page to the current page, the preset page is a preset number of pages before the current page, and the value of the preset number is a positive integer greater than or equal to 1. I.e. each preceding page is a continuous page with the current page.

For example, if the preset page is the 1 st page and the current page is the 5 th page, the previous page has 4 pages, which are the 1 st to 4 th pages respectively. Based on this, the input of the test model may be the control layout information of the 5 th page, the control layout information of each of the 1 st to 4 th pages, and the action position information of each of the 1 st to 4 th pages.

In a preferred implementation, the number of the preceding pages is three, that is, the control layout information of four consecutive pages and the action position information of the preceding three pages are used as the input information of the test model.

Step S202, action position prediction information aiming at the last page in the four pages is obtained through prediction by the test model based on the input information.

And inputting the input information into a test model, wherein the test model can output the profit value of each executable operation contained in the current page, the profit value is in direct proportion to the code coverage rate, and the executable operation with the maximum profit value can be selected as the action position prediction information. The code coverage rate can measure the test integrity of the application program to be tested and represents the proportion of the tested codes of the application program to be tested.

In the embodiments of the present disclosure and subsequent embodiments, the page includes at least one target control, and for any target control, the target control corresponds to at least one executable operation type, for example, one control corresponds to three executable operation types, which are a click type, a long press type, and a slide type.

Step S203 controls the test for the application to be tested based on the motion position prediction information.

For the current page of the application program to be tested, the action position prediction information may be executed on the current page to obtain a next page, and the next page is used as the current page, and step S201 and subsequent steps are executed in a loop until a test stop condition is met, where the test stop condition may be that a preset duration is reached, or a code coverage rate reaches a preset code coverage rate value, or a preset loop number value is reached, and the like, and is not limited herein.

The data processing method is specifically described from the perspective of the method steps, and the data processing apparatus is introduced from the perspective of a virtual module or a virtual unit, which is specifically as follows:

the embodiment of the present disclosure provides a data processing apparatus 50, and as shown in fig. 5, the data processing apparatus 50 may include: an acquisition module 501 and a generation module 502, wherein,

the obtaining module 501 is configured to obtain page text information of a sample application, where the page text information includes page layout description information and corresponding page action operation information.

The generating module 502 is configured to parse and generate control layout information and corresponding action position information of each of a plurality of continuous pages of the sample application program according to the page text information of the sample application program, where the control layout information and the corresponding action position information are used as training samples, the training samples are used for training a test model, and the test model is used for performing an automated test on the application program to be tested.

In another possible implementation manner of the embodiment of the present disclosure, the generating module 502 is specifically configured to:

extracting a text control, a picture control and other visual controls of the page as three-channel information of the page according to the page layout description information of the sample application program, and generating control layout information of the page based on the three-channel information of the page;

extracting action position information aiming at the page according to the page action operation information of the sample application program;

integrating the control layout information and the corresponding action position information of a single page to generate a control layout and action position diagram of the single page;

and taking the control layout and the action position diagram of a plurality of continuous pages as a training sample.

In another possible implementation manner of the embodiment of the present disclosure, the generating module 502 is specifically configured to, when extracting a text type control, a picture type control, and other visual type controls of a page as three-channel information of the page according to the page layout description information of the sample application program, and generating the control layout information of the page based on the three-channel information of the page:

drawing a blank picture according to a preset resolution;

traversing the controls in the page layout description information by adopting a tree subsequent traversal algorithm, and extracting text controls, picture controls and other visual controls of the page as three-channel information of the page, wherein the three-channel information of the page comprises controls corresponding to leaf nodes and/or controls corresponding to non-leaf nodes;

filling the three-channel information of the page into a blank picture according to a preset resolution to obtain control layout information of the page;

correspondingly, when the generating module 502 integrates the control layout information and the corresponding action position information of a single page to generate the control layout and action position map of the single page, it is specifically configured to:

In another possible implementation manner of the embodiment of the present disclosure, when at least one type of control in the three-channel information of the page has control overlap, the generating module 502 is specifically configured to, when filling the three-channel information of the page into a blank picture according to a preset resolution to obtain control layout information of the page:

respectively determining the maximum overlapping quantity corresponding to the text type control, the picture type control and other visual type controls;

determining the color information corresponding to the text type control, the picture type control and other visual type controls according to the maximum overlapping quantity corresponding to the text type control, the picture type control and the other visual type controls;

and filling the three-channel information of the page into the blank picture according to the corresponding color information and the preset resolution to obtain the control layout information of the page.

In another possible implementation manner of the embodiment of the present disclosure, the generating module 502 is specifically configured to, when the control layout and the action position diagram of a plurality of continuous pages are used as a training sample:

and taking the control layout and action position graphs of the four continuous pages as training samples.

In another possible implementation manner of the embodiment of the present disclosure, the obtaining module is specifically configured to:

In another possible implementation manner of the embodiment of the present disclosure, the data processing apparatus 50 further includes:

the method comprises the steps of obtaining a module, wherein the module is used for obtaining control layout information of four continuous pages and action position information of the first three continuous pages in an application program to be tested as input information of a test model;

the prediction module is used for predicting action position prediction information aiming at the last page in the four pages based on the input information through a test model;

and the control module is used for controlling the test of the application program to be tested based on the motion position prediction information.

The data processing apparatus 50 of this embodiment can execute the data processing method shown in the above method embodiments of the present disclosure, and the implementation principles thereof are similar, and are not described herein again.

Compared with the prior art, the data processing device provided by the embodiment of the disclosure obtains page text information of a sample application program, wherein the page text information comprises page layout description information and corresponding page action operation information, and analyzes and generates control layout information and corresponding action position information of a plurality of continuous pages of the sample application program according to the page text information of the sample application program, and the control layout information and the corresponding action position information are used as training samples, the training samples are used for training test models, and the test models are used for carrying out automatic testing on the application program to be tested. Because the control layout information and the action position information are extracted from the page text information and are both text information, the processing speed of the network on the text information is relatively high, and therefore, the processing efficiency of the model can be greatly improved by using the training sample training model formed by the control layout information and the action position information, and the time consumption of the training model is reduced.

The data processing apparatus of the present disclosure is described above from the perspective of a virtual module or a virtual unit, and the electronic device of the present disclosure is described below from the perspective of a physical apparatus.

An electronic device is provided in the embodiments of the present disclosure, and referring to fig. 6 below, a schematic structural diagram of an electronic device (such as a terminal device or a server) 600 suitable for implementing the embodiments of the present disclosure is shown, where the electronic device includes:

one or more processors;

a memory;

one or more application programs, wherein the one or more application programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs configured to: the data processing method shown according to the method embodiment is performed.

The terminal device in the embodiments of the present disclosure may include, but is not limited to, a mobile terminal such as a mobile phone, a notebook computer, a digital broadcast receiver, a PDA (personal digital assistant), a PAD (tablet computer), a PMP (portable multimedia player), a vehicle terminal (e.g., a car navigation terminal), and the like, and a stationary terminal such as a digital TV, a desktop computer, and the like. The electronic device shown in fig. 6 is only an example, and should not bring any limitation to the functions and the scope of use of the embodiments of the present disclosure.

The electronic device includes: a memory and a processor, wherein the processor may be referred to as a processing device 601 described below, and the memory may include at least one of a Read Only Memory (ROM)602, a Random Access Memory (RAM)603, and a storage device 608, which are described below:

as shown in fig. 6, electronic device 600 may include a processing means (e.g., central processing unit, graphics processor, etc.) 601 that may perform various appropriate actions and processes in accordance with a program stored in a Read Only Memory (ROM)602 or a program loaded from a storage means 608 into a Random Access Memory (RAM) 603. In the RAM 603, various programs and data necessary for the operation of the electronic apparatus 600 are also stored. The processing device 601, the ROM 602, and the RAM 603 are connected to each other via a bus 604. An input/output (I/O) interface 605 is also connected to bus 604.

Generally, the following devices may be connected to the I/O interface 605: input devices 606 including, for example, a touch screen, touch pad, keyboard, mouse, camera, microphone, accelerometer, gyroscope, etc.; output devices 607 including, for example, a Liquid Crystal Display (LCD), a speaker, a vibrator, and the like; storage 608 including, for example, tape, hard disk, etc.; and a communication device 609. The communication means 609 may allow the electronic device 600 to communicate with other devices wirelessly or by wire to exchange data. While fig. 6 illustrates an electronic device 600 having various means, it is to be understood that not all illustrated means are required to be implemented or provided. More or fewer devices may alternatively be implemented or provided.

In particular, according to an embodiment of the present disclosure, the processes described above with reference to the flowcharts may be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product comprising a computer program carried on a non-transitory computer readable medium, the computer program containing program code for performing the method illustrated by the flow chart. In such an embodiment, the computer program may be downloaded and installed from a network via the communication means 609, or may be installed from the storage means 608, or may be installed from the ROM 602. The computer program, when executed by the processing device 601, performs the above-described functions defined in the methods of the embodiments of the present disclosure.

It should be noted that the computer readable medium in the present disclosure can be a computer readable signal medium or a computer readable storage medium or any combination of the two. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples of the computer readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the present disclosure, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In contrast, in the present disclosure, a computer readable signal medium may comprise a propagated data signal with computer readable program code embodied therein, either in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: electrical wires, optical cables, RF (radio frequency), etc., or any suitable combination of the foregoing.

In some embodiments, the clients, servers may communicate using any currently known or future developed network protocol, such as HTTP (HyperText transfer protocol), and may be interconnected with any form or medium of digital data communication (e.g., a communications network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), the Internet (e.g., the Internet), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any currently known or future developed network.

The computer readable medium may be embodied in the electronic device; or may exist separately without being assembled into the electronic device.

The computer readable medium carries one or more programs which, when executed by the electronic device, cause the electronic device to: acquiring page text information of a sample application program, wherein the page text information comprises page layout description information and corresponding page action operation information; and analyzing and generating control layout information and corresponding action position information of each of a plurality of continuous pages of the sample application program according to the page text information of the sample application program to serve as training samples, wherein the training samples are used for training test models, and the test models are used for carrying out automatic test on the application program to be tested.

Computer program code for carrying out operations for the present disclosure may be written in any combination of one or more programming languages, including but not limited to an object oriented programming language such as Java, Smalltalk, C + +, and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet service provider).

The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.

The modules or units described in the embodiments of the present disclosure may be implemented by software or hardware. The name of a module or a unit does not in some cases constitute a limitation of the unit itself, and for example, the acquiring module may also be described as a module for acquiring page text information of a sample application.

The functions described herein above may be performed, at least in part, by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field Programmable Gate Arrays (FPGAs), Application Specific Integrated Circuits (ASICs), Application Specific Standard Products (ASSPs), systems on a chip (SOCs), Complex Programmable Logic Devices (CPLDs), and the like.

In the context of this disclosure, a machine-readable medium may be a tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

The disclosed embodiment provides an electronic device, and the electronic device in the disclosed embodiment includes: a memory and a processor; at least one program stored in the memory for execution by the processor, which when executed by the processor, implements: the method comprises the steps of obtaining page text information of a sample application program, wherein the page text information comprises page layout description information and corresponding page action operation information, analyzing and generating control layout information and corresponding action position information of a plurality of continuous pages of the sample application program according to the page text information of the sample application program, and using the control layout information and the corresponding action position information as training samples, wherein the training samples are used for training a test model, and the test model is used for carrying out automatic test on the application program to be tested. Because the control layout information and the action position information are extracted from the page text information and are both text information, the processing speed of the network on the text information is relatively high, and therefore, the processing efficiency of the model can be greatly improved by using the training sample training model formed by the control layout information and the action position information, and the time consumption of the training model is reduced.

The electronic device of the present disclosure is described above from the perspective of a physical apparatus, and the computer-readable medium of the present disclosure is described below from the perspective of a medium.

The embodiment of the present disclosure provides a computer-readable medium, on which a computer program is stored, which, when running on a computer, enables the computer to execute the corresponding content in the foregoing method embodiments. Compared with the prior art, the method includes the steps of obtaining page text information of a sample application program, wherein the page text information comprises page layout description information and corresponding page action operation information, analyzing and generating control layout information and corresponding action position information of a plurality of continuous pages of the sample application program according to the page text information of the sample application program, and using the control layout information and the corresponding action position information as training samples, wherein the training samples are used for training a test model, and the test model is used for carrying out automatic test on an application program to be tested. Because the control layout information and the action position information are extracted from the page text information and are both text information, the processing speed of the network on the text information is relatively high, and therefore, the processing efficiency of the model can be greatly improved by using the training sample training model formed by the control layout information and the action position information, and the time consumption of the training model is reduced.

According to one or more embodiments of the present disclosure, there is provided a data processing method including:

and analyzing and generating control layout information and corresponding action position information of each of a plurality of continuous pages of the sample application program according to the page text information of the sample application program to serve as training samples, wherein the training samples are used for training test models, and the test models are used for carrying out automatic test on the application program to be tested.

According to one or more embodiments of the present disclosure, parsing and generating, according to page text information of the sample application program, control layout information and corresponding action position information of each of a plurality of continuous pages of the sample application program, as a training sample, includes:

extracting a text control, a picture control and other visual controls of a page as three-channel information of the page according to the page layout description information of the sample application program, and generating control layout information of the page based on the three-channel information of the page;

According to one or more embodiments of the present disclosure, the extracting, according to the page layout description information of the sample application program, a text type control, a picture type control, and other visual type controls of a page as three-channel information of the page, and generating control layout information of the page based on the three-channel information of the page includes:

drawing a blank picture according to a preset resolution;

filling the three-channel information of the page into the blank picture according to the preset resolution to obtain control layout information of the page;

correspondingly, the integrating the control layout information and the corresponding action position information of the single page to generate the control layout and action position diagram of the single page includes:

According to one or more embodiments of the present disclosure, when at least one type of control in the three-channel information of the page has control overlap, filling the three-channel information of the page into the blank picture according to the preset resolution to obtain control layout information of the page, including:

and filling the three-channel information of the page into the blank picture according to the corresponding color information and the preset resolution ratio to obtain the control layout information of the page.

According to one or more embodiments of the present disclosure, the using a control layout and action position diagram of a plurality of continuous pages as a training sample includes:

According to one or more embodiments of the present disclosure, the obtaining page text information of the sample application includes:

According to one or more embodiments of the present disclosure, the method further comprises:

acquiring control layout information of four continuous pages and action position information of the first three continuous pages in the application program to be tested as input information of the test model;

predicting action position prediction information aiming at the last page in the four pages based on the input information through the test model;

and controlling the test aiming at the application program to be tested based on the action position prediction information.

According to one or more embodiments of the present disclosure, there is provided a data processing apparatus including:

the system comprises an acquisition module, a display module and a display module, wherein the acquisition module is used for acquiring page text information of a sample application program, and the page text information comprises page layout description information and corresponding page action operation information;

According to one or more embodiments of the present disclosure, the generating module is specifically configured to:

According to one or more embodiments of the present disclosure, when the generation module extracts a text type control, a picture type control, and other visual type controls of a page as three-channel information of the page according to the page layout description information of the sample application program, and generates the control layout information of the page based on the three-channel information of the page, the generation module is specifically configured to:

drawing a blank picture according to a preset resolution;

correspondingly, the generating module is specifically configured to, when integrating the control layout information of a single page and the corresponding action position information to generate a control layout and action position map of the single page:

According to one or more embodiments of the present disclosure, when at least one type of control in the three-channel information of the page has control overlap, the generating module is specifically configured to, when filling the three-channel information of the page into the blank picture according to the preset resolution to obtain control layout information of the page:

According to one or more embodiments of the present disclosure, when the control layout and the action position diagram of a plurality of continuous pages are used as a training sample, the generating module is specifically configured to:

According to one or more embodiments of the present disclosure, the obtaining module is specifically configured to:

According to one or more embodiments of the present disclosure, the apparatus further comprises:

the acquisition module is used for acquiring control layout information of four continuous pages and action position information of the first three pages in the application program to be tested as input information of the test model;

the prediction module is used for predicting action position prediction information aiming at the last page in the four pages based on the input information through the test model;

and the control module is used for controlling the test aiming at the application program to be tested based on the action position prediction information.

In accordance with one or more embodiments of the present disclosure, an electronic device is disclosed, comprising:

one or more processors;

a memory;

one or more applications, wherein the one or more applications are stored in the memory and configured to be executed by the one or more processors, the one or more programs configured to: the data processing method shown according to the method embodiment is performed.

According to one or more embodiments of the present disclosure, a computer-readable medium is disclosed, on which a computer program is stored, which when executed by a processor implements the data processing method shown in the method embodiments.

The foregoing description is only exemplary of the preferred embodiments of the disclosure and is illustrative of the principles of the technology employed. It will be appreciated by those skilled in the art that the scope of the disclosure herein is not limited to the particular combination of features described above, but also encompasses other embodiments in which any combination of the features described above or their equivalents does not depart from the spirit of the disclosure. For example, the above features and (but not limited to) the features disclosed in this disclosure having similar functions are replaced with each other to form the technical solution.

Further, while operations are depicted in a particular order, this should not be understood as requiring that such operations be performed in the particular order shown or in sequential order. Under certain circumstances, multitasking and parallel processing may be advantageous. Likewise, while several specific implementation details are included in the above discussion, these should not be construed as limitations on the scope of the disclosure. Certain features that are described in the context of separate embodiments can also be implemented in combination in a single embodiment. Conversely, various features that are described in the context of a single embodiment can also be implemented in multiple embodiments separately or in any suitable subcombination.

Although the subject matter has been described in language specific to structural features and/or methodological acts, it is to be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are disclosed as example forms of implementing the claims.

Claims

1. A data processing method, comprising:

2. The method according to claim 1, wherein the parsing generates, as a training sample, control layout information and corresponding action position information of each of a plurality of continuous pages of the sample application according to the page text information of the sample application, and includes:

3. The method of claim 2, wherein the extracting text type controls, picture type controls and other visual type controls of the page as three-channel information of the page according to the page layout description information of the sample application program, and generating control layout information of the page based on the three-channel information of the page comprises:

drawing a blank picture according to a preset resolution;

4. The method according to claim 2, wherein when there is control overlap in at least one type of control in the three-channel information of the page, the filling the three-channel information of the page into the blank picture according to the preset resolution to obtain control layout information of the page includes:

5. The method according to claim 2, wherein the step of using the control layout and action position map of the continuous multiple pages as a training sample comprises:

6. The method of claim 1, wherein obtaining page text information for a sample application comprises:

7. The method according to any one of claims 1-6, further comprising:

8. A data processing apparatus, comprising:

9. An electronic device, comprising:

one or more processors;

a memory;

one or more applications, wherein the one or more applications are stored in the memory and configured to be executed by the one or more processors, the one or more programs configured to: performing the data processing method according to any one of claims 1 to 7.

10. A computer-readable medium, on which a computer program is stored, which program, when being executed by a processor, is adapted to carry out the data processing method of any one of claims 1 to 7.