CN112241361A - Test case generation method and device and problem scene automatic reproduction method and device - Google Patents
Test case generation method and device and problem scene automatic reproduction method and device Download PDFInfo
- Publication number
- CN112241361A CN112241361A CN201910654125.9A CN201910654125A CN112241361A CN 112241361 A CN112241361 A CN 112241361A CN 201910654125 A CN201910654125 A CN 201910654125A CN 112241361 A CN112241361 A CN 112241361A
- Authority
- CN
- China
- Prior art keywords
- control
- test case
- screenshot
- interface
- user interface
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/36—Preventing errors by testing or debugging software
- G06F11/3668—Software testing
- G06F11/3672—Test management
- G06F11/3684—Test management for test design, e.g. generating new test cases
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/36—Preventing errors by testing or debugging software
- G06F11/3668—Software testing
- G06F11/3696—Methods or tools to render software testable
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
- G06F3/0484—Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
- G06F3/04845—Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range for image manipulation, e.g. dragging, rotation, expansion or change of colour
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
- G06F3/0484—Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
- G06F3/0485—Scrolling or panning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
- G06F3/0487—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser
- G06F3/0488—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures
Abstract
The invention discloses a test case generation method and a device, wherein the method comprises the following steps: acquiring a user interface screenshot of software to be tested; recognizing the user interface screenshot by utilizing a pre-trained interface element recognition model to obtain each control in the user interface screenshot and related information of the control; and generating an interface test case set according to each control and related information thereof in the user interface screenshot. The invention also discloses a method and a device for automatically reproducing the problem scene. The invention can improve the generation efficiency of the test case and realize the comprehensive test of the user interface.
Description
Technical Field
The invention relates to the field of software testing, in particular to a method and a device for generating a test case and a method and a device for automatically reproducing a problem scene.
Background
A Graphical User Interface (GUI) is a computer operation User Interface displayed in a Graphical manner, and is an Interface display format for human-computer communication, which allows a User to use an input device such as a mouse to manipulate icons or menu options on a screen to select commands, call files, start programs, or perform other daily tasks.
The existing test aiming at the GUI generally needs to manually design a test case according to the type and layout of an interface control, define the contents of a tested object, a test mode, an expected result and the like, and then a test terminal executes the test case to finish the GUI test. Even if the automatic test is adopted, namely, a machine simulation user tests the operation of the GUI, the test case which is designed in advance by manpower is required to be relied on and operation elements are specified to be executed, or the program source code is analyzed to obtain the elements and execute the test case. Because the GUI software comprises a large number of windows, each window comprises a large number of elements, different functions of the GUI software are formed by different combinations of the windows and the elements, and due to the diversity of the windows and the elements and the complexity of the combinations in the GUI software, a large amount of time is usually needed for manually writing test cases, and the efficiency is low.
Disclosure of Invention
An embodiment of the invention provides a test case generation method and device to improve test case generation efficiency.
The embodiment of the invention also provides a method and a device for automatically reproducing the problem scene, which can automatically reproduce the problem scene.
Therefore, the invention provides the following technical scheme:
a test case generation method, the method comprising:
acquiring a user interface screenshot of software to be tested;
recognizing the user interface screenshot by utilizing a pre-trained interface element recognition model to obtain each control in the user interface screenshot and related information of the control; the related information of the control comprises the coordinate and the type of the control;
and generating an interface test case set according to each control and related information thereof in the user interface screenshot.
Optionally, the method further comprises the step of training the interface element recognition model, including:
determining a topological structure of an interface element recognition model;
collecting interface screenshots of a large amount of software as training interface screenshots;
marking a control in the screenshot of the training interface, wherein marking information comprises the coordinate and the type of the control;
and training by using the interface screenshot and the labeling information thereof to obtain an interface element recognition model.
Optionally, the tagging information further includes: and the control corresponds to an operation set.
Optionally, the interface test case set includes an independent test case subset corresponding to each control and a combined test case subset corresponding to a control combination;
generating the independent test case subsets corresponding to the controls comprises:
acquiring an operation set corresponding to the type of each control in the user interface screenshot, and generating an independent test case corresponding to each control according to the operation set corresponding to the type of each control and the coordinates of the control;
adding the independent test cases to independent test case sets corresponding to the controls;
generating a subset of the combined test cases for the corresponding control combination comprises:
all controls in the user interface screenshot are subjected to full-array combination to obtain each combination;
determining the operation sequence of different controls in each combination;
sequentially calling the independent test case subsets corresponding to each control according to the operation sequence to generate a combined test case corresponding to the operation sequence under the combination;
and adding the combined test case to a combined test case set of the corresponding control combination.
Optionally, the generating an independent test case corresponding to each control according to the operation set corresponding to the type to which each control belongs and the coordinates of the control includes:
and acquiring an executable script template corresponding to each operation in an operation set corresponding to the type of the control, filling the coordinates of the control into the executable script template, and generating an independent test case corresponding to the control.
Optionally, the method further comprises:
testing the user interface by using each test case in the interface test case set to obtain a test result;
acquiring a screenshot of the test result;
and establishing and storing the corresponding relation between each test case and the screenshot of the test result.
A method of problem scenario automatic recurrence, the method comprising:
acquiring a screenshot of a problem scene provided by a user as a problem screenshot;
recognizing the problem screenshot by using a pre-trained interface element recognition model to obtain each control in the problem screenshot and related information of the control; the related information of the control comprises the coordinate and the type of the control;
generating a problem test case set according to each control in the problem screenshot and the related information of the control;
and reproducing the problem scene by using the problem test case set.
Optionally, the reproducing the problem scenario by using the problem test case suite includes:
testing the problem scene by using each test case in the problem test case set to obtain a screenshot of a problem scene test result;
selecting a problem reproduction scene screenshot from the screenshots of the problem scene test results;
and acquiring a test case corresponding to the problem recurrence scene screenshot.
Optionally, the selecting a screenshot of a problem recurring scenario from the screenshots of the problem scenario test results includes:
calculating the similarity between the screenshot of the test result of each problem scene and the problem screenshot;
taking the screenshot of the problem scene test result with the similarity larger than a set threshold value as a problem recurrence scene screenshot; or
And selecting the screenshots of the test results of the problem scenes with the set number from large to small according to the similarity as the screenshots of the problem recurrence scenes.
A test case generation apparatus, the apparatus comprising:
the screenshot obtaining module is used for obtaining a user interface screenshot of the software to be tested;
the recognition module is used for recognizing the user interface screenshot by utilizing a pre-trained interface element recognition model to obtain each control in the user interface screenshot and related information of the control; the related information of the control comprises the coordinate and the type of the control;
and the interface test case generation module is used for generating an interface test case set according to each control and relevant information in the user interface screenshot.
Optionally, the apparatus further includes a model training module, configured to train the interface element recognition model; the model training module comprises:
the structure determining unit is used for determining the topological structure of the interface element recognition model;
the information collection unit is used for collecting interface screenshots of a large amount of software as training interface screenshots;
the marking unit is used for marking the control in the screenshot of the training interface, and marking information comprises the coordinate and the type of the control;
and the training unit is used for training by utilizing the interface screenshot and the labeling information thereof to obtain an interface element recognition model.
Optionally, the tagging information further includes: and the control corresponds to an operation set.
Optionally, the interface test case set includes an independent test case subset corresponding to each control and a combined test case subset corresponding to a control combination;
the interface test case generation module comprises: the control combination test case generation module is used for generating a control combination test case subset corresponding to each control;
the first subset generation module comprises:
the independent test case generation unit is used for acquiring an operation set corresponding to the type of each control in the user interface screenshot and generating an independent test case corresponding to each control according to the operation set corresponding to the type of each control and the coordinates of the control;
the independent test case adding unit is used for adding the independent test cases to the independent test case sets corresponding to the controls;
the second subset generation module comprises:
the combination unit is used for carrying out full-arrangement combination on all the controls in the user interface screenshot to obtain each combination;
the operation sequence determining unit is used for determining the operation sequence of different controls in each combination;
the calling unit is used for calling the independent test case subsets corresponding to each control in sequence according to the operation sequence and generating the combined test case corresponding to the operation sequence under the combination;
and the combined test case adding unit is used for adding the combined test case to the combined test case set corresponding to the control combination.
Optionally, the independent test case generating unit is specifically configured to obtain an executable script template corresponding to each operation in an operation set corresponding to the type to which the control belongs, fill the coordinates of the control into the executable script template, and generate an independent test case corresponding to the control.
Optionally, the apparatus further comprises: the test case execution module and the corresponding relation generation module;
the test case execution module is used for testing the user interface by using each test case in the interface test case set to obtain a test result;
the screenshot obtaining module is further used for obtaining a screenshot of the test result;
and the corresponding relation generating module is used for establishing and storing the corresponding relation between each test case and the screenshot of the test result.
An apparatus for automatic problem scenario recurrence, the apparatus comprising:
the problem scene acquisition module is used for acquiring a screenshot of a problem scene provided by a user as a problem screenshot;
the recognition module is used for recognizing the problem screenshot by utilizing a pre-trained interface element recognition model to obtain each control in the problem screenshot and related information of the control;
the problem test case generation module is used for generating a problem test case set according to each control in the problem screenshot and the related information of the control;
and the scene reproduction module is used for reproducing the problem scene by utilizing the problem test case set.
Optionally, the scene reproduction module includes:
the test unit is used for testing the problem scene by utilizing each test case in the problem test case set to obtain a screenshot of a problem scene test result;
the scene screening unit is used for selecting a problem reproduction scene screenshot from the screenshots of the problem scene test results;
and the test case acquisition unit is used for acquiring the test case corresponding to the problem reproduction scene screenshot.
Optionally, the scene filtering unit includes:
the similarity calculation operator unit is used for calculating the similarity between the screenshot of the test result of each problem scene and the problem screenshot;
the screening subunit is used for taking the screenshot of the problem scene test result with the similarity larger than the set threshold as the screenshot of the problem recurrence scene; or selecting the screenshots of the test results of the problem scenes with the set number from large to small according to the similarity as the screenshots of the problem recurrence scenes.
An electronic device, comprising: one or more processors, memory;
the memory is configured to store computer-executable instructions and the processor is configured to execute the computer-executable instructions to implement the method described above.
A readable storage medium having stored thereon instructions which are executed to implement the foregoing method.
The method and the device for generating the test case provided by the embodiment of the invention are based on the image detection and recognition technology, the control in the user interface screenshot and the related information of the control are automatically recognized by utilizing the interface element recognition model trained in advance, and then the interface test case set is generated by utilizing the control and the related information thereof, so that the test case generation efficiency is greatly improved.
Furthermore, the interface test case set comprises independent test case subsets corresponding to the controls and combined test case subsets corresponding to different combinations of the controls, so that the generated interface test case set comprises test cases corresponding to reasonable operation and test cases corresponding to abnormal operation, and the inertial thinking in the process of manually designing the test cases is broken. The robustness of the software system can be better tested by utilizing the test cases.
Furthermore, each test case in the interface test case sets can be used for testing the user interface to obtain a test result, a screenshot of the test result is obtained, and a corresponding relation between each test case and the screenshot of the test result is established and stored. And providing the corresponding relation for a tester, so that the tester can be helped to quickly determine whether each test result is correct.
According to the problem scene automatic reproduction method and device provided by the embodiment of the invention, aiming at the problem scene provided by the user, the pre-trained interface element recognition model is used for automatically recognizing the screenshot of the problem scene, and the problem test case set is generated according to the recognition result, so that the tester can reproduce the problem scene according to the test cases in the problem test case set, and the problem backtracking is realized. Compared with manual repeated endless problem searching, the problem can be quickly and effectively found, and the problem is solved.
Drawings
In order to more clearly illustrate the embodiments of the present application or technical solutions in the prior art, the drawings needed to be used in the embodiments will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments described in the present invention, and other drawings can be obtained by those skilled in the art according to the drawings.
FIG. 1 is a flow chart of the training of an interface element recognition model according to an embodiment of the present invention;
FIG. 2 is a flow chart of a test case generation method according to an embodiment of the present invention;
FIG. 3 is a flowchart of a method for automatic problem scenario reproduction according to an embodiment of the present invention;
FIG. 4 is a flow chart of problem scenario reproduction in an embodiment of the present invention;
FIG. 5 is a block diagram of a test case generating apparatus according to an embodiment of the present invention;
FIG. 6 is another block diagram of a test case generating apparatus according to an embodiment of the present invention;
FIG. 7 is a block diagram showing an arrangement of an apparatus for automatically reproducing problem scenes according to an embodiment of the present invention;
FIG. 8 is a block diagram of a scene replication module according to an embodiment of the present invention;
FIG. 9 is a block diagram illustrating an apparatus for a test case generation method in accordance with an illustrative embodiment;
fig. 10 is a schematic structural diagram of a server in an embodiment of the present invention.
Detailed Description
In order to make the technical field of the invention better understand the scheme of the embodiment of the invention, the embodiment of the invention is further described in detail with reference to the drawings and the implementation mode.
Aiming at the problem of low graphical user interface generation efficiency in the prior art, the embodiment of the invention provides a test case generation method and a test case generation device, wherein a pre-trained interface element recognition model is used for recognizing a user interface screenshot of software to be tested to obtain each control in the user interface screenshot and related information of the control; and generating an interface test case set according to each control and related information in the user interface screenshot, wherein each test case in the interface test case set can be used for comprehensively and effectively testing the user interface, and the robustness of the software system is improved.
In addition, the embodiment of the invention also provides a method and a device for automatically reproducing the problem scene.
The interface element recognition model may adopt a Neural Network structure, such as a CNN (Convolutional Neural Network) structure, and a specific training process is shown in fig. 1, and includes the following steps:
For example, controls such as a search box, a button, a text link and the like in the screenshot of the training interface are labeled, and coordinates and types of the controls are labeled.
Further, the labeling information may further include: and the control corresponds to an operation set.
The controls are packages of data and methods, the content and the style of each control can be set by software developers, but the corresponding operation set of each type of control is fixed. Such as: the operation set corresponding to Button is click; the operation set corresponding to the Link is click; the operation sets corresponding to Text are input and delete. Therefore, after the type of each control is determined, the operation set of the control can be obtained.
And 104, training by using the interface screenshot and the labeling information thereof to obtain an interface element recognition model.
The specific training mode is similar to the training mode of the existing neural network, and is not described in detail here.
The pre-trained interface element recognition model is used for recognizing the user interface screenshot of the software to be tested, so that all controls in the user interface screenshot and related information of the controls can be obtained, and then the corresponding test cases are automatically generated by combining operation sets corresponding to all control types.
As shown in fig. 2, it is a flowchart of a test case generation method according to an embodiment of the present invention, and the method includes the following steps:
The user interface screenshot can be obtained by an existing screenshot tool or a photographing tool, and the user interface screenshot is stored in a picture format, such as a BMP format, a TIF format, a GIF format, a JPEG format, and the like, which is not limited in the embodiment of the present invention.
When the interface element recognition model is used for recognizing the user interface screenshot, the user interface screenshot can be input into the interface element recognition model, and each control in the user interface screenshot and the related information of the control are obtained according to the output of the interface element recognition model.
In embodiments of the present invention, the controls identified using the interface element recognition model may include any controls used in graphical user interfaces, such as text controls, link controls, slider controls, pagination controls, and the like.
The related information of the control includes, but is not limited to, the coordinate and the belonging type of the control.
And 203, generating an interface test case set according to each control and related information thereof in the user interface screenshot.
In the embodiment of the present invention, the interface test case set may include an independent test case subset corresponding to each control and a combined test case subset corresponding to a control combination.
The independent test case subsets corresponding to the controls can be generated according to the following modes:
acquiring an operation set corresponding to the type of each control in the user interface screenshot, for example, a click operation corresponding to a button, an input operation corresponding to a text box, a delete operation and the like, and generating an independent test case corresponding to each control according to the operation set corresponding to the type of each control and the coordinates of the control; and then, adding the independent test cases to the independent test case sets corresponding to the controls.
Corresponding to different types of controls, corresponding executable script templates can be preset according to corresponding operations. Correspondingly, after obtaining each control and coordinate information thereof in the user interface screenshot, obtaining an executable script template corresponding to each operation in an operation set corresponding to the type to which the control belongs, and filling the coordinates of the control into the executable script template, so that an independent test case corresponding to the control can be generated.
The subset of the combined test cases corresponding to the control combination may include a test case corresponding to a normal operation of each combination according to the software design. Further, in order to implement comprehensive testing of the user interface and improve the robustness of the software system, all controls in the screenshot of the user interface may be arranged and combined completely, and corresponding test cases are generated for all operations in the operation sequence in each combination. Specifically, the process of generating the combined test case subset corresponding to the control combination is as follows:
(1) all controls in the user interface screenshot are subjected to full-array combination to obtain each combination;
(2) determining the operation sequence of different controls in each combination;
(3) sequentially calling independent test case subsets corresponding to the control according to the operation sequence to generate a combined test case corresponding to the operation sequence under the combination;
(4) and adding the combined test case to a combined test case set of the corresponding control combination.
For example, in a combination comprising a control button A and an input box B, the operation sequence under the combination is: the button A- - > input box B calls the independent test case subsets corresponding to the button A and the input box B respectively to obtain the test cases corresponding to the operation sequence under the combination, and the test cases comprise:
click A-->input B；
click A-->delete B；
click A-->input B-->delete B；
click A-->delete B-->input B。
the test case generation method provided by the embodiment of the invention is based on the image detection and recognition technology, utilizes the interface element recognition model trained in advance to automatically recognize the control in the user interface screenshot and the related information of the control, and then utilizes the control and the related information thereof to generate the interface test case set, thereby greatly improving the test case generation efficiency.
Furthermore, the interface test case set includes an independent test case subset corresponding to each control and a combined test case subset corresponding to various different combinations of the controls, and as can be seen from the above example, not only all the controls in the user interface screenshot are combined in a full arrangement, but also for each combination, the independent test case subset corresponding to each control is called according to the operation sequence of each control, and the combined test case corresponding to the operation sequence is generated. That is to say, the test cases under each combination cover all the operation conditions under the operation sequence, so that the generated interface test case set contains both the test cases corresponding to reasonable operation and the test cases corresponding to abnormal operation, the inertial thinking in the manual design of the test cases is broken, the test cases for the user interface are richer and more comprehensive, and the robustness of the software system can be better tested by using the test cases.
Further, in another embodiment of the test case generation method of the present invention, each test case in the interface test case sets (including each test case in the independent test case set corresponding to each control and each test case in the combined test case set corresponding to the control combination) may be used to test the user interface to obtain a test result, and a screenshot of the test result is obtained, and a corresponding relationship between each test case and the screenshot of the test result is established and stored, and the corresponding relationship is provided to a tester, which may help the tester to quickly determine whether each test result is correct.
Correspondingly, the embodiment of the invention also provides an automatic problem scene reproduction method, which can realize problem scene reproduction aiming at the problem scene provided by the user.
As shown in fig. 3, it is a flowchart of an automatic problem scenario reproduction method according to an embodiment of the present invention, including the following steps:
The problem scene refers to an abnormal situation of a user graphical interface when the software runs, namely the user graphical interface is not an interface expected to be obtained during design.
And 303, generating a problem test case set according to each control in the problem screenshot and the related information of the control.
The above-mentioned manner of identifying the problem screenshot and generating the problem test case set according to the identification result is similar to the process of generating the interface test case set for the user interface shown in fig. 2, and is not described in detail here.
And 304, reproducing the problem scene by using the problem test case set.
When the problem scene is reproduced by the problem test case set, the problem scene needs to be tested by each test case in the problem test case set, a possible problem reproduction scene is found out from test results, and test cases corresponding to the problem reproduction scene are obtained, so that the problem scene can be reproduced by the test cases.
As shown in fig. 4, it is a flowchart for problem scene reproduction in the embodiment of the present invention, and the flowchart includes the following steps:
A problem scene test result can be obtained corresponding to each test case in the problem test case set.
Specifically, the similarity between the screenshot of the test result of each problem scene and the problem screenshot can be calculated; taking the screenshot of the problem scene test result with the similarity larger than a set threshold value as a problem recurrence scene screenshot; or selecting the screenshots of the test results of the problem scenes with the set number from large to small according to the similarity as the screenshots of the problem recurrence scenes.
In practical application, one or more problem recurrence scene screenshots may be selected in the above manner, and the embodiment of the present invention is not limited.
And 403, acquiring a test case corresponding to the problem recurrence scene screenshot.
Because each problem scene test result corresponds to one test case, the test case corresponding to the problem recurrence scene screenshot can be directly obtained according to the corresponding relation, and the recurrence of the problem scene can be automatically realized by utilizing the test case. Furthermore, the corresponding relation between the screenshot of the problem recurrence scene and the corresponding test case can be stored, and the corresponding relation is provided for the tester, so that the tester can find the problem as soon as possible, and then the problem is solved. It should be noted that, in practical applications, one or more question reproduction scene screenshots meeting the above conditions may be selected in step 402. If a plurality of problem recurrence scene screenshots are selected, a corresponding test case needs to be acquired for each problem recurrence scene screenshot.
According to the problem scene automatic recurrence method provided by the embodiment of the invention, aiming at the problem scene provided by the user, the pre-trained interface element recognition model is used for recognizing the screenshot of the problem scene, the control in the screenshot of the problem scene and the related information of the control are automatically recognized, the problem test case set aiming at the problem scene is generated, and the problem scene is automatically repeated by using the problem test case set.
Correspondingly, an embodiment of the present invention further provides a test case generating apparatus, which is a structural block diagram of the apparatus, as shown in fig. 5.
In this embodiment, the test case generation apparatus includes the following modules:
a screenshot obtaining module 51, configured to obtain a user interface screenshot of software to be tested;
the recognition module 52 is configured to recognize the user interface screenshot by using a pre-trained interface element recognition model, so as to obtain each control in the user interface screenshot and related information of the control; the related information includes but is not limited to the coordinate and type of the control;
and the interface test case generating module 53 is configured to generate an interface test case set according to each control in the user interface screenshot and related information thereof.
The screenshot obtaining module 51 may obtain a user interface screenshot of the software to be tested through an existing screenshot tool or a photographing tool, and store the user interface screenshot as a picture format.
The interface element recognition model can be constructed by a corresponding model training module by adopting a neural network structure, and the model training module can be used as a part of the test case generation device and can also be independent of the test case generation device, so that the embodiment of the invention is not limited.
One specific structure of the model training module may include the following units:
the structure determining unit is used for determining the topological structure of the interface element recognition model;
the information collection unit is used for collecting interface screenshots of a large amount of software as training interface screenshots;
the marking unit is used for marking the control in the screenshot of the training interface, and marking information comprises the coordinate and the type of the control;
and the training unit is used for training by utilizing the interface screenshot and the labeling information thereof to obtain an interface element recognition model. The specific training mode is similar to the training mode of the existing neural network, and is not described in detail here.
It should be noted that, in the interface element identification model, the labeling unit may further obtain an operation set corresponding to the control according to the type of the control, that is, the operation set corresponding to the control may also be used as a label to participate in training of the interface element identification model, which is not limited in the embodiment of the present invention.
In the embodiment of the present invention, the interface test case set may include an independent test case subset corresponding to each control and a combined test case subset corresponding to a control combination. The combined test case subset corresponding to the control combination may include various possible operations under each combination in the full-permutation combinations of all the controls in the user interface screenshot, that is, both the test case corresponding to the reasonable operation under the combination and the test case corresponding to the abnormal operation under the combination are included.
Correspondingly, as shown in fig. 5, the interface test case generating module 53 may specifically include: a first subset generation module 531 for generating a subset of independent test cases for each control and a second subset generation module 532 for generating a subset of combined test cases for a combination of controls. Wherein:
the first subset generating module 531 may include the following units:
the independent test case generation unit is used for acquiring an operation set corresponding to the type of each control in the user interface screenshot and generating an independent test case corresponding to each control according to the operation set corresponding to the type of each control and the coordinates of the control; specifically, an executable script template corresponding to each operation in an operation set corresponding to the type to which the control belongs is obtained, coordinates of the control are filled into the executable script template, and an independent test case corresponding to the control is generated;
the independent test case adding unit is used for adding the independent test cases to the independent test case sets corresponding to the controls;
the second subset generating module 532 may include the following units:
the combination unit is used for carrying out full-arrangement combination on all the controls in the user interface screenshot to obtain each combination;
the operation sequence determining unit is used for determining the operation sequence of different controls in each combination;
the calling unit is used for calling the independent test case subsets corresponding to each control in sequence according to the operation sequence and generating the combined test case corresponding to the operation sequence under the combination;
and the combined test case adding unit is used for adding the combined test case to the combined test case set corresponding to the control combination.
It should be noted that, for each embodiment of the test case generation apparatus, since the functional implementation of each module and unit is similar to that in the corresponding method, the description of each embodiment of the test case generation apparatus is relatively simple, and relevant points can be referred to the corresponding description of the method embodiment.
The test case generation device provided by the embodiment of the invention is based on the image detection and recognition technology, utilizes the pre-trained interface element recognition model to automatically recognize the control in the user interface screenshot and the related information of the control, and then utilizes the control and the related information thereof to generate the interface test case set, thereby greatly improving the test case generation efficiency.
Furthermore, the interface test case set comprises an independent test case subset corresponding to each control and a combined test case subset corresponding to various different combinations of the controls, and as can be seen from the above examples, not only is all the controls in the user interface screenshot fully-arranged, and for each combination, according to the operation sequence of each control, calling the independent test case subset corresponding to the control to generate the combined test case corresponding to the operation sequence, that is, the test case in each combination covers all the operation conditions in the operation sequence, therefore, the generated interface test case set comprises test cases corresponding to reasonable operation and test cases corresponding to abnormal operation, the inertia thinking in the process of manually designing the test cases is broken, the test cases aiming at the user interface are richer and more comprehensive, and the robustness of the software system can be better tested by utilizing the test cases.
Fig. 6 is another block diagram of the test case generating apparatus according to the embodiment of the present invention.
The difference from the embodiment shown in fig. 5 is that, in this embodiment, the test case generation apparatus further includes: a test case execution module 54 and a correspondence relationship generation module 55.
In this embodiment, the test case execution module 54 is configured to test the user interface by using each test case in the interface test case set generated by the test case generation module 53, so as to obtain a test result; the screenshot obtaining module 51 obtains a screenshot of the test result; the corresponding relationship generating module 54 is configured to establish and store a corresponding relationship between each test case and a screenshot of a test result.
The test case generation device provided in this embodiment can not only automatically recognize the control and the related information of the control in the user interface screenshot according to the interface element recognition model trained in advance, and then generate the interface test case set using the control and the related information thereof, but also test the user interface using each test case in the interface test case set to obtain the test result, obtain the screenshot of the test result, and establish and store the corresponding relationship between each test case and the screenshot of the test result. And providing the corresponding relation for a tester, so that the tester can be helped to quickly determine whether each test result is correct.
Correspondingly, the embodiment of the invention also provides an automatic problem scene reproduction device, which can realize problem scene reproduction aiming at the problem scene provided by the user.
Fig. 7 is a block diagram showing a structure of the problem scene automatic reproduction apparatus.
In this embodiment, the automatic problem scenario recurrence device includes the following modules:
a problem scene acquisition module 71, configured to acquire a screenshot of a problem scene provided by a user as a problem screenshot;
the recognition module 72 is configured to recognize the problem screenshot by using a pre-trained interface element recognition model, so as to obtain each control in the problem screenshot and related information of the control;
the problem test case generation module 73 is further configured to generate a problem test case set according to each control in the problem screenshot and the related information of the control;
and a scene reproduction module 74, configured to reproduce the problem scene by using the problem test case set.
The interface test case set may be generated by the test case generation device, that is, generated based on an image detection and recognition technology, or generated based on other devices or by other methods, which is not limited in the embodiment of the present invention.
Accordingly, the principle of the recognition module 72 and the principle of the problem test case generation module 73 are the same as the recognition module 52 and the principle of the interface test case generation module 53 in the test case generation device embodiment shown in fig. 5, and if an integrated system including the test case generation device and the problem scenario automatic reproduction device are adopted, the corresponding modules in the two devices may be independent from each other or integrated into a whole, which is not limited in the embodiment of the present invention.
It should be noted that in practical application, there may be a plurality of problem scenarios, and in the embodiment of the present invention, a problem test case set corresponding to each problem scenario needs to be generated for each problem scenario. That is, each problem scenario corresponds to a problem test case set, and the scenario replication module 74 replicates the problem scenario by using the problem test case set corresponding to the problem scenario.
When the problem scenario is reproduced by the problem test case set, the scenario reproduction module 74 needs to test the problem scenario by using each test case in the problem test case set, find out a possible problem reproduction scenario from the test results, and obtain the test cases corresponding to the problem reproduction scenario, so that the problem scenario can be reproduced by using the test cases.
Accordingly, a specific structure of the scene replication module 74 is shown in fig. 8, and may include the following units:
the test unit 741 is configured to test a corresponding problem scene by using each test case in the problem test case set, so as to obtain a screenshot of a problem scene test result;
a scene screening unit 742, configured to select a problem recurrence scene screenshot from the screenshots of the problem scene test results;
and a test case obtaining unit 743, configured to obtain a test case corresponding to the problem recurrence scene screenshot.
The scene screening unit 742 may select a screenshot of the problem reproduction scene by comparing the screenshot of the test result of the problem scene with the loading map of the problem scene. Accordingly, a specific structure of the scene filtering unit 742 may include the following sub-units:
the similarity calculation operator unit is used for calculating the similarity between the screenshot of the test result of each problem scene and the problem screenshot;
the screening subunit is used for taking the screenshot of the problem scene test result with the similarity larger than the set threshold as the screenshot of the problem recurrence scene; or selecting the screenshots of the test results of the problem scenes with the set number from large to small according to the similarity as the screenshots of the problem recurrence scenes.
It should be noted that, since there may be a plurality of problem scenarios, the scenario recurrence module 68 needs to, for each problem scenario, utilize the test cases in the problem test case set corresponding to the problem scenario to recur the problem scenario.
For the embodiments of the above problem scene automatic reproduction device, since the function implementation of each module and unit is similar to that in the corresponding method, the description of the embodiments of the problem scene automatic reproduction device is relatively simple, and relevant points can be referred to the corresponding description of the method embodiments.
According to the problem scene automatic reproduction device provided by the embodiment of the invention, aiming at the problem scene provided by the user, the pre-trained interface element recognition model is used for recognizing the screenshot of the problem scene, the control in the screenshot of the problem scene and the related information of the control are automatically recognized, the problem test case set aiming at the problem scene is generated, and the problem scene is automatically reproduced by using the problem test case set.
FIG. 9 is a block diagram illustrating an apparatus 800 for a test case generation method in accordance with an example embodiment. For example, the apparatus 800 may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, an exercise device, a personal digital assistant, and the like.
Referring to fig. 9, the apparatus 800 may include one or more of the following components: processing component 802, memory 804, power component 806, multimedia component 808, audio component 810, input/output (I/O) interface 812, sensor component 814, and communication component 816.
The processing component 802 generally controls overall operation of the device 800, such as operations associated with display, telephone calls, data communications, camera operations, and recording operations. The processing elements 802 may include one or more processors 820 to execute instructions to perform all or a portion of the steps of the methods described above. Further, the processing component 802 can include one or more modules that facilitate interaction between the processing component 802 and other components. For example, the processing component 802 can include a multimedia module to facilitate interaction between the multimedia component 808 and the processing component 802.
The memory 804 is configured to store various classes of data to support operations at the device 800. Examples of such data include instructions for any application or method operating on device 800, contact data, phonebook data, messages, pictures, videos, and so forth. The memory 804 may be implemented by any type or combination of volatile or non-volatile memory devices, such as Static Random Access Memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic or optical disks.
The multimedia component 808 includes a screen that provides an output interface between the device 800 and a user. In some embodiments, the screen may include a Liquid Crystal Display (LCD) and a Touch Panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive an input signal from a user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundary of a touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 808 includes a front facing camera and/or a rear facing camera. The front-facing camera and/or the rear-facing camera may receive external multimedia data when the device 800 is in an operating mode, such as a shooting mode or a video mode. Each front camera and rear camera may be a fixed optical lens system or have a focal length and optical zoom capability.
The audio component 810 is configured to output and/or input audio signals. For example, the audio component 810 includes a Microphone (MIC) configured to receive external audio signals when the apparatus 800 is in an operational mode, such as a call mode, a recording mode, and a voice recognition mode. The received audio signals may further be stored in the memory 804 or transmitted via the communication component 816. In some embodiments, audio component 810 also includes a speaker for outputting audio signals.
The I/O interface 812 provides an interface between the processing component 802 and peripheral interface modules, which may be keyboards, click wheels, buttons, etc. These buttons may include, but are not limited to: a home button, a volume button, a start button, and a lock button.
The sensor assembly 814 includes one or more sensors for providing various aspects of state assessment for the device 800. For example, the sensor assembly 814 may detect the open/closed state of the device 800, the relative positioning of the components, such as a display and keypad of the apparatus 800, the sensor assembly 814 may also detect a change in position of the apparatus 800 or a component of the apparatus 800, the presence or absence of user contact with the apparatus 800, orientation or acceleration/deceleration of the apparatus 800, and a change in temperature of the apparatus 800. Sensor assembly 814 may include a proximity sensor configured to detect the presence of a nearby object without any physical contact. The sensor assembly 814 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 814 may also include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.
The communication component 816 is configured to facilitate communications between the apparatus 800 and other devices in a wired or wireless manner. The device 800 may access a wireless network based on a communication standard, such as WiFi, 2G or 3G, or a combination thereof. In an exemplary embodiment, the communication component 816 receives a broadcast signal or broadcast associated information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communications component 816 further includes a Near Field Communication (NFC) module to facilitate short-range communications. For example, the NFC module may be implemented based on Radio Frequency Identification (RFID) technology, infrared data association (IrDA) technology, Ultra Wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.
In an exemplary embodiment, the apparatus 800 may be implemented by one or more Application Specific Integrated Circuits (ASICs), Digital Signal Processors (DSPs), Digital Signal Processing Devices (DSPDs), Programmable Logic Devices (PLDs), Field Programmable Gate Arrays (FPGAs), controllers, micro-controllers, microprocessors or other electronic components for performing the above-described methods.
In an exemplary embodiment, a non-transitory computer-readable storage medium comprising instructions, such as the memory 804 comprising instructions, executable by the processor 820 of the device 800 to perform the key press false touch correction method described above is also provided. For example, the non-transitory computer readable storage medium may be a ROM, a Random Access Memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, and the like.
The present invention also provides a non-transitory computer readable storage medium having instructions which, when executed by a processor of a mobile terminal, enable the mobile terminal to perform all or part of the steps of the above-described method embodiments of the present invention.
Fig. 10 is a schematic structural diagram of a server in an embodiment of the present invention. The server 1900, which may vary widely in configuration or performance, may include one or more Central Processing Units (CPUs) 1922 (e.g., one or more processors) and memory 1932, one or more storage media 1930 (e.g., one or more mass storage devices) that store applications 1942 or data 1944. Memory 1932 and storage medium 1930 can be, among other things, transient or persistent storage. The program stored in the storage medium 1930 may include one or more modules (not shown), each of which may include a series of instructions operating on a server. Still further, a central processor 1922 may be provided in communication with the storage medium 1930 to execute a series of instruction operations in the storage medium 1930 on the server 1900.
The server 1900 may also include one or more power supplies 1926, one or more wired or wireless network interfaces 1950, one or more input-output interfaces 1958, one or more keyboards 1956, and/or one or more operating systems 1941, such as Windows Server, Mac OS XTM, UnixTM, LinuxTM, FreeBSDTM, etc.
It is obvious that the above-described embodiments are only a part of the embodiments of the present invention, and not all of them. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
It should be noted that the terms "first," "second," and the like in the description and claims of the present invention and in the drawings described above are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used is interchangeable under appropriate circumstances such that the embodiments of the invention described herein are capable of operation in sequences other than those illustrated or described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
Other embodiments of the invention will be apparent to those skilled in the art from consideration of the specification and practice of the invention disclosed herein. This invention is intended to cover any variations, uses, or adaptations of the invention following, in general, the principles of the invention and including such departures from the present disclosure as come within known or customary practice within the art to which the invention pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the invention being indicated by the following claims.
It will be understood that the invention is not limited to the precise arrangements described above and shown in the drawings and that various modifications and changes may be made without departing from the scope thereof. The scope of the invention is limited only by the appended claims.
The above description is only for the purpose of illustrating the preferred embodiments of the present invention and is not to be construed as limiting the invention, and any modifications, equivalents, improvements and the like that fall within the spirit and principle of the present invention are intended to be included therein.
Claims (10)
1. A test case generation method, the method comprising:
acquiring a user interface screenshot of software to be tested;
recognizing the user interface screenshot by utilizing a pre-trained interface element recognition model to obtain each control in the user interface screenshot and related information of the control; the related information of the control comprises the coordinate and the type of the control;
and generating an interface test case set according to each control and related information thereof in the user interface screenshot.
2. The method of claim 1, further comprising the step of training an interface element recognition model comprising:
determining a topological structure of an interface element recognition model;
collecting interface screenshots of a large amount of software as training interface screenshots;
marking a control in the screenshot of the training interface, wherein marking information comprises the coordinate and the type of the control;
and training by using the interface screenshot and the labeling information thereof to obtain an interface element recognition model.
3. The method of claim 2, wherein the tagging information further comprises: and the control corresponds to an operation set.
4. The method of claim 1, wherein the interface test case set comprises a subset of independent test cases corresponding to each control and a subset of combined test cases corresponding to a combination of controls;
generating the independent test case subsets corresponding to the controls comprises:
acquiring an operation set corresponding to the type of each control in the user interface screenshot, and generating an independent test case corresponding to each control according to the operation set corresponding to the type of each control and the coordinates of the control;
adding the independent test cases to independent test case sets corresponding to the controls;
generating a subset of the combined test cases for the corresponding control combination comprises:
all controls in the user interface screenshot are subjected to full-array combination to obtain each combination;
determining the operation sequence of different controls in each combination;
sequentially calling the independent test case subsets corresponding to each control according to the operation sequence to generate a combined test case corresponding to the operation sequence under the combination;
and adding the combined test case to a combined test case set of the corresponding control combination.
5. The method according to claim 4, wherein the generating of the independent test case corresponding to each control according to the operation set corresponding to the type to which the control belongs and the coordinates of the control comprises:
and acquiring an executable script template corresponding to each operation in an operation set corresponding to the type of the control, filling the coordinates of the control into the executable script template, and generating an independent test case corresponding to the control.
6. A problem scenario automatic reproduction method, characterized in that the method comprises:
acquiring a screenshot of a problem scene provided by a user as a problem screenshot;
recognizing the problem screenshot by using a pre-trained interface element recognition model to obtain each control in the problem screenshot and related information of the control; the related information of the control comprises the coordinate and the type of the control;
generating a problem test case set according to each control in the problem screenshot and the related information of the control;
and reproducing the problem scene by using the problem test case set.
7. A test case generation apparatus, the apparatus comprising:
the screenshot obtaining module is used for obtaining a user interface screenshot of the software to be tested;
the recognition module is used for recognizing the user interface screenshot by utilizing a pre-trained interface element recognition model to obtain each control in the user interface screenshot and related information of the control; the related information of the control comprises the coordinate and the type of the control;
and the interface test case generation module is used for generating an interface test case set according to each control and relevant information in the user interface screenshot.
8. An apparatus for automatic problem scenario reconstruction, the apparatus comprising:
the problem scene acquisition module is used for acquiring a screenshot of a problem scene provided by a user as a problem screenshot;
the recognition module is used for recognizing the problem screenshot by utilizing a pre-trained interface element recognition model to obtain each control in the problem screenshot and related information of the control;
the problem test case generation module is used for generating a problem test case set according to each control in the problem screenshot and the related information of the control;
and the scene reproduction module is used for reproducing the problem scene by utilizing the problem test case set.
9. An electronic device, comprising: one or more processors, memory;
the memory is for storing computer-executable instructions, and the processor is for executing the computer-executable instructions to implement the method of any one of claims 1 to 5.
10. A readable storage medium having stored thereon instructions that are executed to implement the method of any one of claims 1 to 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910654125.9A CN112241361A (en) | 2019-07-19 | 2019-07-19 | Test case generation method and device and problem scene automatic reproduction method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910654125.9A CN112241361A (en) | 2019-07-19 | 2019-07-19 | Test case generation method and device and problem scene automatic reproduction method and device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112241361A true CN112241361A (en) | 2021-01-19 |
Family
ID=74167248
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910654125.9A Pending CN112241361A (en) | 2019-07-19 | 2019-07-19 | Test case generation method and device and problem scene automatic reproduction method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112241361A (en) |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112685328A (en) * | 2021-03-12 | 2021-04-20 | 北京志翔科技股份有限公司 | Graphical interface testing method and device and storage medium |
| CN113407405A (en) * | 2021-05-18 | 2021-09-17 | 荣耀终端有限公司 | Game scene reproduction method, electronic equipment and system |
| WO2023275887A1 (en) * | 2021-06-28 | 2023-01-05 | L&T Technology Services Limited | System and method for automated software testing |
| CN116932414A (en) * | 2023-09-14 | 2023-10-24 | 深圳市智慧城市科技发展集团有限公司 | Method and equipment for generating interface test case and computer readable storage medium |
| WO2023220902A1 (en) * | 2022-05-16 | 2023-11-23 | 北京小米移动软件有限公司 | User interface testing methods and apparatus |
-
2019
- 2019-07-19 CN CN201910654125.9A patent/CN112241361A/en active Pending
Cited By (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112685328A (en) * | 2021-03-12 | 2021-04-20 | 北京志翔科技股份有限公司 | Graphical interface testing method and device and storage medium |
| CN112685328B (en) * | 2021-03-12 | 2021-06-08 | 北京志翔科技股份有限公司 | Graphical interface testing method and device and storage medium |
| CN113407405A (en) * | 2021-05-18 | 2021-09-17 | 荣耀终端有限公司 | Game scene reproduction method, electronic equipment and system |
| CN113407405B (en) * | 2021-05-18 | 2022-04-08 | 荣耀终端有限公司 | Game scene reproduction method and electronic equipment |
| WO2023275887A1 (en) * | 2021-06-28 | 2023-01-05 | L&T Technology Services Limited | System and method for automated software testing |
| WO2023220902A1 (en) * | 2022-05-16 | 2023-11-23 | 北京小米移动软件有限公司 | User interface testing methods and apparatus |
| CN116932414A (en) * | 2023-09-14 | 2023-10-24 | 深圳市智慧城市科技发展集团有限公司 | Method and equipment for generating interface test case and computer readable storage medium |
| CN116932414B (en) * | 2023-09-14 | 2024-01-05 | 深圳市智慧城市科技发展集团有限公司 | Method and equipment for generating interface test case and computer readable storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112241361A (en) | Test case generation method and device and problem scene automatic reproduction method and device | |
| CN109359056B (en) | Application program testing method and device | |
| CN114240882A (en) | Defect detection method and device, electronic equipment and storage medium | |
| CN109213419B (en) | Touch operation processing method and device and storage medium | |
| CN107659717B (en) | State detection method, device and storage medium | |
| CN111614990B (en) | Method and device for acquiring loading duration and electronic equipment | |
| CN109766473B (en) | Information interaction method and device, electronic equipment and storage medium | |
| CN112200040A (en) | Occlusion image detection method, device and medium | |
| CN107562500B (en) | Debugging device, method and equipment | |
| CN109358788B (en) | Interface display method and device and terminal | |
| CN112948704A (en) | Model training method and device for information recommendation, electronic equipment and medium | |
| CN116069612A (en) | Abnormality positioning method and device and electronic equipment | |
| CN111209195B (en) | Method and device for generating test case | |
| CN113032627A (en) | Video classification method and device, storage medium and terminal equipment | |
| CN112559309A (en) | Method and device for adjusting page performance acquisition algorithm | |
| CN112256563A (en) | Android application stability testing method and device, electronic equipment and storage medium | |
| CN106339452A (en) | Event responding method and device | |
| CN111666936A (en) | Labeling method, labeling device, labeling system, electronic equipment and storage medium | |
| CN112383661B (en) | Mobile terminal automatic test method and device, electronic equipment and storage medium | |
| CN110147817B (en) | Training data set generation method and device | |
| CN107526683B (en) | Method and device for detecting functional redundancy of application program and storage medium | |
| CN105912398A (en) | Memory detection method and device | |
| CN112446366A (en) | Image translation evaluating method and device for image translation evaluating | |
| CN111650554A (en) | Positioning method and device, electronic equipment and storage medium | |
| CN112346968B (en) | Automatic detection method and device for definition of multimedia file |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |