WO2020151301A1 - Procédé et appareil de génération de script de test basé sur un apprentissage de renforcement - Google Patents

Procédé et appareil de génération de script de test basé sur un apprentissage de renforcement Download PDF

Info

Publication number
WO2020151301A1
WO2020151301A1 PCT/CN2019/116263 CN2019116263W WO2020151301A1 WO 2020151301 A1 WO2020151301 A1 WO 2020151301A1 CN 2019116263 W CN2019116263 W CN 2019116263W WO 2020151301 A1 WO2020151301 A1 WO 2020151301A1
Authority
WO
WIPO (PCT)
Prior art keywords
test
state
behavior
target
states
Prior art date
Application number
PCT/CN2019/116263
Other languages
English (en)
Chinese (zh)
Inventor
李佳楠
张新琛
陈忻
黄伟东
孙震
Original Assignee
阿里巴巴集团控股有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 阿里巴巴集团控股有限公司 filed Critical 阿里巴巴集团控股有限公司
Publication of WO2020151301A1 publication Critical patent/WO2020151301A1/fr

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/36—Preventing errors by testing or debugging software

Definitions

  • the present disclosure relates to the field of testing technology, and in particular to a method and device for generating test scripts based on reinforcement learning.
  • Testing is the process of operating a program under specified conditions to find program errors, measure software quality, and evaluate whether it can meet design requirements.
  • Automated testing is a process that transforms human-driven testing behavior into machine execution.
  • testers need to manually write test scripts for automated testing, and the writing of test scripts often takes most of the testers' time.
  • one or more embodiments of this specification provide a method and device for generating test scripts based on reinforcement learning, so as to save labor costs for automated testing and make testing more convenient.
  • a method for generating test scripts based on reinforcement learning where the test scripts are used to test target software under test; the method includes:
  • the state includes a plurality of test states from the test initial state to the test target state, and the switching between the plurality of test states is triggered by the behavior, and one state corresponds to multiple Possible actions
  • a test script is obtained, the test script corresponds to an execution path from the initial test state to the test target state, and the execution path includes a sequence of behaviors that can reach the target state to pass the test script pair The target software under test is tested.
  • a test script generation device based on reinforcement learning is provided, the device is used to generate a test script; the device includes:
  • the information acquisition module is used to acquire the state and behavior used for testing, the state includes a plurality of test states from the test initial state to the test target state, and the switching between the plurality of test states is triggered by the behavior, One said state corresponds to multiple possible actions;
  • the model running module is used to run the reinforcement learning model to determine the corresponding Q value between each state and each behavior;
  • the script generation module is used to obtain a test script according to the Q value, the test script corresponding to an execution path from the initial test state to the test target state, and the execution path includes a sequence of behaviors that can reach the target state to Test through the test script.
  • a test script generation device based on reinforcement learning.
  • the device includes a memory and a processor.
  • the memory is used to store computer instructions that can run on the processor.
  • the processor is used to execute the The following steps are implemented when the computer commands:
  • the state includes a plurality of test states from the test initial state to the test target state, and the switching between the plurality of test states is triggered by the behavior, and one state corresponds to multiple Possible actions
  • a test script is obtained, the test script corresponds to an execution path from the initial test state to the test target state, and the execution path includes a sequence of behaviors that can reach the target state to pass the test script pair The target software under test is tested.
  • the method and device for generating test scripts based on reinforcement learning uses a reinforcement learning model to automatically generate test scripts through the reinforcement learning model, thereby saving labor costs.
  • Fig. 1 is an application system of a test script generation method provided by at least one embodiment of this specification
  • FIG. 2 is a scenario of automatic path generation of a login example provided by at least one embodiment of this specification
  • Figure 3 is a process of automatically generating paths according to Figure 2;
  • FIG. 4 is a flow of automatic generation of test scripts based on Qianghu learning provided by at least one embodiment of this specification
  • Fig. 5 is a test script generation device based on reinforcement learning according to at least one embodiment of this specification.
  • At least one embodiment of this specification uses a reinforcement learning model to automatically generate test scripts.
  • the system may include: a test script generation device 11 based on reinforcement learning (hereinafter referred to as a script generation device), an agent device 12 and a device under test 13.
  • a test script generation device 11 based on reinforcement learning (hereinafter referred to as a script generation device)
  • an agent device 12 based on reinforcement learning (hereinafter referred to as a script generation device)
  • a device under test 13 the system to which this method is applied. 1.
  • the system may include: a test script generation device 11 based on reinforcement learning (hereinafter referred to as a script generation device), an agent device 12 and a device under test 13.
  • a reinforcement learning model can be run in the script generation device 11.
  • the reinforcement learning model may be a Q Learning model, or may also be a deep Q-Network (Deep Q-Network, referred to as DQN) model, etc.
  • DQN deep Q-Network
  • the agent device 12 can receive the command sent by the script generation device 11 and operate the device under test 13 according to the command.
  • the proxy device 12 can receive a command to execute a certain action sent by the script generation device 11, and the proxy device 12 can operate and control the device under test 13 to execute the action.
  • the device under test 13 may feed back the feedback result after executing the action to the proxy device 12.
  • the feedback result may include the state reached after executing the action, and the proxy device 12 may return the feedback state to the script generating device 11 to make the script The generating device 11 performs the next processing.
  • the device under test 13 may be a device running the test software.
  • the device under test 13 may be a mobile terminal device or a PC terminal device.
  • FIG. 1 takes the mobile terminal as an example.
  • the purpose of Q_learning is to update the Q table to make it an accurate Q table.
  • Such a Q table can be maintained in the script generation device 11, and the Q table can include status and action.
  • Table 1 The following example in Table 1:
  • the "state” may include multiple test states from the test initial state to the test target state.
  • the test state may include the test initial state and the test intermediate state.
  • the initial state of the test can be that the user opens the application homepage
  • the test target state can be that the user enters the application homepage.
  • the test initial state to the test target state may include multiple test intermediate states. For example, after the user clicks the login button, the page jumps to the login page, and the displayed login page can be a test state. For another example, after the user enters the user name, the entered user name is displayed on the page, which can be used as a test state.
  • Switching between multiple test states is triggered by action actions. For example, if the user clicks the login button, this operation triggers the test state to switch from the application homepage display to the login page display.
  • the various states mentioned above are observable, and the display information of the current interface can also be obtained through some excuses provided by the system as the state.
  • an android device can automatically obtain the observation status of the current interface control information through adb dump.
  • the state of the Q table can be designed as follows: Taking the android device as an example, the description information of the current device interface controls can be obtained through adb dump and recorded in an xml format file, which can be recorded The type and coordinate information of all controls on the current interface. A state vector of the interface state can be extracted to identify the state of the current interface. The state vector represents the test state. The state vector can include multiple state features, and each state feature corresponds to the target software under test running in the test state. An interface description dimension of the interface.
  • the state feature x1 represents the number of controls on the current interface
  • the state feature x2 represents the maximum level of the current interface layout
  • x3 is the total area of all controls on the current interface
  • x4 is the x value of the average center coordinates of all controls
  • x5 is the y value of the average center coordinates of all controls.
  • the number of controls, the number of levels, the total area, etc. can each be regarded as an interface description dimension. Using this information, the test status of an interface can be roughly expressed in a numerical manner.
  • the "behavior” multiple action operations that may be encountered in a certain state. In the test scenario, you can test as many possible behaviors as possible in a certain state.
  • behavior action can be the user's operational behavior, such as clicking the login button.
  • the method in this specification can also be applied to other types of testing, such as functional testing or interface testing, and is not limited to UI testing.
  • the login button is in a certain position on the login page, but the user may not click on the correct position when clicking, for example, clicking an unambiguous empty position, or clicking a wrong Place, wait.
  • the page screen can be divided into multiple units, each unit is represented by a position, and clicking on that position is used as an action.
  • click (30,10), click (10,10), the above (30,10) can be the coordinates of the click position. How to divide it can be customized.
  • a design method of click coordinates In order to improve the universality of the operation, the definition of the operation should be as universal as possible.
  • a click operation can be defined as a click on the interface coordinates, and the specific coordinates can be divided according to the screen resolution. For example, define a total of 20*40 for all click operations.
  • Unit width (current interface width/20)
  • Unit length (current interface length/40)
  • the design of the click operation coordinate can manually specify the size of the grid.
  • the grid corresponding to the click operation can be 20*40 or 30*60, which can be adjusted according to the performance of the operation in specific use.
  • actions can also be included, such as input behavior and sliding behavior.
  • the specific type of action can be related to the design of the interface function of the test software. For example, if the user wants to enter a user name and password on the page, it can include an input action; if the page requires the user to slide, it can include a slide action. Further, the sliding action may also include sliding to the left and sliding to the right, which are respectively an action.
  • Other types of actions can also be designed as universally as possible. For example, for sliding operations, four parameters can be specified, including the coordinates of the specified starting point (x, y) and the coordinates of the ending point (x, y).
  • test case When testing software, there are many situations that need to be tested. For example, when testing a certain software, the software can include multiple tests. Here are a few test cases, each of which can be called a test case:
  • Test case 1 Enter the correct user name and password, and click the submit button to verify whether you can log in correctly. (Normal input)
  • Test case 2 Enter the wrong user name or password, the authentication login will fail, and the corresponding error message will be prompted. (Error checking);
  • Test case 3 If the user name and password are too short or too long, what should be done (security, whether there is a prompt when the password is too short)
  • Each case can design its own Q table, but in order to improve the test efficiency, you can try to design a universal Q table.
  • the correct click position can be different for the case of different test target states, but all possible positions can be listed in the Q table, and the behavior in the Q table can include respective corresponding Clicking behaviors at different screen coordinate positions can be guided by setting different reward values for different position behaviors in different cases.
  • Different tested cases can have the same action, and some cases have exactly the same action.
  • part of the status may be the same.
  • the initial status of both login and registration cases may go through the registration login page, and the final status may also reach the application homepage.
  • the intermediate links may be different. If there are two different cases, after the corresponding operations are performed, the states on a series of operation paths are the same, then there will be situations where the states are also exactly the same, but this kind of situation is rare.
  • the first test target state and the second test target state are two different test target states (these are two different cases), for example, the first test target state can display the login success page, and the second test The target status may be a pop-up error message. Then, when the test target state is the first test target state, the multiple test state statuses are used when the test target state is the second test target state. Status, at least part of the test status can be the same. As mentioned above, the initial test state of the two cases may be the same as the intermediate test state.
  • At least part of the behaviors used when the test target state is the first test target state and the plurality of behavior actions used when the test target state is the second test target state may be the same.
  • both cases include click operations and include the same multiple possible click coordinates.
  • the corresponding Q value between the state and behavior Can be different. For example, suppose the above test case 1 and test case 2 can use the same Q table. When the Q value is updated, for test case 1, if the login is successful, the reward value is set higher; and for test case 2, if If login fails, the reward value is set higher. Then the Q values in the Q tables of the two cases can be different.
  • the script generation device 11 can update the Q table, and after the update is completed, the execution path during the test is obtained according to the Q table.
  • Figure 2 illustrates the automatic path generation scenario of the login example.
  • Figure 3 is based on the path automatic generation process shown in Figure 2. Please refer to Figure 2 and Figure 3.
  • the method may include:
  • step 300 the Q table is initialized, and the Q table includes the state and behavior.
  • Table 2 is the Q table in the user login application example.
  • state 1 may be the home page of the application; a login button is displayed on the home page of the application, and the corresponding operation action in state 1 may include clicking on various positions of the home page, including clicking the login button.
  • State 2 can be an application login page, which displays input boxes for the user name and password, and also includes the display of a login button.
  • possible actions may include clicking on different locations, and may also include input operations such as entering a user name or entering a password.
  • input operations such as entering a user name or entering a password.
  • State 3 may be to display the user name entered by the user. There can also be state 4, state 5 and other states before the user successfully logs in to the application.
  • the state switch be triggered. For example, if the user clicks on a meaningless location on the application homepage, the state may remain in state 1, and it will not switch to state 2.
  • the state is a number of intermediate states that may be reached during the test.
  • the action action is the possible user behavior in each state.
  • the Q value in the table indicates what behavior the user takes, which is more likely to achieve the goal of this test. If the behavior performed by the user is closer to the test target, a higher Q value can be set; it is equivalent to a guide to the user's behavior, so that the path to the test target can be found more quickly. When in a certain state, select a certain behavior action in that state, the behavior with a higher Q value will have a greater probability of being selected.
  • the status, behavior, and Q value setting in the Q table are all related to the characteristics of the test software itself and the test target status, and are determined accordingly.
  • Different test software and test target states can have different states and behaviors, and different Q values can be set to guide the generation of test paths.
  • the values in the Q table can be initialized to all 0s, or other values can be used.
  • the Q table is updated by means of Q Learning to obtain the updated Q table.
  • the Q table includes Q values corresponding to various behaviors in each state.
  • the script generation device 11 can randomly select an action in that state, and notify the agent device 12 to execute the action.
  • the agent device 12 controls the execution behavior of the device under test 13 according to instructions, and the ring device under test 13 is equivalent to an operating environment of the software under test.
  • the proxy device 12 may feed back a feedback result to the script generation device 11, and the feedback result may include whether the state reached after executing the action is the test target state.
  • the process of updating the Q table can be performed according to the conventional Q Learning method, which will not be described in detail.
  • state 1 an action is randomly selected to instruct the agent device 12 to execute.
  • the proxy device 12 reports back to enter state 2.
  • the state 2 is not the target state (successful login).
  • the script generation device 11 continues to select an action from the action corresponding to state 2, and continues to instruct the proxy device 12 to execute .
  • the result returned by the proxy device 12 is that a successful login has been achieved.
  • a reward value can be given, and the reward value may be used to update the action in the state that triggered the login. For example, after the user enters the user name and password, and clicks the login button, it leads to the successful login to the application. You can enter the user name, enter the password, and click the login button in the above three states. The value is updated slightly higher. For example, the Q values corresponding to other actions in the same state are all 0, and the Q values of these actions can be 0.8 or 0.9.
  • step 304 the execution path of the test script is obtained according to the Q value, and the execution path includes a sequence of actions that can reach the target state.
  • the execution path of the test script can be obtained accordingly.
  • the action with the highest Q value in each state can be selected to form a behavior sequence, which is the execution path of the test.
  • the script generating device 11 has automatically generated a test script so far, and the test script can be sent to the agent device 12 for execution, and the test can be performed through the test script.
  • the generated test script may be at least one.
  • the test script generation method of this example can save labor costs by using a reinforcement learning model and can generate better test scripts.
  • the above example is based on Q Learning in reinforcement learning, and other reinforcement learning models can also be used for processing, such as DQN.
  • the DQN network can be pre-trained, and the input of the DQN network can be the state in the Q table, which is the state during the test, such as the image of the software interface of the test.
  • the output of the DQN model can be the Q value corresponding to each action action in the corresponding state.
  • the Q value is the state-action value, which is a function of state and action.
  • the trained DQN network can be obtained.
  • the Q values corresponding to each behavior in each state have been obtained.
  • test scripts The method for generating test scripts is not limited to the above Q Learning and DQN, and other reinforcement learning models can also be used.
  • the flow in Figure 4 illustrates the processing flow when reinforcement learning is applied to the automatic generation of test scripts, which may include the following processing:
  • step 400 a status status and a behavior action for testing are obtained, the status includes a plurality of test states from the test initial state to the test target state, and the switching of the plurality of test states is triggered by the behavior, one The state corresponds to multiple possible behaviors;
  • step 402 run a reinforcement learning model to determine the corresponding Q value between each state and each behavior
  • step 404 the execution path of the test script is obtained according to the Q value, and the execution path includes a sequence of actions that can reach the target state, so as to pass the test script for testing.
  • the model based on reinforcement learning realizes the automatic generation of automated test cases, which greatly reduces the investment in labor costs, and can achieve the effect of automatically generating use cases without manually writing use case scripts.
  • Fig. 5 provides a test script generation device based on reinforcement learning according to at least one embodiment of this specification, and the device is used to generate a test script.
  • the device may include: an information acquisition module 51, a model running module 52, and a script generation module 53.
  • the information acquisition module 51 is configured to acquire the state and behavior used for testing, the state includes a plurality of test states from the test initial state to the test target state, and the switching between the plurality of test states is triggered by the behavior , One state corresponds to multiple possible behaviors;
  • the model running module 52 is configured to run a reinforcement learning model to determine the corresponding Q value between each state and each behavior;
  • the script generation module 53 is configured to obtain a test script according to the Q value, the test script corresponding to an execution path from the initial test state to the test target state, and the execution path includes a sequence of actions that can reach the target state, To pass the test script for testing.
  • the behavior when the type of the behavior is click, the behavior includes click behaviors corresponding to different interface coordinates, and the interface is the running interface of the target software under test.
  • the model running module 52 is specifically configured to: initialize the Q table, which includes the state and behavior; update the Q table by means of Q Learning, to obtain the updated Q table .
  • the model running module 52 is specifically used to train a DQN, where the input of the DQN is the state, and the output is the Q value corresponding to the state and behavior; and the DQN after the training is obtained.
  • At least one embodiment of the present specification also provides a test script generation device based on reinforcement learning.
  • the device includes a memory and a processor.
  • the memory is used to store computer instructions that can run on the processor; the processor is used for The following steps are implemented when the computer instructions are executed:
  • the state includes a plurality of test states from the test initial state to the test target state, and the switching between the plurality of test states is triggered by the behavior, and one state corresponds to multiple Possible actions
  • a test script is obtained, the test script corresponds to an execution path from the initial test state to the test target state, and the execution path includes a sequence of behaviors that can reach the target state to pass the test script pair The target software under test is tested.
  • one or more embodiments of this specification can be provided as a method, a system, or a computer program product. Therefore, one or more embodiments of this specification may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, one or more embodiments of this specification may adopt a computer program implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code. The form of the product.
  • computer-usable storage media including but not limited to disk storage, CD-ROM, optical storage, etc.
  • One or more embodiments of this specification may be described in the general context of computer-executable instructions executed by a computer, such as program modules.
  • program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types.
  • One or more embodiments of this specification can also be practiced in distributed computing environments. In these distributed computing environments, tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media including storage devices.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Hardware Design (AREA)
  • Quality & Reliability (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Debugging And Monitoring (AREA)

Abstract

Les modes de réalisation de la présente invention concernent un procédé et un appareil de génération de script de test basé sur un apprentissage de renforcement. Le procédé comprend : l'acquisition d'états et de comportements utilisés pour tester, les états comprenant une pluralité d'états de test allant d'un état de test initial à un état de test cible, la commutation entre la pluralité des états de test étant déclenchée par les comportements, et un des états correspondant à une pluralité de comportements possibles ; l'exécution d'un modèle d'apprentissage de renforcement pour déterminer une valeur Q correspondante entre chacun des états et chaque comportement ; selon la valeur Q, l'obtention d'un script de test, le script de test correspondant à un chemin d'exécution de l'état de test initial à l'état de test cible, le chemin d'exécution comprenant une séquence de comportements qui peuvent atteindre l'état cible de façon à tester le logiciel cible à tester au moyen du script de test.
PCT/CN2019/116263 2019-01-21 2019-11-07 Procédé et appareil de génération de script de test basé sur un apprentissage de renforcement WO2020151301A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201910054193.1 2019-01-21
CN201910054193.1A CN109901994B (zh) 2019-01-21 2019-01-21 一种基于强化学习的测试脚本生成方法和装置

Publications (1)

Publication Number Publication Date
WO2020151301A1 true WO2020151301A1 (fr) 2020-07-30

Family

ID=66943961

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2019/116263 WO2020151301A1 (fr) 2019-01-21 2019-11-07 Procédé et appareil de génération de script de test basé sur un apprentissage de renforcement

Country Status (2)

Country Link
CN (1) CN109901994B (fr)
WO (1) WO2020151301A1 (fr)

Families Citing this family (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109901994B (zh) * 2019-01-21 2020-05-15 阿里巴巴集团控股有限公司 一种基于强化学习的测试脚本生成方法和装置
CN110442515B (zh) * 2019-07-12 2023-07-25 创新先进技术有限公司 应用测试方法、装置、设备及可读存储介质
CN111008154B (zh) * 2019-12-31 2021-09-28 南京大学 一种基于强化学习的Android应用自动测试方法及系统
CN111694755B (zh) * 2020-07-31 2023-07-18 抖音视界有限公司 应用程序测试方法、装置、电子设备及介质
CN112383482B (zh) * 2020-11-16 2021-10-08 北京邮电大学 基于数据平面的动态q值路由计算方法及装置
CN112685318A (zh) * 2021-01-07 2021-04-20 广州三星通信技术研究有限公司 产生测试脚本的方法和系统

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108334439A (zh) * 2018-03-14 2018-07-27 百度在线网络技术（北京）有限公司 一种压力测试方法、装置、设备和存储介质
US20180246801A1 (en) * 2017-02-24 2018-08-30 International Business Machines Corporation Controlling a system under test using a cognitive control based test runner
CN109901994A (zh) * 2019-01-21 2019-06-18 阿里巴巴集团控股有限公司 一种基于强化学习的测试脚本生成方法和装置

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104572123B (zh) * 2015-01-27 2019-05-14 Oppo广东移动通信有限公司 一种脚本生成方法及装置
CN106991030B (zh) * 2017-03-01 2020-04-14 北京航空航天大学 一种基于在线学习的系统功耗优化的轻量级方法
CN109062782B (zh) * 2018-06-27 2022-05-31 创新先进技术有限公司 一种回归测试用例的选择方法、装置及设备

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20180246801A1 (en) * 2017-02-24 2018-08-30 International Business Machines Corporation Controlling a system under test using a cognitive control based test runner
CN108334439A (zh) * 2018-03-14 2018-07-27 百度在线网络技术（北京）有限公司 一种压力测试方法、装置、设备和存储介质
CN109901994A (zh) * 2019-01-21 2019-06-18 阿里巴巴集团控股有限公司 一种基于强化学习的测试脚本生成方法和装置

Also Published As

Publication number Publication date
CN109901994B (zh) 2020-05-15
CN109901994A (zh) 2019-06-18

Similar Documents

Publication Publication Date Title
WO2020151301A1 (fr) Procédé et appareil de génération de script de test basé sur un apprentissage de renforcement
CN109726093B (zh) 用于执行测试用例的方法、设备和计算机程序产品
CN108959068B (zh) 软件界面测试方法、设备及存储介质
US8370816B2 (en) Device, method and computer program product for evaluating a debugger script
US20080209405A1 (en) Distributed debugging for a visual programming language
WO2017165071A1 (fr) Analyse de régression visuelle
CN103984626A (zh) 一种生成测试用例脚本的方法及装置
US11468880B2 (en) Dialog system training using a simulated user system
US20170371304A1 (en) Method and system for determining mapping of test case(s) with code snippets of computer program
CN111475390A (zh) 日志采集系统部署方法、装置、设备及存储介质
US20100312541A1 (en) Program test device and program
CN111400167A (zh) Redfish服务合规性验证方法、装置及设备和介质
US9612944B2 (en) Method and system for verifying scenario based test selection, execution and reporting
US20050203717A1 (en) Automated testing system, method and program product using testing map
CN107783867B (zh) 手势测试方法、装置、计算机设备和存储介质
US7827540B2 (en) Method for program debugging
CN111414309A (zh) 应用程序的自动化测试方法、计算机设备和存储介质
CN110717315A (zh) 系统数据批量修改方法、装置、存储介质及电子设备
KR101753314B1 (ko) 이미지 매칭 학습을 이용한 어플리케이션 테스트 방법 및 이를 수행하기 위한 장치
CN111949510B (zh) 测试处理方法、装置、电子设备及可读存储介质
CN112486802A (zh) Flutter测试辅助方法及装置
CN113220511A (zh) 测试bios的方法、设备和计算机可读存储介质
US10545858B2 (en) Method for testing a graphical interface and corresponding test system
CN112445711A (zh) 一种基于Web网页可视化拖拽生成模拟测试场景的测试方法
US20210133628A1 (en) Analysing Machine-Learned Classifier Models

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19911865

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 19911865

Country of ref document: EP

Kind code of ref document: A1