TWI750849B - Automatic test system and method for testing application of device under test - Google Patents

Abstract

An automatic test system and a method for testing an application of a device under test (DUT) are provided. The method includes: obtaining a test script by a test device, wherein the test script includes at least one test command and at least one verification command; obtaining interpretation information from a cloud server by the test device; executing the at least one test command according to the interpretation information to operate the application of the DUT by the test device, and obtaining a first parameter set of the application; pre-storing a plurality of verification information, selecting a first verification information from the plurality of verification information by executing the at least one verification command according to the interpretation information, and determining if the first parameter set matching the first verification information to generate a test report; and outputting the test report by the test device.

Description

Automated test system and method for testing device-under-test applications

The present invention relates to an automatic testing system and method for testing an application program of a device under test.

In order to meet user needs, the application (application, APP) used by the terminal device needs to be updated frequently. In order to ensure the quality of the application, the manufacturer needs to use the test program to test the application. Manufacturers need to write different test programs according to different terminal devices (for example, different operating systems, hardware configurations or software configurations). Generally speaking, test programs need to be written by testers with professional skills, so manufacturers need to spend a lot of costs on human resources. In addition, with the frequent revision of the application, the tester also needs to spend a lot of time rewriting the test program. Therefore, how to automate the process of testing applications is one of the goals of those in the art.

The present invention provides an automatic testing system and method, which can automatically test the application program of the device under test (DUT), so that the user can concentrate on improving the function of the application program.

An automatic testing system for testing an application program of a device to be tested of the present invention includes a cloud server and a testing device. The test apparatus includes a processor, a storage medium, and a transceiver. The transceiver communicates with the cloud server and the device under test. The storage medium stores multiple modules. The processor is coupled to the storage medium and the transceiver, and accesses and executes a plurality of modules, wherein the plurality of modules includes a data collection module, an execution module, a verification module and an output module. The data collection module obtains the test script through the transceiver, and obtains the interpretation information from the cloud server, wherein the test script includes at least one test command and at least one verification command. The execution module executes at least one test command according to the interpretation information to operate the application program of the device under test, and obtains the first parameter set of the application program. The verification module pre-stores a plurality of verification information, executes at least one verification command according to the interpretation information to select first verification information from the plurality of verification information, and determines whether the first parameter set matches the first verification information to generate a test report. The output module outputs the test report through the transceiver.

In an embodiment of the present invention, the above-mentioned data collection module obtains a general script through the transceiver, wherein the general script includes at least one general command, and the execution module executes the at least one general command according to the interpretation information to operate the device under test. application to obtain the first parameter set.

In an embodiment of the present invention, the above-mentioned interpretation information includes first interpretation information corresponding to the first operating system, wherein the data collection module obtains the device information from the device under test through the transceiver, and the execution module according to The device information is used to select the first interpretation information matching the device information from the interpretation information, and execute at least one test command to operate the application program according to the first interpretation information.

In an embodiment of the present invention, the above-mentioned verification module selects the first interpretation information matching the device information from the interpretation information according to the device information, and executes at least one verification command according to the first interpretation information to multiple The first verification information is selected from the verification information.

In an embodiment of the present invention, the above-mentioned execution module pre-stores a plurality of external test scripts, wherein at least one test instruction instructs the execution module to call a first external test script in the external test scripts, and operate according to the first external test script The app for the device under test.

In an embodiment of the present invention, the above-mentioned transceiver is communicatively connected to the second device under test, wherein the execution module executes at least one test command according to the interpretation information to operate the second application program of the second device under test, and obtains the corresponding In the second parameter set of the second application, the verification module determines whether the second parameter set matches the first verification information to generate a second test report, and the output module outputs the second test report through the transceiver.

A method for testing an application program of a device under test of the present invention is suitable for an automatic test system including a cloud server and a test device, wherein the automatic test method includes: obtaining a test script from the test device, wherein the test script includes at least one test instruction and at least one verification instruction; the test device obtains the interpretation information from the cloud server; the test device executes at least one test instruction according to the interpretation information to operate the application program of the device under test, and obtains the first parameter set of the application program; Pre-store a plurality of verification information by the test device, execute at least one verification command according to the interpretation information to select first verification information from the plurality of verification information, and determine whether the first parameter set matches the first verification information to generate a test report; and The test device outputs a test report.

Based on the above, the automatic test system of the present invention can generate instructions for testing the application program of the device under test based on an easy-to-write script. The cloud server can provide interpretation information for different operating systems to the test device for download. The test device can read the content of the script by using the interpretation information, so as to test the application program according to the flow in the script. For different applications, the test device can store different verification information. The testing device can select verification information corresponding to the application under test according to the script, and judge whether the test result is normal according to the verification information.

In order to make the content of the present invention more comprehensible, the following specific embodiments are given as examples according to which the present invention can indeed be implemented. Additionally, where possible, elements/components/steps using the same reference numerals in the drawings and embodiments represent the same or similar parts.

1 is a schematic diagram of an automatic test system 10 for testing an application program of a device under test according to an embodiment of the present invention, wherein the automatic test system 10 can be used to test the application program installed in the device under test, wherein the test program The device is, for example, an electronic device with computing capabilities. The automatic testing system 10 may include a testing device 100 and a cloud server 200 . The test device 100 is communicatively connected to the cloud server 200 . The cloud server 200 may have a processing unit (such as a processor but not limited thereto), a communication unit (such as various communication chips, mobile communication chips, bluetooth chips, WiFi chips, etc., but not limited thereto) and a storage unit (such as : removable random access memory, flash memory, hard disk, etc. (but not limited to) and other necessary components to run the cloud server 200 . FIG. 2 is a schematic diagram of the testing device 100 according to an embodiment of the present invention. The testing apparatus 100 may include a processor 110 , a storage medium 120 and a transceiver 130 .

The processor 110 is, for example, a central processing unit (CPU), or other programmable general-purpose or special-purpose micro control unit (micro control unit, MCU), microprocessor (microprocessor), digital signal processing digital signal processor (DSP), programmable controller, application specific integrated circuit (ASIC), graphics processor (graphics processing unit, GPU), image signal processor (image signal processor, ISP) ), image processing unit (IPU), arithmetic logic unit (ALU), complex programmable logic device (CPLD), field programmable gate array (field programmable gate array) , FPGA) or other similar elements or a combination of the above. The processor 110 may be coupled to the storage medium 120 and the transceiver 130 , and access and execute a plurality of modules and various application programs stored in the storage medium 120 .

The storage medium 120 is, for example, any type of fixed or removable random access memory (random access memory, RAM), read-only memory (ROM), and flash memory (flash memory). , a hard disk drive (HDD), a solid state drive (SSD), or similar components or a combination of the above components for storing a plurality of modules or various application programs executable by the processor 110 . In this embodiment, the storage medium 120 can store a plurality of modules including a data collection module 121 , an execution module 122 , a verification module 123 , and an output module 124 , the functions of which will be described later.

The transceiver 130 transmits and receives signals in a wireless or wired manner. Transceiver 130 may also perform operations such as low noise amplification, impedance matching, frequency mixing, up or down frequency conversion, filtering, amplification, and the like. The test device 100 can be communicatively connected to the cloud server 200 and the device under test through the transceiver 130 .

The data collection module 121 can obtain the general script and the test script for testing the application program of the device under test through the transceiver 130 , and can obtain the interpretation information from the cloud server 200 through the transceiver 130 . The interpretation information can be used to decode the general script and the test script, and assist the testing apparatus 100 to understand the meaning of the general script and the instructions in the test script. The general script and the test script of this embodiment are text files generated by software with a text editing function such as Excel. Compared with writing programming languages such as C language, the threshold for writing general scripts and test scripts of this embodiment is lower. In other words, even if the user does not have a programming background, the user can easily write the general script and the test script of this embodiment. Users can write their own test scripts for a specific application of a specific device under test. The content of the test script may vary based on factors such as the device type of the device under test, the operating environment, or the deployment of system parameters.

The data collection module 121 can obtain various interpretation information corresponding to various operating systems (or various application programs) from the cloud server 200 , and can obtain the device information of the device under test from the device under test through the transceiver 130 . Obtaining multiple interpretation information ensures that each test device 100 has a dedicated algorithm for each application to process each application. Device information can include information such as hardware device information (eg: the ID of the device under test, the location of the connected server), operating system or application environment (eg: the path or name of the application). The execution module 122 can select interpretation information suitable for the device under test according to the device information of the device under test. For example, the execution module 122 may select the first interpretation information from a variety of interpretation information in response to the matching of the operating system corresponding to the first interpretation information and the operating system recorded in the device configuration. The selected first interpretation information can be used to execute the test process of the device under test.

The generic script can record at least one generic command applicable to a variety of devices under test or a variety of applications. The test script can record at least one test command and at least one verification command dedicated to a specific device under test or a specific application. In one embodiment, the general script or test script may also record the number of tests for the application. After the data collection module 121 obtains the interpretation information, the general script and the test script, the data collection module 121 can arrange the general script and the test script into a work queue. The execution module 122 can execute the scripts in the work queue in sequence. When it is the turn to execute the general script and the test script (for example, when the test device 100 finds the device under test through the transceiver 130 ), the execution module 122 can execute at least one general command in the general script and one of the test scripts according to the interpretation information. The at least one test command is used to operate the application program of the device under test according to at least one general command and at least one test command.

Table 1 is an example of general commands. The execution module 122 can execute the general command "RunExcel". Based on the execution method "excelName", the execution module 122 can select the test script to be opened according to the file name, and select the function in the field corresponding to the action "sheetName" in the test script. Next, the execution module 122 can input the attribute value "sheetValue" to the function to execute the corresponding action. Table 1 instruction execution method action attribute value RunExcel excelName sheetName sheetValue

Table 2 is an example of the test command. In one embodiment, the execution module 122 may pre-store one or more external test scripts. The execution module 122 can execute the test instruction "Script" to open an external test script matching the execution method "methodName", so as to execute the instructions in the external test script to operate the application program of the device under test.

The execution module 122 can execute the test instruction "Object". Based on the execution method "byXPath", the execution module 122 may find elements in the application whose paths match the attribute value "xpathValue", and operate on the elements. Alternatively, the execution module 122 may look for a component in the application whose name matches the attribute value "idValue", and operate on the component. For example, the execution module 122 can click on the element according to the action "click", can input information into the element according to the action "sendKeys", or can long press the element according to the action "longPress".

The execution module 122 can execute the test command "Coordinate". The execution module 122 can locate the coordinates in the graphical user interface (GUI) of the display application according to the attribute values "xValue, yValue", and operate the components on the coordinates, wherein the GUI can be, for example, the device under test. displayed on the display interface. For example, the execution module 122 can click the component on the coordinate according to the action "click", slide the component on the coordinate according to the action "swipe", and move and drag the component on the coordinate to other places according to the action "dragAndDrop", Or you can long press the component on the coordinates according to the action "longPress". Table 2 instruction execution method action attribute value Script methodName Object byXPath, byID click sendKeys longPress xpathValue, idValue Coordinate click swipe dragAndDrop longPress xValue, yValue

After the execution module 122 operates the application program according to at least one general command or at least one test command in the general script or the test script, the parameter set of the application program will be updated in response to the operation. The execution module 122 can obtain the updated parameter set from the application.

After the execution module 122 obtains the updated parameter set from the application, the verification module 123 can verify whether the parameter set of the application meets the user's expectation. Specifically, the verification module 123 can pre-store a plurality of verification information. The verification module 123 can execute at least one verification command in the test script according to the interpretation information to select verification information suitable for the application program of the device under test from a plurality of verification information. Table 3 is an example of the verification command. The verification module 123 can execute the verification instruction "Validate", so as to select the verification information specified by the test script from a plurality of verification information based on the execution method "methodName". table 3 instruction execution method action attribute value Validate methodName

The verification module 123 can determine whether the parameter set output by the application program matches the verification information to generate a test report. The verification module 123 can compare the parameter sets and verify the information according to, for example, regular notation. For example, if the set of parameters output by the application does not match the verification information, the test report may record that the test results of the application of the device under test are not as expected. If the set of parameters output by the application matches the verification information, the test report can record that the application of the device under test passed the test. The output module 124 can output the test report through the transceiver 130 for the user's reference.

In one embodiment, the storage medium 120 of the testing apparatus 100 can pre-store the number of tests of the application. The test device 100 can perform multiple tests on the application program of the device under test according to the number of tests and generate corresponding test reports.

In one embodiment, the test device 100 may be communicatively connected to a plurality of devices under test through the transceiver 130, thereby simultaneously testing applications of each of the plurality of devices under test. The testing device 100 can test multiple applications through the same or different scripts. For example, the test device 100 can be communicatively connected to the first device under test and the second device under test. The execution module 122 can execute at least one general command in the general script and at least one test command in the test script according to the interpretation information obtained from the cloud server 200 to operate the first application corresponding to the first device under test and A second application program corresponding to the second device under test, and a first parameter set corresponding to the first application program and a second parameter set corresponding to the second application program are obtained. The verification module 123 can select one verification information from a plurality of verification information according to at least one verification command in the test script. The verification module 123 can determine whether the first parameter set matches the verification information to generate the first test report of the first device under test, and can determine whether the second parameter set matches the verification information to generate the second test report of the second device under test. testing report. The output module 124 can output the first test report and the second test report through the transceiver 130 .

FIG. 3 is a flowchart illustrating a method for testing an application program of a device under test according to an embodiment of the present invention, wherein the method may be implemented by the automatic test system 10 as shown in FIG. 1 . In step S301, a test script is obtained from the test device, wherein the test script includes at least one test command and at least one verification command. In step S302, the test device obtains the interpretation information from the cloud server. In step S303, the test device executes at least one test command according to the decoded information to operate the application program of the device under test, and obtains a first parameter set of the application program. In step S304, the testing device pre-stores a plurality of verification information, executes at least one verification command according to the interpretation information to select the first verification information from the plurality of verification information, and determines whether the first parameter set matches the first verification information to generate testing report. In step S305, a test report is output by the test device.

In summary, the present invention has the following features. (1) Logical business separation can be independent, and writing test scripts does not need to write code. (2) Can use text editing programs such as Excel to write test scripts, lowering the testers' threshold for automated testing. (3) Test applications for devices under test with different operating systems (eg iOS or Android). (4) Some scripts are available for different applications. (5) An application program that can test multiple devices under test at the same time. (6) An independent test report can be generated for each device to be tested.

Although the present invention has been disclosed above by the embodiments, it is not intended to limit the present invention. Anyone with ordinary knowledge in the technical field can make some changes and modifications without departing from the spirit and scope of the present invention. Therefore, The protection scope of the present invention shall be determined by the scope of the appended patent application.

10: Automatic test system 100: Test device 110: Processor 120: Storage Media 121: Data Collection Module 122: Execution mod 123: Verification Module 124: output module 130: Transceiver 200: Cloud server S301, S302, S303, S304, S305: Steps

FIG. 1 is a schematic diagram of an automatic test system for testing an application program of a device under test according to an embodiment of the present invention. FIG. 2 is a schematic diagram of a testing device according to an embodiment of the present invention. 3 is a flowchart illustrating a method for testing an application of a device under test according to an embodiment of the present invention.

S301, S302, S303, S304, S305: Steps

Claims (7)

An automatic test system for testing an application program of a device under test, comprising: a cloud server; and a test device, including: a transceiver, communicating with the cloud server and the device under test; a storage medium, storing multiple a module; and a processor, coupled to the storage medium and the transceiver, and accessing and executing the multiple modules, wherein the multiple modules include: a data collection module, through which the transceiver The device obtains a test script, and obtains interpretation information from the cloud server, wherein the test script includes at least one test command and at least one verification command; the execution module executes the at least one test command according to the interpretation information to operate the application program of the device under test, and obtain a first parameter set of the application program; the verification module pre-stores a plurality of verification information, and executes the at least one verification command according to the interpretation information to automatically The plurality of verification information selects first verification information, and determines whether the first parameter set matches the first verification information to generate a test report; and an output module outputs the test report through the transceiver. The automatic testing system of claim 1, wherein the data collection module obtains a generic script through the transceiver, wherein the generic script includes at least a general-purpose instruction, wherein the execution module executes the at least one general-purpose instruction according to the interpretation information to operate the application program of the device under test to obtain the first parameter set. The automatic test system of claim 1, wherein the interpretation information includes first interpretation information corresponding to a first operating system, wherein the data collection module passes the transceiver to the device under test Obtaining device information, wherein the execution module selects the first interpretation information matching the device information from the interpretation information according to the device information, and executes the execution according to the first interpretation information The at least one test command is used to operate the application. The automatic test system of claim 3, wherein the verification module selects the first interpretation information matching the device information from the interpretation information according to the device information, and according to the device information The first interpretation information executes the at least one verification command to select the first verification information from the plurality of verification information. The automatic testing system according to claim 1, wherein the execution module pre-stores a plurality of external test scripts, wherein the at least one test instruction instructs the execution module to call a first external test script among the external test scripts , and operate the application program of the device under test according to the first external test script. The automatic test system of claim 1, wherein the transceiver is communicatively connected to a second device under test, wherein the execution module executes the at least one test command to operate the second device under test according to the interpretation information test the second application of the device, and obtain the second application corresponding to the second A second set of parameters for the application, wherein the verification module determines whether the second set of parameters matches the first verification information to generate a second test report, wherein the output module outputs all the data through the transceiver. the second test report. A method for testing an application program of a device under test, suitable for an automatic test system including a cloud server and a test device, wherein the method comprises: obtaining a test script from the test device, wherein the test script includes at least one A test command and at least one verification command; the test device obtains interpretation information from the cloud server; the test device executes the at least one test command according to the interpretation information to operate the device under test the application program, and obtain a first parameter set of the application program; a plurality of verification information are pre-stored by the test device, and the at least one verification command is executed according to the interpretation information to select from the plurality of verification information generating first verification information, and judging whether the first parameter set matches the first verification information to generate a test report; and outputting the test report by the testing device.

Images