CN111176972A - Test method and system - Google Patents
Test method and system Download PDFInfo
- Publication number
- CN111176972A CN111176972A CN201811340891.XA CN201811340891A CN111176972A CN 111176972 A CN111176972 A CN 111176972A CN 201811340891 A CN201811340891 A CN 201811340891A CN 111176972 A CN111176972 A CN 111176972A
- Authority
- CN
- China
- Prior art keywords
- test
- target program
- testing
- case
- cases
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000010998 test method Methods 0.000 title claims description 12
- 238000012360 testing method Methods 0.000 claims abstract description 440
- 230000002159 abnormal effect Effects 0.000 claims abstract description 35
- 238000000034 method Methods 0.000 claims abstract description 25
- 230000010076 replication Effects 0.000 claims description 18
- 230000005856 abnormality Effects 0.000 claims description 10
- 238000004088 simulation Methods 0.000 claims description 3
- 238000010586 diagram Methods 0.000 description 16
- 238000013112 stability test Methods 0.000 description 13
- 230000006870 function Effects 0.000 description 12
- 230000008569 process Effects 0.000 description 7
- 238000012430 stability testing Methods 0.000 description 6
- 238000004590 computer program Methods 0.000 description 4
- 230000003287 optical effect Effects 0.000 description 4
- 230000002085 persistent effect Effects 0.000 description 4
- 238000013461 design Methods 0.000 description 3
- 238000011056 performance test Methods 0.000 description 3
- 230000008030 elimination Effects 0.000 description 2
- 238000003379 elimination reaction Methods 0.000 description 2
- 230000004044 response Effects 0.000 description 2
- 230000006978 adaptation Effects 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 239000002355 dual-layer Substances 0.000 description 1
- 238000011990 functional testing Methods 0.000 description 1
- 238000011835 investigation Methods 0.000 description 1
- 230000007774 longterm Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000002441 reversible effect Effects 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/36—Preventing errors by testing or debugging software
- G06F11/3668—Software testing
- G06F11/3672—Test management
- G06F11/3688—Test management for test execution, e.g. scheduling of test suites
Abstract
The invention provides a new testing method and a system, wherein the method comprises the following steps: testing the target program by using the test case; storing a test case when the target program is abnormal; and performing exception reproduction on the target program based on the saved test case. The test scheme is particularly suitable for testing the stability of the black box, and the debugging efficiency of the target program is greatly improved by introducing the test case to participate in abnormal reproduction and debugging.
Description
Technical Field
The invention relates to the field of testing, in particular to a testing method and a testing system.
Background
With the popularization of terminal devices, especially smart phones, people increasingly use various types of applications (e.g., APP) installed on the terminal devices for learning, entertainment, and information acquisition. People use a variety of terminals and applications in a variety of environments, for example, various cell phone brands, models, operating systems, and perform regular, long-term, or non-compliant operations on applications under different geographical locations, operators, and network conditions.
In order to improve the adaptability of the application program to various conditions, the application program needs to be subjected to all-around tests, such as black box stability tests. Existing black box stability tests typically only save dump files when an exception occurs in the program under test. It may be very difficult to investigate the cause of an anomaly only through the dump file.
Therefore, a testing method and system capable of more conveniently and accurately locating the program operation exception are needed.
Disclosure of Invention
In order to solve at least one of the above problems, the present invention provides a new test scheme, which can store a test case when an application under test is abnormal, so as to facilitate the subsequent abnormal reproduction and debugging based on the above case.
According to an aspect of the present invention, a testing method, in particular a black box stability testing method, is provided, comprising: testing the target program by using the test case; storing a test case when the target program is abnormal; and performing exception reproduction on the target program based on the saved test case. By introducing the test cases to participate in abnormal reproduction and debugging, the debugging efficiency of the target program is greatly improved.
Preferably, the test method may further comprise: generating a plurality of test cases for the target program, and testing the target program using the test cases comprises: the plurality of test cases or portions thereof are used to generate simulated user events according to a test strategy. Therefore, the test efficiency is improved through batch test.
Testing the target program using the test case may include: connecting a test device; installing the target program in the test equipment; and traversing the plurality of test cases or a part thereof in the test equipment to test the target program. Further, the plurality of test cases or portions thereof may be automatically traversed using monkeyunner to test the target program. Therefore, the method is suitable for program testing aiming at various application scenes.
The test method of the present invention may further comprise: saving a dump (dump) file when an exception occurs in a target program, and performing exception reproduction on the target program based on a saved test case comprises: and reproducing the exception of the target program based on the saved test case and the dump file. Preferably, the dump file may be saved in a manner that can correspond to the test case when the exception occurs. Therefore, more relevant information can be provided for reproduction and debugging.
The test method of the present invention may further comprise: ending the test, and performing exception reproduction on the target program based on the saved test case comprises: starting an abnormal recurrence test; loading the stored test case; and attempting to reproduce the exception of the target program using the saved test case. Therefore, the overall efficiency of testing and debugging can be improved by executing the testing and the recurrence testing in different tasks.
The test method of the present invention may further comprise: carrying out user-defined operation before the test is started; and/or performing user-defined operation after the test is finished. Thereby further improving the adaptation range and flexibility of the test scheme of the invention.
According to another aspect of the present invention, there is provided a test system comprising: a test module having a test interface that is invoked to perform a test, the test module further comprising: a test engine to: loading a test case and testing the target program by using the test case; and storing a test case when the target program is abnormal, wherein the stored test case is used for performing abnormal reproduction on the target program. In this way, test and debug efficiency is improved by a dedicated test framework, particularly a test framework dedicated to black box stability testing.
The test system of the present invention may further include: a replication module having a replication interface that is called to obtain the saved test case for replication, the replication module further comprising: a recurring test engine to: and performing exception reproduction on the target program based on the saved test case.
Alternatively, the test module of the present invention further comprises: a reproduction interface called to acquire the saved test case for reproduction; and a recurring test engine to: and performing exception reproduction on the target program based on the saved test case.
The test module may further include: and the case generating module is used for generating a plurality of test cases aiming at the target program and sending the test cases or a part of the test cases to the test engine.
The test engine may be configured to: connecting a test device; installing the target program in the test equipment; and traversing the plurality of test cases or a part thereof in the test equipment to test the target program. Preferably, the test engine may automatically traverse the plurality of test cases or a portion thereof using monkeyunner to test the target program.
The test system of the present invention may further include: and the saving module is used for saving a dump (dump) file when the target program is abnormal, and the dump file and the saved test cases are used for reproducing the abnormality of the target program. Preferably, the saving module can save the dump file in a manner corresponding to the test case when the exception occurs.
The test system of the present invention may further include: a custom module to: carrying out user-defined operation before the test is started; and/or performing user-defined operation after the test is finished.
The test system of the present invention may further include: and the strategy module is used for providing a test strategy for the test engine based on default and/or custom setting, and the test engine traverses the test case to generate a simulation user event according to the test strategy.
According to yet another aspect of the invention, a computing device is presented, comprising: a processor; and a memory having executable code stored thereon, which when executed by the processor, causes the processor to perform the above-described test method.
According to yet another aspect of the invention, a non-transitory machine-readable storage medium is proposed, having stored thereon executable code, which when executed by a processor of an electronic device, causes the processor to perform the above-mentioned testing method.
The testing method and system of the present invention may be implemented as a testing framework, for example, a black box stability testing framework. The test framework can be used for reproduction by saving the test cases when the exception occurs (for example, crash), so that the debugging efficiency for the target program is greatly improved. The test aiming at the test case and the debugging aiming at the abnormal occurrence case can be automatically completed in batch in different tasks. Therefore, the test efficiency is improved, and meanwhile, the abnormity reproduction and the target program debugging are facilitated.
Drawings
The above and other objects, features and advantages of the present disclosure will become more apparent by describing in greater detail exemplary embodiments thereof with reference to the attached drawings, in which like reference numerals generally represent like parts throughout.
FIG. 1 shows a flow diagram of a testing method according to one embodiment of the invention.
Fig. 2 shows a flow chart of the testing steps according to a preferred embodiment of the invention.
Fig. 3 shows a flow chart of recurring test steps according to a preferred embodiment of the present invention.
FIG. 4 shows a block diagram of a test system according to one embodiment of the invention.
FIG. 5 shows a block diagram of a test system according to another embodiment of the invention.
FIG. 6 shows a block diagram of a test system according to yet another embodiment of the invention.
FIG. 7 shows a schematic diagram of a black box stability test framework according to one embodiment of the invention.
FIG. 8 illustrates a schematic structural diagram of a computing device in accordance with an embodiment of the present invention.
Detailed Description
Preferred embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. While the preferred embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be embodied in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.
As described above, at the present stage, an application (e.g., APP) needs to be adapted to a wide variety of usage environments. In particular, applications need to remain functional under the correct or incorrect operation of various terminal (e.g., handset) brands, models, operating systems, geographic locations, operators, network conditions, and users. For example, an APP issuer needs to control the ratio of start-up crash (flash back), device crash (device black screen or restart), and application unresponsiveness (ANR) of the APP in various environments to an acceptable range. Thus, there is a need for comprehensive testing of applications, particularly black box stability testing.
The existing black box stability test only saves some dump files for the abnormality of the application program, such as program crash, and only the dump files are difficult to restore the current test scenario, so that the investigation of the crash cause is very difficult, especially under the condition of rich test scenarios (for example, under the condition of a large number of test cases). Therefore, the invention provides a testing framework which can be realized as a testing scheme and a system, and can save the test cases when the program runs abnormally, so as to facilitate the subsequent reproduction of the abnormality and the positioning of the cause of the abnormality.
To illustrate the technical solution of the present invention more clearly, a brief introduction will be made to the relevant background knowledge of the test.
Before or even during the release of an application to a user, it needs to be tested to ensure the correct functioning of the application. Tests for applications typically include white box tests and black box tests. The white box test is carried out by the source code of the program, and the defects or errors of the internal code in the algorithm, overflow, path, condition and the like are searched from the code, so that the internal code is corrected. Black box testing, which may also be referred to as functional testing, data driven testing, or specification based testing, is strictly tested by using the entire software or some software functionality without checking the source code of the program or knowing clearly the specific design of the source code program. The tester can know whether the software can work normally under various test conditions (test cases) by inputting data and checking output results.
The white box test is an exhaustive path test and the black box test is an exhaustive input test. The black box test focuses on the functional requirements of test software, focuses on the external structure of a program, mainly tests a software interface and software functions, and is mostly applied to the later stage of the test process. The method is an actual test which is carried out along with a system by designing a document and simulating a customer scene according to software requirements. The testing technology covers the aspect of testing and is mainly used for discovering the following types of errors: whether a functional error or omission occurs; whether correct input and output can be performed on the interface; whether there is a data structure error or an external database access error; whether the performance can meet the requirements; whether there is an initialization or termination error, etc. The black box test is a functional test from the user perspective, and is particularly suitable for testing the performance of the application program under various conditions.
Performance is a non-functional characteristic of an application, and not whether an application has completed a particular function, but the timeliness that an application exhibits when performing a particular function is a concern. The performance test is a test for testing whether the application program can complete a specific function in time under different conditions. Timeliness represents different indicators from different perspectives, e.g., response time for a user; aiming at the resource utilization rate, expandability, system stability and system capacity of a system administrator; and system architecture, database design, design and code implementation for developers, etc. The system stability is significant to a system administrator, and the quality of the stability can directly affect the response time concerned by the end user, so that the stability test is very important in the performance test. The stability test, also called reliability test, is to load a certain service pressure to the system, so that the system is continuously operated for a period of time (for example, 12 hours), and detect whether the system can operate stably.
In order to perform the stability test, the test cases are generally designed and operated according to different conditions. A test case is a set of test inputs, execution conditions, and expected results tailored for a particular purpose to test a certain program path or verify that a certain requirement is met. Due to the many situations in which testing is actually required, not only are various application scenarios tested, but also those inputs that are not legitimate but possible are tested, and stability testing often involves long testing times.
In the existing test scenario, especially the black box stability test, when an exception occurs (for example, a crash occurs) in the application under test, only a dump file is often saved. The Dump file is a memory image of the process. It is difficult to investigate the cause of a crash based on only the process information when the crash occurred. The existence of a large number of test cases or long-time test further improves the difficulty of abnormal reproduction and elimination.
Therefore, the invention provides a test scheme, in particular to a black box stability test framework which can be realized as a test method and a test system, and the framework can store the test cases causing the abnormity when the tested application program is abnormal, thereby being convenient for reappearing and eliminating the abnormity.
FIG. 1 shows a flow diagram of a testing method according to one embodiment of the invention. The test method shown in fig. 1 is suitable for black box tests and performance tests with a large number of test cases, and is particularly suitable for black box stability tests.
In step S110, the target program is tested using the test case. In order to fully test a target program, multiple test cases are typically used to perform the test. To this end, a plurality of test cases for the target program may be generated, and a test may be performed using the generated plurality of test cases (or a part thereof). In particular, the plurality of test cases, or portions thereof, may be used to generate simulated user events according to a test strategy. The test policies may be default policies, such as conventional click events and drag events, or may include user-defined policies. After the test strategy is set, a plurality of test cases can be automatically generated, and the plurality of test cases can be automatically used for testing, for example, each test case is traversed. Depending on the actual application, for example, in the case where the target program includes a plurality of functions, a test may be performed for a part (for example, a partial function) of the target program. For example, a batch of test cases (e.g., a) is used to test for function a. Subsequently, function B is tested using the same or different test cases (e.g., B).
In the process of using the test case for testing, if an abnormality is detected, for example, an application program crashes, the test case causing the abnormality can be automatically saved. Then, in step S120, the test case when the target program is abnormal (e.g., crashed) is saved, for example, the test case causing the abnormality may be written into the memory and then stored to a specific location or folder.
Subsequently, if necessary, in step S130, the target program may be subjected to exception replication based on the saved test case (e.g., loading the stored test case into the memory).
The testing method is particularly suitable for testing application programs (such as mobile phone APP) which can be installed on various terminal devices (such as smart phones of various brands and models). To this end, the step S110 of testing the target program using the test case may include: connecting a test device; installing the target program in the test equipment; and traversing the plurality of test cases or a part thereof in the test equipment to test the target program. Due to the fact that terminal manufacturers supporting Android (Android) systems are numerous and the open source of the systems, testing of Android APPs generally needs to be performed on various types of testing equipment. The test equipment connected in the test can be, for example, physical mobile phones of various brands or models, and can also be various virtual equipment. The Android operating system is installed in the test equipment, and the target program to be tested is installed in the operating system.
Subsequently, for the test device in which the target program is installed, a traversal test of multiple test cases for the target program (or a part of its functions) may be performed. The current test case can be stored when the target program is abnormal, and then the subsequent test case test is automatically carried out until the test case test of the current batch is completed. Further, after the test case batch for the current test device is executed, the next test device is automatically connected and the target program is installed, and then the test cases of the next batch (which may be the same as or different from the previous batch) are automatically tested. Automatic testing for multiple test cases or portions thereof, and even for multiple test devices, may be implemented using monkeyyrner.
As described above, the testing method of the present invention can be used for testing android APP. In other embodiments, the application used for testing may also be an APP for the iOS or other operating system.
In one embodiment, the testing method of the present invention may further comprise saving a dump (dump) file when the target program is abnormal. Thus, in step S130, the exception generated by the target program can be reproduced based on both the saved test case and the dump file. Preferably, in order to facilitate the use of both the test cases and the dump files, the dump files may be saved in a manner that can correspond to the test cases when the abnormality occurs. In one embodiment, the generated dump files may be arranged in time sequence by the tested object program, or timestamps may be added to the dump files, so as to find the test cases according to the dump files. Correspondingly, under the condition that the test case runs for a long time, the specific running condition aiming at the test case when the target program is abnormal can be saved, so that more clues are provided for subsequent abnormal recurrence and elimination.
Although the test case may be reproduced immediately after the test case in which the abnormality occurs is saved, in some embodiments, the test of the test case and the saving of the test case in the abnormal state, and the abnormal reproduction of the saved test case may be performed by different functional modules, or performed by the same functional module in different tasks. Preferably, after the execution of the single batch or multiple batches of test cases of the target program is completed, the abnormal reproduction and debugging of the saved test cases can be intensively performed. To this end, in one embodiment, the testing method of the present invention may further include ending the test, and step S130 may include: starting an abnormal recurrence test; loading the stored test case; and attempting to reproduce the exception of the target program using the saved test case. Subsequently, the execution subject of the test and reproduction task will be described in detail with reference to fig. 4.
In a preferred embodiment, the user can also customize the operations before and after the test begins, so as to improve the automation level of the test scheme of the present invention.
As described above, in the test scheme of the present invention, the test of the test case and the test case saving in the abnormal state can be performed in different tasks, and the abnormal reproduction can be performed with respect to the saved test case. Fig. 2 shows a flow chart of the testing steps according to a preferred embodiment of the invention. Fig. 3 shows a flow chart of recurring test steps according to a preferred embodiment of the present invention.
In the test case execution task, as shown in fig. 2, performing input analysis, and waiting for a device (for example, a test device installed with an android operating system) to be successfully connected; subsequently installing an APP (e.g., an android APP); generating a random test case; traversing the test cases, and generating various simulated user events according to corresponding test strategies; if the crash happens, the test case is saved; and killing the APP process after the test is finished, and repeating the process to know the test times set by the user.
In the abnormal reproduction task, as shown in fig. 3, performing input analysis and waiting for the successful connection of the equipment; subsequently installing the APP; loading the saved test case (e.g., into memory); traversing the test cases, and generating various simulated user events according to corresponding test strategies; and (5) trying to perform abnormal reproduction and debugging, and killing the APP process after the test is completed.
The test scheme of the present invention may be implemented, inter alia, as a test system, e.g., a black box stability test framework.
FIG. 4 shows a block diagram of a test system according to one embodiment of the invention. As shown, the system 400 may include a test module 410, and the test module 410 may further include a test engine 411. The test module 410 has a test interface that is called to perform a test, and the test engine 411 is operable to: loading a test case and testing the target program by using the test case; and storing the test case when the target program is abnormal. The saved test cases may be used to perform exception replication on the target program. The above-described reproduction may be performed by the test engine 411 in a subsequent reproduction task, or may be performed by a reproduction engine included in the test module as shown in fig. 5 and 6 below, or by a reproduction engine included in a separate reproduction module.
FIG. 5 shows a block diagram of a test system according to another embodiment of the invention. In addition to the test module 510 corresponding to that shown in FIG. 4 and the test engine 511 included therein, the system 500 also includes a separate replication module 520. The replication module 520 has a replication interface that is called to retrieve the saved test cases for replication and includes a replication test engine 521 for: and performing exception reproduction on the target program based on the saved test case.
FIG. 6 shows a block diagram of a test system according to yet another embodiment of the invention. In addition to the test module 610 and its included test engine 611 corresponding to those shown in fig. 4 and 5, the system 600 also includes a recurring test engine 612 disposed within the test module 610. Further, the test module 610 has a replication interface that is called to retrieve the saved test cases for replication, and the replication test engine 612 may be configured to: and performing exception reproduction on the target program based on the saved test case.
In a preferred embodiment, the test module (e.g., the test module 410, 510 or 610) further includes a use case generation module (not shown) for generating a plurality of test cases for the target program and distributing the plurality of test cases or a part thereof to the test engine. The use case generation module may be included within a test engine.
Further, a test engine (e.g., test engine 411, 511, or 611) is to: connecting a test device; installing the target program in the test equipment; and traversing the plurality of test cases or a part thereof in the test equipment to test the target program. The test engine may automatically traverse the plurality of test cases or a portion thereof using monkeyunner to test the target program.
Preferably, the test system (or the test module included in the test system) may further include a saving module (not shown in the figure) for saving a dump (dump) file when the exception occurs in the target program, where the dump file is used together with the saved test case to reproduce the exception occurring in the target program. Similarly, the saving module saves the dump file in a manner that can correspond to the test case when the exception occurs.
Preferably, the test system may further comprise a custom module for: carrying out user-defined operation before the test is started; and/or performing user-defined operation after the test is finished.
The test system may also include a policy module to provide test policies to the test engine based on default and/or custom settings, and the test engine may traverse test cases to generate simulated user events according to the test policies.
FIG. 7 shows a schematic diagram of a black box stability test framework according to one embodiment of the invention. As shown, the main body of the frame 7 is composed of a test portion and a reproduction portion. The Test part includes a Test Shell (Test Shell) including a Test Engine (Test Engine). The reproduction section includes a reproduction Shell (reproduction Shell) including a reproduction Test Engine (reproduction Test Engine).
The test shell is a shell of the frame and an external interface, and the stability test can be carried out by calling the test shell from the outside.
The test engine is used to execute specific test logic including, but not limited to: generating a random test case; storing the test case; simulation of click events, drag events, back button clicks, etc.; installing an app; uninstalling the app; starting the app; kill progression, etc.
The recurrence shell can be called from the outside to transmit test cases so as to facilitate the recurrence of problems.
The recurrence test engine is a test engine for recurrence of problems, and is used for analyzing the stored test cases and running the test engine by using the transmitted test cases to help recurrence of exception and program debugging.
Further, the framework may further include a test Policy module (e.g., Policy 1, Policy 2, and Policy 3 in the figure) for generating policies of various click events, drag events, etc., and the user may also customize the policies.
Further, the framework may also include a custom module, such as the Hook module (Hook) shown in the figure, which the user can use to customize the associated logic before and after testing, or to default to not implement.
In practical applications, a 2-hour running test for mobile APP (XX browser) can be performed by the following commands:
./auto_test_shell.py android-free-debug.apk com.XXMobilecom.xx.browser.InnerUCMobile 2
if a crash occurs, the problem can be replicated using the test case saved when the crash occurred:
./auto_test_reproduce.py android-free-debug.apk com.XXMobilecom.xx.browser.InnerXXMobile./test_case/2018_08_27_17_31_47.log
the test system of the present invention and its preferred embodiments have been described above in connection with fig. 4-7. It should be understood that the test system of the present invention can be used to implement the test method of the present invention and the preferred embodiments thereof described above, and will not be described herein again.
Fig. 8 is a schematic structural diagram of a computing device that can be used to implement the above-described testing method according to an embodiment of the present invention.
Referring to fig. 8, computing device 800 includes memory 810 and processor 820.
The processor 820 may be a multi-core processor or may include multiple processors. In some embodiments, processor 820 may include a general-purpose host processor and one or more special coprocessors such as a Graphics Processor (GPU), a Digital Signal Processor (DSP), or the like. In some embodiments, processor 820 may be implemented using custom circuitry, such as an Application Specific Integrated Circuit (ASIC) or a Field Programmable Gate Array (FPGA).
The memory 810 may include various types of storage units, such as system memory, Read Only Memory (ROM), and permanent storage. Wherein the ROM may store static data or instructions for the processor 820 or other modules of the computer. The persistent storage device may be a read-write storage device. The persistent storage may be a non-volatile storage device that does not lose stored instructions and data even after the computer is powered off. In some embodiments, the persistent storage device employs a mass storage device (e.g., magnetic or optical disk, flash memory) as the persistent storage device. In other embodiments, the permanent storage may be a removable storage device (e.g., floppy disk, optical drive). The system memory may be a read-write memory device or a volatile read-write memory device, such as a dynamic random access memory. The system memory may store instructions and data that some or all of the processors require at runtime. In addition, the memory 810 may include any combination of computer-readable storage media, including various types of semiconductor memory chips (DRAM, SRAM, SDRAM, flash memory, programmable read-only memory), magnetic and/or optical disks, may also be employed. In some embodiments, memory 810 may include a removable storage device that is readable and/or writable, such as a Compact Disc (CD), a read-only digital versatile disc (e.g., DVD-ROM, dual layer DVD-ROM), a read-only Blu-ray disc, an ultra-density optical disc, a flash memory card (e.g., SD card, min SD card, Micro-SD card, etc.), a magnetic floppy disc, or the like. Computer-readable storage media do not contain carrier waves or transitory electronic signals transmitted by wireless or wired means.
The memory 810 has stored thereon executable code that, when processed by the processor 820, may cause the processor 820 to perform the testing methods described above.
The test method and system according to the invention have been described in detail above with reference to the accompanying drawings. The testing method and system of the present invention may be implemented as a testing framework, for example, a black box stability testing framework. The test framework can be used for reproduction by saving the test cases when the exception occurs (for example, crash), so that the debugging efficiency for the target program is greatly improved. The test aiming at the test case and the debugging aiming at the abnormal occurrence case can be automatically completed in batch in different tasks. Therefore, the test efficiency is improved, and meanwhile, the abnormity reproduction and the target program debugging are facilitated.
Furthermore, the method according to the invention may also be implemented as a computer program or computer program product comprising computer program code instructions for carrying out the above-mentioned steps defined in the above-mentioned method of the invention.
Alternatively, the invention may also be embodied as a non-transitory machine-readable storage medium (or computer-readable storage medium, or machine-readable storage medium) having stored thereon executable code (or a computer program, or computer instruction code) which, when executed by a processor of an electronic device (or computing device, server, etc.), causes the processor to perform the steps of the above-described method according to the invention.
Those of skill would further appreciate that the various illustrative logical blocks, modules, circuits, and algorithm steps described in connection with the disclosure herein may be implemented as electronic hardware, computer software, or combinations of both.
The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems and methods according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
Having described embodiments of the present invention, the foregoing description is intended to be exemplary, not exhaustive, and not limited to the embodiments disclosed. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen in order to best explain the principles of the embodiments, the practical application, or improvements made to the technology in the marketplace, or to enable others of ordinary skill in the art to understand the embodiments disclosed herein.
Claims (22)
1. A method of testing, comprising:
testing the target program by using the test case;
storing a test case when the target program is abnormal; and
and performing exception reproduction on the target program based on the saved test case.
2. The method of claim 1, further comprising:
generating a plurality of test cases for the target program, and
the testing the target program by using the test case comprises the following steps:
the plurality of test cases or portions thereof are used to generate simulated user events according to a test strategy.
3. The method of claim 2, wherein testing the target program using the test cases comprises:
connecting a test device;
installing the target program in the test equipment; and
traversing the plurality of test cases or a portion thereof in the test device to test the target program.
4. The method of claim 2, wherein testing the target program using the test cases comprises:
and automatically traversing the plurality of test cases or the part of the test cases by using the Monkeyunner to test the target program.
5. The method of claim 1, further comprising:
saves a dump (dump) file when an exception occurs in the target program, and
the abnormal reproduction of the target program based on the saved test case comprises the following steps:
and reproducing the exception of the target program based on the saved test case and the dump file.
6. The method of claim 5, wherein saving the dump file when the exception occurs in the target program comprises:
and saving the dump file in a mode corresponding to the test case when the exception occurs.
7. The method of claim 1, further comprising:
the test is ended, and
the abnormal reproduction of the target program based on the saved test case comprises the following steps:
starting an abnormal recurrence test;
loading the stored test case; and
attempting to replicate the exception of the target program using the saved test case.
8. The method of claim 7, further comprising:
carrying out user-defined operation before the test is started; and/or
And carrying out user-defined operation after the test is finished.
9. The method of claim 1, wherein the test is at least one of:
testing a black box; and
and (5) testing the stability.
10. A test system, comprising:
a test module having a test interface that is invoked to perform a test, the test module further comprising:
a test engine to:
loading a test case and testing the target program by using the test case;
and storing a test case when the target program is abnormal, wherein the stored test case is used for performing abnormal reproduction on the target program.
11. The system of claim 10, further comprising:
a replication module having a replication interface that is called to obtain the saved test case for replication, the replication module further comprising:
a recurring test engine to:
and performing exception reproduction on the target program based on the saved test case.
12. The system of claim 10, wherein the test module further comprises:
a reproduction interface called to acquire the saved test case for reproduction; and
a recurring test engine to:
and performing exception reproduction on the target program based on the saved test case.
13. The system of claim 10, the test module further comprising:
and the case generating module is used for generating a plurality of test cases aiming at the target program and sending the test cases or a part of the test cases to the test engine.
14. The system of claim 13, wherein the test engine is to:
connecting a test device;
installing the target program in the test equipment; and
traversing the plurality of test cases or a portion thereof in the test device to test the target program.
15. The system of claim 14, wherein the test engine automatically traverses the plurality of test cases or a portion thereof using monkeyunner to test the target program.
16. The system of claim 10, further comprising:
and the saving module is used for saving a dump (dump) file when the target program is abnormal, and the dump file and the saved test cases are used for reproducing the abnormality of the target program.
17. The system of claim 16, wherein the save module saves the dump file in a manner that can correspond to a test case at the time of the exception.
18. The system of claim 17, further comprising:
a custom module to:
carrying out user-defined operation before the test is started; and/or
And carrying out user-defined operation after the test is finished.
19. The system of claim 10, further comprising:
a policy module for providing a test policy to the test engine based on default and/or custom settings, and
and the test engine traverses the test cases to generate a simulation user event according to the test strategy.
20. The system of claim 10, wherein the test performed by the test system is at least one of:
testing a black box; and
and (5) testing the stability.
21. A computing device, comprising:
a processor; and
a memory having executable code stored thereon, which when executed by the processor, causes the processor to perform the method of any one of claims 1-9.
22. A non-transitory machine-readable storage medium having stored thereon executable code, which when executed by a processor of an electronic device, causes the processor to perform the method of any one of claims 1-9.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811340891.XA CN111176972A (en) | 2018-11-12 | 2018-11-12 | Test method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811340891.XA CN111176972A (en) | 2018-11-12 | 2018-11-12 | Test method and system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111176972A true CN111176972A (en) | 2020-05-19 |
Family
ID=70653592
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811340891.XA Pending CN111176972A (en) | 2018-11-12 | 2018-11-12 | Test method and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111176972A (en) |
Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101165658A (en) * | 2006-10-18 | 2008-04-23 | 国际商业机器公司 | Method and system for automatically generating unit test case of reproduced operation problem |
| CN101295280A (en) * | 2008-06-18 | 2008-10-29 | 腾讯科技（深圳）有限公司 | Automatic software debugging method and system |
| CN103139010A (en) * | 2012-11-13 | 2013-06-05 | 深圳中兴网信科技有限公司 | Terminal and testing server and testing method |
| CN103249069A (en) * | 2012-02-14 | 2013-08-14 | 展讯通信（上海）有限公司 | Method, equipment and system for testing mobile terminal |
| CN103778056A (en) * | 2012-10-23 | 2014-05-07 | 腾讯科技（深圳）有限公司 | Wireless application stability testing method and system |
| CN105487966A (en) * | 2014-09-17 | 2016-04-13 | 腾讯科技（深圳）有限公司 | Program testing method, device and system |
| CN106407102A (en) * | 2015-08-03 | 2017-02-15 | 腾讯科技（深圳）有限公司 | A fuzz testing method, device and system for application programs |
| CN106502895A (en) * | 2016-10-21 | 2017-03-15 | 郑州云海信息技术有限公司 | A kind of automatic test information generation device and method |
| CN107305529A (en) * | 2016-04-25 | 2017-10-31 | 阿里巴巴集团控股有限公司 | The method and apparatus for detecting target software |
| CN107967214A (en) * | 2017-08-15 | 2018-04-27 | 北京摩拜科技有限公司 | The test method and test equipment of application program |
-
2018
- 2018-11-12 CN CN201811340891.XA patent/CN111176972A/en active Pending
Patent Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101165658A (en) * | 2006-10-18 | 2008-04-23 | 国际商业机器公司 | Method and system for automatically generating unit test case of reproduced operation problem |
| CN101295280A (en) * | 2008-06-18 | 2008-10-29 | 腾讯科技（深圳）有限公司 | Automatic software debugging method and system |
| CN103249069A (en) * | 2012-02-14 | 2013-08-14 | 展讯通信（上海）有限公司 | Method, equipment and system for testing mobile terminal |
| CN103778056A (en) * | 2012-10-23 | 2014-05-07 | 腾讯科技（深圳）有限公司 | Wireless application stability testing method and system |
| CN103139010A (en) * | 2012-11-13 | 2013-06-05 | 深圳中兴网信科技有限公司 | Terminal and testing server and testing method |
| CN105487966A (en) * | 2014-09-17 | 2016-04-13 | 腾讯科技（深圳）有限公司 | Program testing method, device and system |
| CN106407102A (en) * | 2015-08-03 | 2017-02-15 | 腾讯科技（深圳）有限公司 | A fuzz testing method, device and system for application programs |
| CN107305529A (en) * | 2016-04-25 | 2017-10-31 | 阿里巴巴集团控股有限公司 | The method and apparatus for detecting target software |
| CN106502895A (en) * | 2016-10-21 | 2017-03-15 | 郑州云海信息技术有限公司 | A kind of automatic test information generation device and method |
| CN107967214A (en) * | 2017-08-15 | 2018-04-27 | 北京摩拜科技有限公司 | The test method and test equipment of application program |
Non-Patent Citations (1)
| Title |
|---|
| 路晓丽: "《Web应用软件的测试技术研究》", vol. 1, 31 March 2007, 陕西人民出版社, pages: 128 - 133 * |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106951345B (en) | Consistency test method and device for disk data of virtual machine | |
| CN110083543B (en) | Regression testing method, device, electronic equipment and storage medium | |
| US20070220370A1 (en) | Mechanism to generate functional test cases for service oriented architecture (SOA) applications from errors encountered in development and runtime | |
| CN109885480B (en) | Automatic interface compatibility testing method and device based on debugging bridge | |
| CN110879781B (en) | Program debugging method, device, electronic equipment and computer readable storage medium | |
| CN111767226B (en) | Cloud computing platform resource testing method, system and equipment | |
| CN115080398A (en) | Automatic interface test system and method | |
| CN109683997B (en) | Method for accessing application program interface through sandbox, sandbox and sandbox equipment | |
| US8997048B1 (en) | Method and apparatus for profiling a virtual machine | |
| CN109739756B (en) | Method and equipment for testing application of mobile terminal | |
| CN115422091B (en) | Firmware debugging method and device, electronic equipment and storage medium | |
| CN111427777A (en) | SO L test method, system, equipment and medium | |
| CN111176972A (en) | Test method and system | |
| CN115454856A (en) | Multi-application security detection method, device, medium and electronic equipment | |
| CN115470141A (en) | Fault simulation method, device and related equipment | |
| CN113986263A (en) | Code automation test method, device, electronic equipment and storage medium | |
| CN111258878B (en) | Application testing method, device, equipment and storage medium | |
| US10922249B2 (en) | Input/output control code filter | |
| CN114036042A (en) | Model testing method, device, computer and readable storage medium | |
| US11714744B2 (en) | System and method for diagnosing a computing device in safe mode | |
| CN109032955B (en) | UI automation test method, device, equipment and readable storage medium | |
| CN112560035B (en) | Application detection method, device, equipment and storage medium | |
| CN115982018B (en) | UI test method, system, computer device and storage medium based on OCR | |
| US8898636B1 (en) | Method and apparatus for testing an application running in a virtual machine | |
| CN114265782B (en) | Application program based shell adding tool testing method, storage medium and electronic equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |