TWI493336B - Application of new case feedback in automated software verification system and its method - Google Patents

Description

Application of new case feedback system and method for automated software verification

The invention relates to a system and a method for applying the new case feedback to the automation software verification, and is applied to a complex data computing system, such as a telecom account system, and the monthly data is calculated to generate a bill, so the test is mainly The digital data is computed to see if the results are correct; for this type of system, when the tester performs regression testing or parallel testing of new and old systems, it is especially necessary to have a complete test case coverage to avoid side-effect.

In the past, when the automatic test was performed, the verification personnel would first prepare the verification case library, add all the cases to be tested to the verification case library, and then execute the automated test, the case of the verification case library will be executed one by one, so the test coverage That is to say, based on all cases of the verification case library, if the case of the verification case library is more diverse and richer, the representative test coverage is higher. The need to verify the case inventory in the case of automated testing has been seen in the previous Republic of China invention patent number I322350 "test devices, methods, applications and computer-readable recording media to select one from a test case database In the case of the “Qualified Test Case Combination”, the test case database here is the verification case database. The method of selecting the case must manually establish the test case database in advance, and then manually set the test rules and use the selection module. For the purpose of setting The test rule is selected, a suitable test case conforming to the test rule is selected, and a binary decision chart of one of the suitable test cases is established, and a qualified test case combination is selected according to the binary decision chart, including at least one suitable test case. Therefore, it is first to find a suitable test case from the prior database through manual test rules. One of the binary decision diagrams can save storage space, but the test case database must be manually set by the system engineer according to the new demand function or the exception exclusion experience after the impact assessment and traceability analysis, and there is no self-increasing function, so unless Manually add new cases to the test case database, otherwise the case of the test case database will remain the same. The speed of manually adding new cases is actually very slow and not necessarily diverse and effective. It may be repeated with new cases. It may also be inexperienced. The proposed cases cannot cover the most difficult exceptions, resulting in incomplete testing coverage. There is a lack of a way to automatically extend existing cases and self-discover and build new state cases to test case databases.

It can be seen that there are still many shortcomings in the above-mentioned methods of use, which is not a good design, but needs to be improved.

In view of the shortcomings derived from the above-mentioned conventional methods, the inventor of the present invention has improved and innovated, and after years of painstaking research, he finally succeeded in researching and developing the system and its method of returning to the automated software verification.

The object of the present invention is to provide a system and method for applying the new case feedback to the automated software verification. When the system performs the regression test or the parallel test, having the complete test case coverage is very important for avoiding the side-effect, and High test coverage is also a test for designers The goal is also a measure to ensure the correct function of the product. However, the high test coverage also represents the time and material cost. Therefore, the present invention provides a method for quickly and effectively establishing a verification case library and allowing the case to self-feed back, which not only saves time and labor, but also effectively improves the test for the verifier. Coverage. A second object of the present invention is to provide a remote monitoring system and method for a network television, which is capable of monitoring the current situation of a child watching a program in a home through a human-machine interface on a multi-flash terminal device. In addition, the method of the present invention transmits commands through a remote network management mechanism, and can transmit text information in addition to the general remote control button command transmission, so that an emergency can be made to the home network television system during the monitoring process. Processing (such as turntable or shutdown) and displaying text alert notifications on the TV screen.

A new case for the purpose of the above-mentioned invention is to give back to the automated software verification system and method thereof, using the attribute self-learning module and the existing case of the verification case library, and through the automatic verification module, the case intelligent screening module and the case Reciprocal interaction between modules, find out the case from the existing test collection system different from the current verification case library, let the case automatically extend and self-explore to generate a new state case, and automatically save the case information back to the verification case A method of the library.

A new application case is fed back to the system of automated software verification. The system consists of: a system to be tested, which aggregates all potential cases to be tested; an attribute self-learning module that generates attribute information and weights required for the verification case library. An automated verification module performs a comprehensive logic check for each of the to-be-tested assembly systems and removes a list of candidate cases that are not applicable; a case intelligent screening module that lists the candidate case list and the presence verification case library Cases are analyzed and calculated one by one, and the case where there is no verification case library is found, and the new case list is selected; a case feedback module, system parameters The attribute self-learning module determines that the new test attribute of the verification case library should be added at the end, and automatically finds the attribute of the new case list through the extensible markup language file and returns the case to the verification case library; and a verification case The library is used to store the verification case and receive the case generated by the case feedback module.

The to-be-tested aggregation system mainly collects detailed information of all the cases to be tested, and the detailed information such as user type, attribute, offer, or various transaction information, the verification case library needs at least one case.

The attribute self-learning module comprises: an attribute weight calculation module, which calculates each test attribute weight value input by the verification personnel; an attribute setting table records the test attribute and the level generated by the attribute weight calculation module. With weight values.

The component of the case intelligent screening module comprises: an intelligent analysis module, which is to perform attribute analysis of the candidate case; and an attribute association module, which is to find out whether the candidate case in the intelligent analysis module is present and the verification is present. The case library has the same candidate case for the test attribute; an operation module, the existing case and the candidate case found by the attribute association module are operated and compared, and the comparison result is a new case.

The attribute association module is configured to recursively by using a tag, and the intelligent analysis module analyzes the test attribute of the candidate case, and compares the attributes in the existing verification case library to confirm whether the test attribute has the same test attribute. If the candidate case is successfully found, the calculation is performed by the computing module. If the same case is not found at all, the case feedback module is processed. The composition of the case feedback module includes: an attribute extension module The new test attribute is found in the new case and the existing test case library; a case production module is added to the verification case library by the new attribute and case information found by the attribute extension module. In this case, the production module is in tagged language. The way to generate a swap file, such as an extensible markup language format.

A method of applying new cases to feedback on automated software verification, The steps are as follows: Step 1: The attribute self-learning module generates attribute information required for the verification case library; Step 2: The automatic verification module executes the to-be-tested aggregation system one by one, and finds cases that fail the criteria, and lists them as candidate cases. List; Step 3: Find the candidate case list through the case intelligent screening module to find the attribute of the candidate case, and compare it with the current case of the verification case library; Step 4: If the step 3 comparison result finds the same case Then, the candidate case and the current case of the verification case library are calculated through the operation module; Step 5: If the results of the step 4 operation are the same, the end does not perform any feedback, and if the result of the step 4 operation is inconsistent, the case will be passed. The feedback module finds the new attribute and generates the case exchange file and stores it back to the verification case library. Step 6: If the case of the third comparison result is inconsistent, the case feedback module is used to find the new attribute and save the case to the verification case. Library.

Apply the new case feedback as described in item 9 of the patent application scope. The method of the automated software verification method, wherein the step of the attribute self-learning module generating the attribute information comprises: Step 1: confirming whether the attribute setting table has at least one piece of the attribute information; Step 2: if it is confirmed that there is no data in the attribute setting table, Then, the attribute weight calculation module calculates the effective test attribute input by the verification personnel and the weight of the attribute by the hierarchy; Step 3: Record the weight, the test attribute and the level generated in the second step into the attribute setting table.

The invention provides a system for applying the new case to the automated software verification system and the method thereof, and has the following advantages when compared with the foregoing citation and other conventional technologies:

1. The system and method for effectively establishing a verification case library and allowing the case to self-reward, no need to add manually, no for the verification personnel It saves time and effort and automatically increases the diversity and richness of the case.

2. By classifying the weights of the test attributes, the more people use or change the attribute weights, the more the priority is applied in the case application, so that the case coverage first includes the most important test factors.

3. Through the self-excavation case method of the present invention, not only can be used in regression testing to find out possible side-effect cases; when encountering new and old system conversion, this method can be used to find out the old system case and carry out the case. Parallel testing.

1‧‧‧Attribute Self-learning Module

11‧‧‧Attribute weight calculation module

12‧‧‧Attribute setting table

2‧‧‧Testing system

3‧‧‧Automatic verification module

4‧‧‧ Case Intelligent Screening Module

41‧‧‧Intelligent Analysis Module

42‧‧‧Attribute Association Module

43‧‧‧ Computing Module

5‧‧‧Case feedback module

51‧‧‧Property extension module

52‧‧‧Case production module

6‧‧‧Verification case library

701~710‧‧‧ Process steps

1 is a system architecture diagram of a system and method for applying the new case feedback to the automation software verification according to the present invention; FIG. 2 is a schematic diagram of the attribute self-learning module of the system and method for applying the new case feedback to the automation software verification according to the present invention; 3 is a detailed schematic diagram of a case intelligent screening module for applying the new case to the automated software verification system and method thereof; FIG. 4 is a case feedback of the system and method for applying the new case feedback to the automated software verification method. A detailed schematic diagram of the module; and FIG. 5 is a flow chart of a method for applying the new case feedback to the automated software verification.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS The preferred embodiments of the present invention will be described in detail below with reference to the drawings.

Please refer to FIG. 1 , which is a system architecture diagram for the application of a new case to the automated software verification, and the system and method thereof can be The automatic feedback produces a new type of case, which mainly includes: an attribute self-learning module 1, providing test attributes and weights; a test set system 2 for automatic verification module 3 to test; an automatic verification module 3 The tester can set the logic check to be performed by the module, and then the automatic verification module 3 executes the test assembly system 2 one by one, and according to the logic check rule of the selected automatic verification module 3, the test fails. The data is listed as a candidate case list; a case intelligent screening module 4 analyzes and calculates the candidate case list and the case in the verification case library 6 one by one to find out that there is no verification case library 6 at present, and the case can extend the new test attribute. a new case list; a case feedback module 5, the case feedback module 5 can refer to the attribute self-learning module 1 to determine the new test attribute that should be added to the verification case library and pass the eXtensible Markup Language (eXtensible Markup Language, XML) file automatically saves the case back to the verification case library 6; a verification case library 6, the verification case library 6 needs to have at least 1 case, negative Case to Case provides intelligent screening module 4 to analyze operations, but also through feedback module 5 case will keep the case back to verify the case base 6.

As described above, the to-be-tested aggregation system 2 mainly collects detailed information of all the cases to be tested. Taking the telecom accounting system as an example, the detailed information includes each user's type, attribute, offer, various transaction information, and the like. The automatic verification module 3 is mainly responsible for the logic verification test, and the system comprehensive logic test is performed for each of the to-be-tested aggregation systems 2 in the automatic verification module 3, and the scope of the logic test is as small as a simple numerical check, up to Complex business algorithm rules. For example, when the verification personnel wants to test the accounting system, in the automatic verification module 3, for the numerical verification, it can be checked whether a program person has caused other side-effects in a series of processing processes of writing the program, resulting in Customers with the same rule conditions have inconsistent costs; for business calculation rules, it can be judged that the monthly bill for each expense account cannot be exceeded. 30 days, or when the equipment does not apply for any changes in the current month, and does not involve the overall rate reduction, the monthly bill should be consistent with the previous month's bill.

Each case in which the verification case library 6 exists must contain at least information such as case code, test attributes, and pass/fail criteria. The case code establishes the correlation between the cases through the coding rules of the hierarchical structure, and the test attribute field can have many The attribute values, in addition to the pass/fail criteria for each case, are mainly used to record information such as the amount criteria for each case.

Please refer to FIG. 2 , which is a detailed schematic diagram of a system for applying the new case to the automated software verification system and its attribute self-learning module, wherein the attribute self-learning module 1 further includes an attribute weight calculation module 11 and an attribute setting table 12 . . In the present invention, a test attribute that may affect the case is generated mainly through the attribute self-learning module 1, wherein the verification person first inputs a valid test attribute and a hierarchy, and then the attribute weight calculation module 11 performs an operation on the input test attribute. The weight of each test attribute, and finally the test attribute, hierarchy, and weight are recorded in the attribute setting table 12.

If the test attribute is X and the user with this test attribute is user n , then the weight of each test attribute is found by the following formula (1):

In the test, the more users represent the attributes used by more people, it is a very important test factor for the case, so the formula (1) is mainly for the number of users of each test attribute in the system. The weight value Weight is found for each attribute, and then the test attribute X and the corresponding Weight( X ) are stored one-to-one in the attribute setting table 12. When the value of the Weight is larger, it means that the number of users with this test attribute is more or the frequency of the change is very frequent, so the more important the test attribute is, the priority of the subsequent judgment will be prioritized, so at least the attribute setting table 12 includes Test attributes, levels, and weights.

Please refer to FIG. 3 , which is a detailed schematic diagram of a case intelligent screening module for applying a new case to the automated software verification system and method thereof. The case intelligent screening module 4 mainly includes an intelligent analysis module 41 and an attribute correlation module. 42 and a computing module 43. First, each case in the candidate case list is analyzed by the intelligent analysis module 41 one by one, and all the test attributes possessed by the case are listed. This attribute can be found one by one in the relevant main file, and the existing attribute setting table 2 is filtered out. The test attributes are listed in an XML format, and then the attribute association module 42 compares the test attributes provided by the intelligent analysis module 41 to the verification case library 6, and the comparison method can take an XML tag to visit, starting from the root tag. Back to the sub-tab, until the candidate case or the verification case library no longer has a common attribute label in any case. At this time, the test attribute existing in the case of the verification case library 6 is used as a benchmark to find the candidate case list and the verification case library 6 attribute. For all cases that are identical and the case number is the largest in the verification case library 6, this comparison method can also use the index structure to enhance the search matching ability. For example, case 1 of the verification case library 6 has the test attributes a, b and the case list of the candidate case has the test attributes a, b, c, d (arranged according to the weight from large to small), but since the present invention is used to verify the case library 6 The case test attribute is the benchmark, so it is judged that the two cases have the same test attributes a, b. Finally, the operation module 43 is used to calculate the case of the candidate case list and the case result of the verification case library 6 through the pass/fail criteria of the case existence in the verification case library 6, and the pass/fail criteria record finally records that the verification case should appear. The result value, so if the calculation result is the same, the verification case library 6 is the same as the candidate case attribute, then it is not necessary to add this case to the verification case library 6; if the case of the candidate case list is different from the result of the verification case library 6 That means the two cases must have Different test attributes and this attribute does not exist in the case of the verification case library 6, which will lead to the inconsistency of the operation results. Therefore, the case of this verification case library 6 becomes the comparison list D(n), and the candidate case list is It is summarized into an extended new type of case, listed as a new case list A (n), and the case feedback module 5 is used for the case production action.

Please refer to FIG. 4 , which is a detailed schematic diagram of a case feedback module for applying a new case to the automated software verification system and a method thereof. The case feedback module 5 is composed of an attribute extension module 51 and a case production module. 52 is composed of a new case list A(n) listed in the case intelligent screening module 4, and different test attributes are found and produced into new cases to be added to the verification case library 6. Firstly, the attribute extension module 51 determines the attributes, first lists all the test attributes of the new case list A(n) in an XML format, and then searches for the tag visit from the maximum weight to find the A(n) ratio. D(n) has more new test attributes, and the new test attribute does not cause A(n) to overlap with other cases in the verification case library 6. Next, the case production module 52 will generate all the exchange information and list the case information in the XML format and add it to the verification case library 6.

Please refer to FIG. 5 , which is a flowchart of a method for applying the new case feedback to the automated software verification according to the present invention. First, step 701 determines whether there is an attribute setting table. If not, step 702 is performed, and the attribute weight in the attribute self-learning module is performed. The calculation module performs attribute weight calculation, and step 703 records the attribute weight calculation module calculation result and attribute to the attribute setting table. In step 704, it is determined whether there is a system to be tested, and the system to be tested needs at least the pen data to continue to step 705. In step 705, an automatic verification module is executed, and step 706 determines whether the system to be tested passes the automatic verification module. In step 707, the case of failing the criteria is found, and the case intelligent screening mode is used. The group finds the case attribute. First, step 708 analyzes whether the case of the failed case and the case of the verification case library have the same test attribute. If the analysis result is found to have the same test attribute, the step 709 is performed, and the case intelligent screening module is used. The calculation module performs calculations to see if the two calculation results are the same. If the results are the same, they represent two identical cases. However, if the calculation results are different, it means that the failed cases belong to the new type of case, and step 710 is performed. Step 708 Analyze the result If it is found that there is no case with the same test attribute, step 710 is directly performed. In step 710, the case feedback module is used to find new attributes of the case and add the case directly back to the verification case library.

In the case of telecom accounting system, the application scenarios are as follows. For example, there are 6 test attributes, including light generation, MOD, availability, and presence or absence, which exist in the attribute setting table, as shown in Table 1:

Please refer to Table 2 below. The verification case library must first save a case. Then, after the execution of the automated verification module, the test aggregation system assumes that one failed case is generated, so it is included in the candidate case list. Then, using the case intelligent screening module, it was found that the candidate case list has the same test attribute as the case code of the verification case library - "Light Generation", but the calculation result is based on the pass/fail criteria candidate case list. The result of the verification case library is 483. It is found that the calculation results are different. Therefore, the candidate case list is listed in the new case list A(n), and the case of the verification case library is listed as D(n). Then use the attribute extension module to find all the test attributes and weights of D(n) in order. 2 For example, it is found that D(1) has the test attribute of light generation. The new case list A(1) has two test attributes of light generation and MoD. Through the comparison, only the unique test attribute of A(1) is found. MOD, and D(1) does not, so the test attribute of MOD makes the case result of the new case list and the verification case library 6 inconsistent, so the test attribute of MOD must be added to the verification case library 6 through the case production module.

The case production module will add all the case information listing the new case list to the verification case library through the exchange file. In the above example, combined with the test attributes of the case code in Table 2 and other relevant information of the new case list, the exchange file is produced, and the content of the exchange file can be expressed in XML, for example, a tag named MOD_TYPE is TRUE. And a label named EQUIP_TYPE value is TRUE and so on to the verification case library, so in the verification case library, the results as shown in Table 3 will be presented.

The detailed description of the present invention is intended to be illustrative of a preferred embodiment of the invention, and is not intended to limit the scope of the invention. The patent scope of this case.

To sum up, this case is not only innovative in terms of technical thinking, but also has many of the above-mentioned functions that are not in the traditional methods of the past. It has fully complied with the statutory invention patent requirements of novelty and progressiveness, and applied for it according to law. You have approved this invention patent application, in order to invent invention, to the sense of virtue.

1‧‧‧Attribute Self-learning Module

2‧‧‧Testing system

3‧‧‧Automatic verification module

4‧‧‧ Case Intelligent Screening Module

5‧‧‧Case feedback module

6‧‧‧Verification case library

Claims (8)

A system for applying new cases to the automated software verification system, the system consists of: a system to be tested, which aggregates all potential cases to be tested, and aggregates detailed information of all cases to be tested, such as user type, Attributes, offers, or various changes; an attribute self-learning module that generates attribute information and weights required for the verification case library; an automated verification module that performs a comprehensive logical test for each of the to-be-tested assembly systems, and Remove the list of candidate cases that are not applicable; a case intelligent screening module, analyze and calculate the candidate case list and the case in the verification case database one by one, and find out the case where there is no verification case library at present, and is responsible for screening out A new case list, wherein the verification case library requires at least one case; a case feedback module refers to the attribute self-learning module to determine the new test attribute that should be added to the verification case library at the end, and is extensible The tagged language file automatically finds the attributes of the new case list and feeds the case back to the verification case library And a verification case library for storing the verification case and receiving the case generated by the case feedback module. For example, the application of the new case described in claim 1 is fed back to the automated software verification system, wherein the attribute self-learning module comprises: an attribute weight calculation module, which calculates each test attribute input by the verification personnel. The weight value; an attribute setting table records the test attribute, the level and the weight value generated by the attribute weight calculation module. For example, the application of the new case described in the first application of the patent scope is fed back to the automated software verification system, wherein the case of the intelligent screening module comprises: an intelligent analysis module, which performs attribute analysis of candidate cases; The module is a candidate case in the intelligent analysis module to find out whether the test attribute has the same test attribute as the existing verification case library; an operation module, the existing case and the candidate identified by the attribute association module Cases are calculated and compared, and the results of the comparison are inconsistent. For example, the application of the new case described in the third application of the patent scope is fed back to the automated software verification system, wherein the attribute association module is recursively accessed by the tag, and the intelligent analysis module analyzes the test attribute of the candidate case, and The existing attributes in the verification case library are compared and judged to confirm whether there are candidate cases with the same test attribute. If the same case is successfully found, the calculation is performed by the operation module, and if the same case is not found at all, the case is The feedback module is processed. For example, the application of the new case described in claim 1 is fed back to the automated software verification system, wherein the composition of the case feedback module includes: an attribute extension module, which is to find the new case and the existing verification case library. Different new test attributes; a case production module, which is added to the verification case library by the new attribute and case information found by the attribute extension module. The application case described in item 5 of the patent application scope is fed back to the system for automated software verification, wherein the case production module generates an exchange file in a tagged language manner, such as an extensible markup language format. A method for applying a new case to the automated software verification method, the steps of which are: Step 1: The need for the verification case library generated by the attribute self-learning module The attribute information; step 2, the automatic verification module executes the aggregate system to be tested one by one, and aggregates the detailed information of all the cases to be tested, such as the user type, attribute, offer, or various information, to find out the failure criteria. The case is formed as a candidate case list; step 3, the candidate case list is found through the case intelligent screening module to find the attribute of the candidate case, and compared with the current case of the verification case library; step four, if the step is three If the result is found to have the same case, the candidate case and the current case of the verification case library are calculated through the operation module; step 5, if the result of the step 4 operation is consistent, the end does not do any feedback, if the step 4 operation If the results are inconsistent, the new attribute will be found through the case feedback module and the case exchange file will be returned to the verification case database. Step 6. If the result of the comparison in step 3 is inconsistent, find the new attribute through the case feedback module. The case is stored in the verification case library, and the verification case library requires at least one case. The method for applying the new case to the automated software verification method as described in claim 7 of the patent application scope, wherein the step of the attribute self-learning module generating the attribute information comprises: step 1: confirming whether the attribute setting table has at least one attribute Information; Step 2: If there is no data in the attribute setting table, use the attribute The weight calculation module calculates the weight of the valid test attribute and the level of the attribute input by the verification personnel; and the third step, records the weight generated by the second step, the test attribute and the level into the attribute setting table.