CN113190460B - Automatic test case generation method and device - Google Patents
Automatic test case generation method and device Download PDFInfo
- Publication number
- CN113190460B CN113190460B CN202110569902.7A CN202110569902A CN113190460B CN 113190460 B CN113190460 B CN 113190460B CN 202110569902 A CN202110569902 A CN 202110569902A CN 113190460 B CN113190460 B CN 113190460B
- Authority
- CN
- China
- Prior art keywords
- case
- test
- test case
- decision tree
- cases
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000012360 testing method Methods 0.000 title claims abstract description 320
- 238000000034 method Methods 0.000 title claims abstract description 57
- 238000003066 decision tree Methods 0.000 claims abstract description 111
- 238000007637 random forest analysis Methods 0.000 claims abstract description 66
- 238000003860 storage Methods 0.000 claims description 27
- 238000012549 training Methods 0.000 claims description 27
- 238000013138 pruning Methods 0.000 claims description 19
- 238000004590 computer program Methods 0.000 claims description 17
- 238000010276 construction Methods 0.000 claims description 6
- 238000013473 artificial intelligence Methods 0.000 abstract description 2
- 230000006870 function Effects 0.000 description 17
- 238000004891 communication Methods 0.000 description 16
- 230000004048 modification Effects 0.000 description 16
- 238000010586 diagram Methods 0.000 description 15
- 238000012545 processing Methods 0.000 description 10
- 238000004519 manufacturing process Methods 0.000 description 8
- 238000012216 screening Methods 0.000 description 8
- 238000012986 modification Methods 0.000 description 6
- 239000000872 buffer Substances 0.000 description 4
- 238000010801 machine learning Methods 0.000 description 4
- 230000008569 process Effects 0.000 description 4
- 230000005540 biological transmission Effects 0.000 description 3
- 238000004422 calculation algorithm Methods 0.000 description 2
- 238000013461 design Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 238000012827 research and development Methods 0.000 description 2
- 238000012546 transfer Methods 0.000 description 2
- 230000005856 abnormality Effects 0.000 description 1
- 230000008901 benefit Effects 0.000 description 1
- 230000001413 cellular effect Effects 0.000 description 1
- 239000002131 composite material Substances 0.000 description 1
- 230000008878 coupling Effects 0.000 description 1
- 238000010168 coupling process Methods 0.000 description 1
- 238000005859 coupling reaction Methods 0.000 description 1
- 230000003247 decreasing effect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 239000011521 glass Substances 0.000 description 1
- 230000008676 import Effects 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 239000011159 matrix material Substances 0.000 description 1
- 238000010295 mobile communication Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000002360 preparation method Methods 0.000 description 1
- 238000002407 reforming Methods 0.000 description 1
- 238000000926 separation method Methods 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 238000000638 solvent extraction Methods 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/36—Preventing errors by testing or debugging software
- G06F11/3668—Software testing
- G06F11/3672—Test management
- G06F11/3684—Test management for test design, e.g. generating new test cases
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/243—Classification techniques relating to the number of classes
- G06F18/24323—Tree-organised classifiers
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q40/00—Finance; Insurance; Tax strategies; Processing of corporate or income taxes
- G06Q40/02—Banking, e.g. interest calculation or account maintenance
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Physics & Mathematics (AREA)
- Business, Economics & Management (AREA)
- General Engineering & Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Accounting & Taxation (AREA)
- Evolutionary Biology (AREA)
- Bioinformatics & Computational Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Artificial Intelligence (AREA)
- Finance (AREA)
- Evolutionary Computation (AREA)
- Life Sciences & Earth Sciences (AREA)
- Quality & Reliability (AREA)
- Computer Hardware Design (AREA)
- Development Economics (AREA)
- Economics (AREA)
- Marketing (AREA)
- Strategic Management (AREA)
- Technology Law (AREA)
- General Business, Economics & Management (AREA)
- Debugging And Monitoring (AREA)
Abstract
The application provides a test case automatic generation method and device, relates to the field of artificial intelligence, can be used in the financial field, and comprises the following steps: obtaining a test case to be selected; inputting the test cases to be selected into a pre-constructed random forest case selection model to determine the current test cases according to the attributes of the selected test cases; the random forest case selection model includes a plurality of pre-generated decision tree classifiers. The method and the device can automatically generate the test cases by utilizing the pre-built random forest case selection model.
Description
Technical Field
The application relates to the field of artificial intelligence, and can be used in the financial field, in particular to a method and a device for automatically generating test cases.
Background
The acceptance test is the last test link before software deployment, and has the characteristic of black box/gray box test, that is, software testers often only know the program list to be tested, and lack of grasp on specific calling, modes and implementation modes of the program. In the conventional acceptance test, a tester can develop a test according to a test case provided by a developer and a corresponding program list to be tested.
However, for enterprise-level software engineering, the code is often provided with services after being packaged in multiple ways, and the coupling degree of part of the code is high, so that test holes such as insufficient test and incomplete coverage of stock transaction transformation can easily occur if test cases are formulated by testers only according to software system designs and program to be tested lists provided by developers. In order to reduce the risk of vulnerability occurrence, the traditional method is to convert the past product requirements into test assets, and then continuously accumulate stock cases with higher production risks to form a more complete test case library. However, the above method needs to rely on a great deal of manpower to complete the coverage of the full test scene, and has high cost and low benefit.
Disclosure of Invention
Aiming at the problems in the prior art, the application provides a method and a device for automatically generating test cases, which can automatically generate the test cases by utilizing a pre-constructed random forest case selection model.
In order to solve the technical problems, the application provides the following technical scheme:
in a first aspect, the present application provides a method for automatically generating test cases, including:
obtaining a test case to be selected;
inputting the test cases to be selected into a pre-constructed random forest case selection model to determine the current test cases according to the attributes of the selected test cases; the random forest case selection model includes a plurality of pre-generated decision tree classifiers.
Further, before inputting the test case to be selected into the pre-constructed random forest case selection model to determine the current test case according to the attribute of the selected test case, the method further comprises:
acquiring a list of program reformation in the current period;
determining whether a to-be-selected test case to be input into the random forest case selection model is a test case corresponding to the program which is reformed in the current period according to the corresponding relation between the pre-established program and the case;
if so, the test case is determined to be the current test case option.
Further, the step of pre-generating a decision tree classifier includes:
randomly dividing the acquired historical testing cases and historical non-testing cases into a training set and a testing set according to a preset dividing proportion;
determining information gain values of all attributes corresponding to the test cases to be selected in the training set;
constructing the decision tree classifier according to the magnitude of the information gain value; the decision tree classifier comprises attribute nodes corresponding to the attributes.
Further, after constructing the decision tree classifier according to the magnitude of the information gain value, the method further comprises:
pruning the decision tree classifier by using the test set.
Further, pruning the decision tree classifier with the test set includes:
each attribute node in the decision tree classifier is removed in sequence, and the accuracy of the decision tree classifier after removing the attribute node is calculated according to the test set when determining the test case;
comparing the accuracy corresponding to the decision tree classifier after the attribute node is removed with the accuracy of the decision tree classifier before the attribute node is removed when the test case is determined;
pruning is carried out on the decision tree classifier according to the comparison result.
Further, the inputting the test case to be selected into a pre-constructed random forest case selection model to determine the current test case according to the attribute of the selected test case includes:
determining corresponding initial cases to be tested in the period according to the attribute of the selected test cases by utilizing each decision tree classifier;
voting is carried out on the initial case to be tested in each period, and the case to be tested in the period is determined according to the voting result.
In a second aspect, the present application provides an automatic test case generating device, including:
the case acquisition unit is used for acquiring a test case to be selected;
The case generation unit is used for inputting the to-be-selected test case into a pre-constructed random forest case selection model so as to determine a current period to-be-tested case according to the attribute of the selected test case; the random forest case selection model includes a plurality of pre-generated decision tree classifiers.
Further, the automatic test case generating device further includes:
the reconstruction list acquisition unit is used for acquiring a list of the current program reconstruction;
the corresponding case determining unit is used for determining whether the to-be-selected test case to be input into the random forest case selection model is the test case corresponding to the program which is reformed in the current period according to the corresponding relation between the pre-established program and the case;
and the test case corresponding unit is used for determining the test case as the current test case.
Further, the automatic test case generating device further comprises:
the training set test set dividing unit is used for randomly dividing the acquired historical to-be-tested cases and the historical to-be-tested cases into a training set and a test set according to a preset dividing proportion;
the information gain determining unit is used for determining information gain values of all the attributes corresponding to the test cases to be selected in the training set;
The decision tree classifier construction unit is used for constructing the decision tree classifier according to the magnitude of the information gain value; the decision tree classifier comprises attribute nodes corresponding to the attributes.
Further, the decision tree classifier construction unit is further specifically configured to:
pruning the decision tree classifier by using the test set.
Further, the decision tree classifier construction unit includes:
the accuracy rate determining module is used for sequentially eliminating each attribute node in the decision tree classifier, and calculating the accuracy rate of the decision tree classifier after eliminating the attribute node according to the test set when determining the test case;
the accuracy rate comparison module is used for comparing the accuracy rate corresponding to the decision tree classifier after the attribute node is removed with the accuracy rate of the decision tree classifier before the attribute node is removed when the test case is determined;
and the pruning module is used for pruning the decision tree classifier according to the comparison result.
Further, the case generating unit includes:
the test initial case determining module is used for determining a corresponding initial case to be tested in the period according to the attribute of the selected test case by utilizing each decision tree classifier;
And the current period supposed test case determining module is used for voting the initial supposed test cases in each current period and determining the current period supposed test cases according to voting results.
In a third aspect, the present application provides an electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, the processor implementing the steps of the test case automatic generation method when the program is executed by the processor.
In a fourth aspect, the present application provides a computer readable storage medium having stored thereon a computer program which, when executed by a processor, performs the steps of the test case automatic generation method.
Aiming at the problems in the prior art, the automatic test case generation method and the automatic test case generation device can automatically generate test cases by utilizing the pre-built random forest case selection model, assist software testers in working, and can also perform black box/gray box test even if the software testers do not know the content of the program codes, and the screened test cases have stronger relevance with the program code list and the risks of the cases.
Drawings
In order to more clearly illustrate the embodiments of the invention or the technical solutions in the prior art, the drawings that are required in the embodiments or the description of the prior art will be briefly described, it being obvious that the drawings in the following description are only some embodiments of the invention, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
Fig. 1 is a flowchart of a method for automatically generating test cases according to an embodiment of the present application;
FIG. 2 is a second flowchart of a method for automatically generating test cases according to an embodiment of the present application;
fig. 3 is a third flowchart of a method for automatically generating test cases according to an embodiment of the present application;
FIG. 4 is a flow chart of pruning a decision tree classifier using a test set in an embodiment of the present application;
FIG. 5 is a flow chart of determining the current test case in the embodiment of the present application;
fig. 6 is a diagram of one of the structures of the automatic test case generating apparatus according to the embodiment of the present application;
FIG. 7 is a second block diagram of an automatic test case generating apparatus according to an embodiment of the present application;
fig. 8 is a third diagram of a test case automatic generation device according to an embodiment of the present application;
FIG. 9 is a block diagram of a decision tree classifier building unit in an embodiment of the present application;
fig. 10 is a block diagram of a case generating unit in the embodiment of the present application;
fig. 11 is a schematic structural diagram of an electronic device in an embodiment of the present application;
FIG. 12 is a schematic diagram of the trend of data flow in an embodiment of the present application;
fig. 13 is a schematic diagram of a decision tree classifier in an embodiment of the present application.
Detailed Description
The following description of the technical solutions in the embodiments of the present application will be made clearly and completely with reference to the drawings in the embodiments of the present application, and it is apparent that the described embodiments are only some embodiments of the present application, not all embodiments. All other embodiments, which can be made by one of ordinary skill in the art without undue burden from the present disclosure, are within the scope of the present disclosure.
It should be noted that, the method and the device for automatically generating the test case provided by the application can be used in the financial field and any field except the financial field, and the application field of the method and the device for automatically generating the test case provided by the application is not limited.
Referring to fig. 1, in order to automatically generate test cases using a pre-constructed random forest case selection model, the present application provides a test case automatic generation method, including:
s101: obtaining a test case to be selected;
s102: inputting the test cases to be selected into a pre-constructed random forest case selection model to determine the current period of the test cases according to the attributes of the selected test cases; the random forest case selection model includes a plurality of pre-generated decision tree classifiers.
It can be appreciated that the automatic test case generation method provided by the application automatically generates test cases by utilizing a pre-constructed random forest case selection model, and assists software testers in test preparation work. The software black box/gray box test can be performed even if the software tester does not fully understand the specific contents of the program code. The current period of the detected cases screened by using the pre-constructed random forest case selection model has stronger relevance with the program function abnormality occurrence risks corresponding to the detected cases.
Specifically, the present application can screen out the current test case using the test case screening system as shown in fig. 12. The test case screening system comprises: (1) the test case output server is used for screening the current test case and outputting the current test case; (2) the random forest case selection model server is used for storing a pre-constructed random forest case selection model; (3) the test case information storage server is used for storing all the test cases to be selected; (4) the case and code corresponding relation storage server (hbase) is used for storing the corresponding relation between the test case to be selected and the code, and the corresponding relation can represent the test case to be selected corresponding to each section of code; (5) the research and development test management server and the portable computer are used for facilitating software test deployment of software developers and software testers.
In specific implementation, a software tester enters a test case corresponding to a historical expected modification program provided by a software developer and a program list associated with the test case into a case and code corresponding relation storage server (hbase) shown in fig. 12. When the new (current) version test period starts, the test case screening system screens out reasonable test cases according to the new (current) program modification list.
The case-code correspondence storage server may store test cases and associated program listings provided by a software developer. Considering that the number of programs associated with cases can be different, and the associated programs also have the characteristics of expandability and the like, the device is intended to store the data of the corresponding relation between the cases and the codes on the Bigtable.
Bigtable is a sparse, distributed, persisted, multi-dimensional ordered mapping table, with each piece of data being a key-value pair. The bigable keys have three dimensions, row key (row key), column key (column key), and timestamp (timestamp), respectively, and value is a string. In bigable, the number of columns corresponding to each row is not limited, and can be increased or decreased at any time. In this way, the model may store the candidate test case number as a row key for each key-value pair, and store the engineering and code paths involved in the candidate test case as column keys, with the value stored as 0 or 1. So that any number of code to case correspondence can be stored.
For example, the scan code registration case number 10 relates to four codes of the scan code component a of the system A, the login public component b, the inquiry bank card number code c and the httpser ivet code d, so that four pieces of data can be stored in the Bigtable, the row key is 10, the column keys are A-a, A-b, A-c and A-d, and the value is 1. If the new associated B system e code is needed in the subsequent modification of the case, a key-value pair with a row key of 10 and a column key of B-e is added, so that the requirement can be met. In this way, when the A system program a is modified, all cases related to the A system program a are queried, only all cases with the column key of A-a are queried, and then the queried results are displayed to software testers.
According to the embodiment of the application, the random forest case selection model can be utilized to screen the current test cases according to whether the test cases to be selected relate to money transactions such as transfer and remittance payment (hereinafter referred to as dynamic account), whether the test cases are high in concurrence, whether large-scale complaints of clients are easy to cause, whether production problems occur recently and the like (hereinafter referred to as case attributes).
From the above description, the test case automatic generation method provided by the application can automatically generate test cases by using a pre-constructed random forest case selection model, assist software testers in working, and perform black box/gray box test even if the software testers do not know the content of the program codes, and the screened test cases have strong relevance with the program code list and the risks of the cases themselves.
Referring to fig. 2, before inputting the test case to be selected into the pre-built random forest case selection model to determine the current test case according to the attribute of the selected test case, the method further includes:
s201: acquiring a list of program reformation in the current period;
s202: determining whether a to-be-selected test case to be input into a random forest case selection model is a test case corresponding to the program which is reformed in the present period according to the corresponding relation between the pre-established program and the case;
S203: if yes, determining the test case as the optional case to be tested in the period;
s204: and continuing to screen other test cases to be selected.
It will be appreciated that, considering that the software test will be performed using the test cases after each program modification is completed, only the program modified in this period needs to be tested, and no repeated test is required for the program modified in the past period. The software tester can learn which programs belong to the current-period program modification program according to the current-period program modification list provided by the developer. Because the corresponding relation between the program and the case is stored in the corresponding relation between the case and the code storage server (hbase) shown in fig. 12, a software tester can determine whether the to-be-selected test case to be input into the random forest case selection model is the test case corresponding to the program modified in the present period according to the corresponding relation between the pre-established program and the case; if yes, determining the test case as the optional case to be tested in the period; otherwise, continuing to screen other test cases to be selected. The optional case to be tested in the present period is input into a random forest case selection model as a test case to be selected to participate in screening, so that the optional case to be tested in the present period is finally obtained.
From the above description, the test case automatic generation method provided by the application can determine the optional case to be tested in the present period.
Referring to fig. 3, the step of pre-generating a decision tree classifier includes:
s301: randomly dividing the acquired historical testing cases and historical non-testing cases into a training set and a testing set according to a preset dividing proportion;
s302: in a training set, determining information gain values of all attributes corresponding to the test cases to be selected;
s303: constructing a decision tree classifier according to the magnitude of the information gain value; the decision tree classifier comprises attribute nodes corresponding to the attributes.
It will be appreciated that the random forest case selection model in embodiments of the present application includes a plurality of pre-generated decision tree classifiers. The random forest case selection model can select the corresponding case to be tested of the current program modification version according to the attributes of the cases. The attributes of the cases may be used as criteria for determining whether the test case to be selected is the case to be tested. Attributes in embodiments of the present application include, but are not limited to, whether the test case under selection causes production problems, whether the test case under selection involves dynamic accounting, whether the test case under selection is highly concurrent, whether the test case under selection is prone to complaints, and the like.
The decision tree classifiers may be generated according to the program modified version, that is, each decision tree classifier corresponds to a first-stage program modified version, but the application is not limited thereto, and the decision tree classifiers may be generated according to other partitioning manners.
Specifically, referring to fig. 12, a decision tree classifier can be generated as follows:
(1) the software tester imports the candidate test cases and the program list related to the candidate test cases provided by the development colleague aiming at the program modification version of a certain period of history in the case test output system. The case test output system stores the information into a case and code corresponding relation storage server.
(2) And the software tester divides all the test cases to be selected into a training set and a test set in the case test output system and stores the training set and the test set into the test case information storage server. And according to the test condition and the production condition of the past modification version, all the test cases to be selected in the training set and the test set are characterized as the test cases of the past modification version or the past modification version without test cases. Then, triggering a random forest algorithm to obtain a decision tree classifier corresponding to the modified version of the period (a period in the past).
By applying the method, the embodiment of the application can generate the corresponding decision tree classifier aiming at the program modified version in each past period, and the decision tree classifiers can jointly form a random forest case selection model for screening out the current period to-be-tested cases.
It should be noted that the decision tree classifier is a tree-structured selector, and the uppermost node is the root node. The root node is the beginning of the entire decision tree classifier, each branch is a new decision node, and each decision node represents a problem or a decision. The machine learning process of the decision tree classifier can be completed through a training set, and the specific machine learning process can refer to the existing machine learning technology. The branching points of the decision tree classifier can be determined by machine learning.
For example, if a decision tree classifier corresponding to a modified version of a previous program is generated using the results of the previous test, the following steps are typically required:
a. and the software tester classifies all the test cases to be selected into the upper-period test cases or the upper-period test cases without test cases according to the upper-period software test conditions and the running conditions after production.
b. The cases are randomly divided into training sets and test sets. The training set can be used for fitting out the decision tree classifier, and the testing set can be used for pruning the fitted decision tree classifier so as to solve the problem of model overfitting. In a preferred embodiment, the ratio of training set to test set may be 4 to 1.
c. Generating a root node in the training set, and selecting a certain attribute of the training set case as a first branch node according to a certain dividing principle (such as an information gain maximum principle).
It should be noted that the information gain principle relates to the concept of information entropy, which is the most commonly used index for measuring the purity of a sample set. Taking the embodiment of the present application as an example, assuming that the proportion of the to-be-tested cases in the current sample set D (training set) is p1 and the proportion of the unnecessary test cases is p2, the information entropy of the training set is Ent (D) = - (p1×log (p1) +p2×log (p 2)), and the operation of taking the logarithm can be generally based on 2.
Then, the information gain of each attribute needs to be calculated, and the current sample set D is divided into two parts D1 and D2 according to a certain attribute a of the training set as a dividing condition, for example, whether the test case to be selected involves dynamic account.
According to the above formula, the information entropy D1 and D2 of the two subsets can be calculated respectively. Considering that the number of samples (to-be-selected test cases) of two sets is different, corresponding weights |d1|/|d| and |d2|/|d| can be given to the two subsets, so that an information gain obtained by dividing the sample set D by the to-be-selected test case on the attribute of whether dynamic account is involved is calculated:
Gain (D, a) =ent (D) - (|d1|/|d) ×ent (D1) - (|d2|/|d) ×ent (D2); wherein the sum of |d1|/|d| and |d2|/|d| should be one.
According to the method, the information gain of all the attributes can be calculated, and the attribute with the maximum information gain is selected as the first dividing principle (first node). And removing the cases (samples) which are selected as the to-be-tested cases according to the dividing principle, and using the removed residual sample (test case) set for the next dividing use.
d. And then selecting the next division principle from the residual sample attributes in the residual sample set in the step c according to the same division principle (such as the information gain maximum principle), and repeating the step c. When all the attributes are selected, the decision tree classifier is obtained.
From the above description, the test case automatic generation method provided by the application can generate a decision tree classifier in advance, so as to generate a random forest case selection model.
In one embodiment, after constructing the decision tree classifier according to the magnitude of the information gain value, the method further comprises: pruning the decision tree classifier by using the test set.
Referring to fig. 4, pruning the decision tree classifier with the test set includes:
S401: each attribute node in the decision tree classifier is removed in sequence, and the accuracy of the decision tree classifier after removing the attribute node is calculated according to the test set when determining the test case;
s402: comparing the accuracy corresponding to the decision tree classifier after the attribute node is removed with the accuracy of the decision tree classifier before the attribute node is removed when determining the test case;
s403: pruning is carried out on the decision tree classifier according to the comparison result.
It can be understood that, in the embodiment of the present application, the above test set may be used to perform pruning processing on the generated decision tree classifier, that is, to reject redundant decision tree branches, so as to avoid model overfitting. For example, a post pruning method may be adopted to perform pruning, after each node of the decision tree is removed by using each to-be-selected test case in the test set in sequence, the accuracy of the decision tree classifier after removing the node is calculated and compared with the accuracy of the decision tree classifier before removing the node, and if the accuracy after removing is increased, the node is removed from the decision tree. The accuracy refers to the truly reasonable proportion of the test cases screened by the decision tree classifier. Whether or not it is reasonable can be judged by a computer program or manually.
From the above description, the automatic test case generation method provided by the application can utilize the test set to prune the decision tree classifier.
Referring to fig. 5, inputting the test cases to be selected into a pre-built random forest case selection model to determine the current test case according to the attributes of the selected test cases, including:
s501: determining corresponding initial cases to be tested in the period according to the attribute of the selected test cases by utilizing each decision tree classifier;
s502: voting is carried out on the initial case to be tested in each period, and the case to be tested in the period is determined according to the voting result.
It will be appreciated that the foregoing steps have described how to generate a decision tree classifier, and that the random forest case selection model is a model that contains multiple decision tree classifiers, one for each decision tree. For one input sample (test case to be selected), N trees may give N results, some of which are considered as the application cases and some of which are considered as the no-need test cases.
The random forest case selection model needs to inherit the voting results of all classifiers. For a test case to be selected, if more than one third of the decision tree classifier considers it to be the test case, then the random forest case selection model considers it to be the test case. The decision tree classifiers in the embodiment of the application can be respectively fitted according to the test conditions of the modified versions of the historical programs to obtain the decision tree classifier corresponding to the modified versions of the historical programs.
In terms of code implementation, the packet sklearn. Tree of the Python decision tree can be directly used to form different decision trees according to the principles of id3, cart, c4.5 and the like.
From the above description, the test case automatic generation method provided by the application can input the test case to be selected into a pre-constructed random forest case selection model to determine the current test case according to the attribute of the selected test case.
In one embodiment, the random forest case selection model server may be configured to select a corresponding test case of the current modified version of the program according to the attributes of the case. The attribute is a standard for determining whether the division case is a case to be tested, and the attribute in the application includes but is not limited to: whether the case causes production problems, whether the case relates to a movable account, a product line to which the case belongs, whether the case is high concurrency, whether the case related case is a modified case of the current modification (the information is stored in a test case information storage server), and whether a program related to the case is in a modified program list (the part of information is stored in the case and code corresponding relation storage server). The specific screening process may be as follows:
step one: different software testers input case numbers needing to be screened by a random forest case selection model according to the test product line where the software testers are located, and initiate requests to a test case output server.
For example, an electronic banking basic function product line may only enter case numbers for test cases related thereto, such as login, registration, transfer, etc., without concern for test cases of other product lines.
This step involves the test case output server: the system is a user-oriented Web project and is used for communicating engineering users (namely software testers) with other devices in a test case screening system, and the functions of the system comprise receiving test content information input by the testers and storing the test content information in a corresponding storage device, displaying the test cases needing to be tested in the present period to the testers, generating the test cases according to a corresponding relation between version reconstruction program list query cases and codes, scheduling a random forest selection algorithm generation selector, scheduling a random forest case selection model to obtain the test cases in the present period and the like. The test case output server can adopt a front-end and back-end separation method in engineering realization, wherein the front end is realized by using a vue framework, and the back end is realized by using a springboot framework. The design can lead in information from the paging directly by the user, read data from each storage system and generate a new random forest case selection model. After the random forest case selection model is built, the rear end reads a program list which is reformed in the present period in a software management system (existing system), a to-be-selected test case of a reforming code which is associated with the present period is obtained through a case-to-code correspondence relation storage server, case characteristic information is obtained through a test case characteristic information storage device, the attribute of each sample case is generated, the data are transmitted to the random forest case selection model for entering, the result calculated by the model is displayed back to a software tester through vue, and the cases of different product lines which are displayed back to the software tester possibly are different.
Step two: the test case output server acquires a current version modification program list from the research and development test management server;
step three: the test case output server screens out the case to be tested according to the corresponding relation between the modification program list query case and the code storage server;
this step involves the test case information storage server: the device is used for storing the Chinese name, the case number, the associated case number of the case test, whether the case causes production problems, whether the case relates to a dynamic account, the product line to which the case belongs, whether the case is a high concurrency case attribute and the like of each to-be-tested case, and the case to be tested is required in the present period. The data may be stored in a mysql database in a distributed manner, e.g., each case chinese name and case number is stored in a separate centralized repository, and the remaining cases are stored in different mysql repositories in a distributed manner based on the hash value of the case number. The case attributes are used as input attributes of a decision tree classifier of a random forest case selection model.
Step four: the test case output server dispatches the random forest case selection model server, and case matching is started according to the fitted random forest matrix.
Step five: and D, storing the detected cases screened in the fifth step into a test case information storage server by a random forest case selection model.
For example, an example of the result of a decision tree classifier is shown in FIG. 13. Taking an electronic bank login test case as an example, firstly judging whether a first condition is that a case association is not dynamic account, then judging whether the case is easy to cause customer complaints or not under a second condition, and judging that the case is changed into a case to be tested by the decision tree.
Based on the same inventive concept, the embodiment of the present application also provides an automatic test case generating device, which can be used to implement the method described in the above embodiment, as described in the following embodiment. Because the principle of the test case automatic generation device for solving the problem is similar to that of the test case automatic generation method, the implementation of the test case automatic generation device can be referred to the implementation of the determination method based on the software performance benchmark, and the repetition is not repeated. As used below, the term "unit" or "module" may be a combination of software and/or hardware that implements the intended function. While the system described in the following embodiments is preferably implemented in software, implementation in hardware, or a combination of software and hardware, is also possible and contemplated.
Referring to fig. 6, in order to automatically generate test cases using a random forest case selection model constructed in advance, the present application provides an automatic test case generation apparatus, including:
a case acquisition unit 601, configured to acquire a test case to be selected;
a case generation unit 602, configured to input the test case to be selected into a pre-constructed random forest case selection model to determine a current test case according to the attribute of the selected test case; the random forest case selection model includes a plurality of pre-generated decision tree classifiers.
Referring to fig. 7, the automatic test case generating device further includes:
a modification list obtaining unit 701, configured to obtain a list of current-period program modifications;
a corresponding case determining unit 702, configured to determine, according to a pre-established correspondence between a program and a case, whether a to-be-selected test case to be input into the random forest case selection model is a test case corresponding to the program that is modified in the present period;
the test case is determined as the current test case by the test case correspondence unit 703.
Referring to fig. 8, the automatic test case generating device further includes:
the training set test set dividing unit 801 is configured to randomly divide the obtained historical to-be-tested cases and the obtained historical to-be-tested cases into a training set and a test set according to a preset dividing ratio;
An information gain determining unit 802, configured to determine, in the training set, an information gain value of each attribute corresponding to the test case to be selected;
a decision tree classifier construction unit 803, configured to construct the decision tree classifier according to the magnitude of the information gain value; the decision tree classifier comprises attribute nodes corresponding to the attributes.
In an embodiment, the automatic test case generating device, the decision tree classifier building unit is further specifically configured to:
pruning the decision tree classifier by using the test set.
Referring to fig. 9, the decision tree classifier construction unit 803 includes:
the accuracy rate determining module 901 is configured to reject each attribute node in the decision tree classifier in turn, and calculate, according to the test set, the accuracy rate of the decision tree classifier after rejecting the attribute node when determining the test case;
the accuracy rate comparison module 902 is configured to compare the accuracy rate corresponding to the decision tree classifier after the attribute node is removed with the accuracy rate of the decision tree classifier before the attribute node is removed when the test case is determined;
and the pruning module 903 is configured to prune the decision tree classifier according to the comparison result.
Referring to fig. 10, the case generation unit 602 includes:
the initial case to be tested determining module 1001 is configured to determine a corresponding initial case to be tested in the present period according to the attribute of the selected test case by using each decision tree classifier;
the current period should be measured case determining module 1002 is configured to vote on initial cases of each current period should be measured, and determine the current period should be measured case according to the voting result.
In order to automatically generate test cases by using a pre-constructed random forest case selection model from a hardware level, the application provides an embodiment of an electronic device for implementing all or part of contents in the automatic test case generation method, wherein the electronic device specifically comprises the following contents:
a Processor (Processor), a Memory (Memory), a communication interface (Communications Interface), and a bus; the processor, the memory and the communication interface complete communication with each other through the bus; the communication interface is used for realizing information transmission between the test case automatic generation device and related equipment such as a core service system, a user terminal, a related database and the like; the logic controller may be a desktop computer, a tablet computer, a mobile terminal, etc., and the embodiment is not limited thereto. In this embodiment, the logic controller may refer to an embodiment of the automatic test case generating method and an embodiment of the automatic test case generating device, and the contents thereof are incorporated herein and are not repeated here.
It is understood that the user terminal may include a smart phone, a tablet electronic device, a network set top box, a portable computer, a desktop computer, a Personal Digital Assistant (PDA), a vehicle-mounted device, a smart wearable device, etc. Wherein, intelligent wearing equipment can include intelligent glasses, intelligent wrist-watch, intelligent bracelet etc..
In practical applications, part of the method for automatically generating test cases may be performed on the electronic device side as described above, or all operations may be performed in the client device. Specifically, the selection may be made according to the processing capability of the client device, and restrictions of the use scenario of the user. The present application is not limited in this regard. If all operations are performed in the client device, the client device may further include a processor.
The client device may have a communication module (i.e. a communication unit) and may be connected to a remote server in a communication manner, so as to implement data transmission with the server. The server may include a server on the side of the task scheduling center, and in other implementations may include a server on an intermediate platform, such as a server on a third party server platform having a communication link with the task scheduling center server. The server may include a single computer device, a server cluster formed by a plurality of servers, or a server structure of a distributed device.
Fig. 11 is a schematic block diagram of a system configuration of an electronic device 9600 of an embodiment of the present application. As shown in fig. 11, the electronic device 9600 may include a central processor 9100 and a memory 9140; the memory 9140 is coupled to the central processor 9100. Notably, this fig. 11 is exemplary; other types of structures may also be used in addition to or in place of the structures to implement telecommunications functions or other functions.
In one embodiment, the test case automatic generation method functionality may be integrated into the central processor 9100. The central processor 9100 may be configured to perform the following control:
s101: obtaining a test case to be selected;
s102: inputting the test cases to be selected into a pre-constructed random forest case selection model to determine the current test cases according to the attributes of the selected test cases; the random forest case selection model includes a plurality of pre-generated decision tree classifiers.
From the above description, the test case automatic generation method provided by the application can automatically generate test cases by using a pre-constructed random forest case selection model, assist software testers in working, and perform black box/gray box test even if the software testers do not know the content of the program codes, and the screened test cases have strong relevance with the program code list and the risks of the cases themselves.
In another embodiment, the test case automatic generation device may be configured separately from the central processing unit 9100, for example, the test case automatic generation device of the data composite transmission device may be configured as a chip connected to the central processing unit 9100, and the function of the test case automatic generation method is implemented by the control of the central processing unit.
As shown in fig. 11, the electronic device 9600 may further include: a communication module 9110, an input unit 9120, an audio processor 9130, a display 9160, and a power supply 9170. It is noted that the electronic device 9600 need not include all of the components shown in fig. 11; in addition, the electronic device 9600 may further include components not shown in fig. 11, and reference may be made to the related art.
As shown in fig. 11, the central processor 9100, sometimes referred to as a controller or operational control, may include a microprocessor or other processor device and/or logic device, which central processor 9100 receives inputs and controls the operation of the various components of the electronic device 9600.
The memory 9140 may be, for example, one or more of a buffer, a flash memory, a hard drive, a removable media, a volatile memory, a non-volatile memory, or other suitable device. The information about failure may be stored, and a program for executing the information may be stored. And the central processor 9100 can execute the program stored in the memory 9140 to realize information storage or processing, and the like.
The input unit 9120 provides input to the central processor 9100. The input unit 9120 is, for example, a key or a touch input device. The power supply 9170 is used to provide power to the electronic device 9600. The display 9160 is used for displaying display objects such as images and characters. The display may be, for example, but not limited to, an LCD display.
The memory 9140 may be a solid state memory such as Read Only Memory (ROM), random Access Memory (RAM), SIM card, etc. But also a memory which holds information even when powered down, can be selectively erased and provided with further data, an example of which is sometimes referred to as EPROM or the like. The memory 9140 may also be some other type of device. The memory 9140 includes a buffer memory 9141 (sometimes referred to as a buffer). The memory 9140 may include an application/function storage portion 9142, the application/function storage portion 9142 storing application programs and function programs or a flow for executing operations of the electronic device 9600 by the central processor 9100.
The memory 9140 may also include a data store 9143, the data store 9143 for storing data, such as contacts, digital data, pictures, sounds, and/or any other data used by an electronic device. The driver storage portion 9144 of the memory 9140 may include various drivers of the electronic device for communication functions and/or for performing other functions of the electronic device (e.g., messaging applications, address book applications, etc.).
The communication module 9110 is a transmitter/receiver 9110 that transmits and receives signals via an antenna 9111. The communication module (transmitter/receiver) 9110 is coupled to the central processor 9100 to provide an input signal and receive an output signal, which may be the same as in the case of a conventional mobile communication terminal.
Based on different communication technologies, a plurality of communication modules 9110, such as a cellular network module, a bluetooth module, and/or a wireless lan module, may be provided in the same electronic device. The communication module (transmitter/receiver) 9110 is also coupled to a speaker 9131 and a microphone 9132 via an audio processor 9130 to provide audio output via the speaker 9131 and to receive audio input from the microphone 9132 to implement usual telecommunications functions. The audio processor 9130 can include any suitable buffers, decoders, amplifiers and so forth. In addition, the audio processor 9130 is also coupled to the central processor 9100 so that sound can be recorded locally through the microphone 9132 and sound stored locally can be played through the speaker 9131.
The embodiments of the present application further provide a computer readable storage medium capable of implementing all the steps in the test case automatic generation method in which the execution subject is a server or a client in the above embodiments, where the computer readable storage medium stores a computer program, and when the computer program is executed by a processor, the computer program implements all the steps in the test case automatic generation method in which the execution subject is a server or a client in the above embodiments, for example, the processor implements the following steps when executing the computer program:
S101: obtaining a test case to be selected;
s102: inputting the test cases to be selected into a pre-constructed random forest case selection model to determine the current test cases according to the attributes of the selected test cases; the random forest case selection model includes a plurality of pre-generated decision tree classifiers.
From the above description, the test case automatic generation method provided by the application can automatically generate test cases by using a pre-constructed random forest case selection model, assist software testers in working, and perform black box/gray box test even if the software testers do not know the content of the program codes, and the screened test cases have strong relevance with the program code list and the risks of the cases themselves.
It will be apparent to those skilled in the art that embodiments of the present invention may be provided as a method, apparatus, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (devices), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
The principles and embodiments of the present invention have been described in detail with reference to specific examples, which are provided to facilitate understanding of the method and core ideas of the present invention; meanwhile, as those skilled in the art will have variations in the specific embodiments and application scope in accordance with the ideas of the present invention, the present description should not be construed as limiting the present invention in view of the above.
Claims (9)
1. A method for automatically generating test cases, comprising:
obtaining a test case to be selected;
inputting the test cases to be selected into a pre-constructed random forest case selection model to determine the current test cases according to the attributes of the selected test cases; the random forest case selection model comprises a plurality of pre-generated decision tree classifiers;
Wherein, the step of generating the decision tree classifier in advance comprises the following steps:
randomly dividing the acquired historical testing cases and historical non-testing cases into a training set and a testing set according to a preset dividing proportion;
determining information gain values of all attributes corresponding to the test cases to be selected in the training set;
constructing the decision tree classifier according to the magnitude of the information gain value; the decision tree classifier comprises attribute nodes corresponding to the attributes.
2. The method of automatic test case generation according to claim 1, further comprising, before inputting the test case to be selected into a pre-built random forest case selection model to determine the current test case to be tested according to the attributes of the selected test case:
acquiring a list of program reformation in the current period;
determining whether a to-be-selected test case to be input into the random forest case selection model is a test case corresponding to the program which is reformed in the current period according to the corresponding relation between the pre-established program and the case;
if so, the test case is determined to be the current test case option.
3. The method of claim 1, further comprising, after constructing the decision tree classifier based on the magnitude of the information gain value:
Pruning the decision tree classifier by using the test set.
4. The method of claim 3, wherein pruning the decision tree classifier with the test set comprises:
each attribute node in the decision tree classifier is removed in sequence, and the accuracy of the decision tree classifier after removing the attribute node is calculated according to the test set when determining the test case;
comparing the accuracy corresponding to the decision tree classifier after the attribute node is removed with the accuracy of the decision tree classifier before the attribute node is removed when the test case is determined;
pruning is carried out on the decision tree classifier according to the comparison result.
5. The method of claim 1, wherein the inputting the test case to be selected into a pre-built random forest case selection model to determine the current expected test case based on the properties of the selected test case comprises:
determining corresponding initial cases to be tested in the period according to the attribute of the selected test cases by utilizing each decision tree classifier;
voting is carried out on the initial case to be tested in each period, and the case to be tested in the period is determined according to the voting result.
6. An automatic test case generation device, comprising:
the case acquisition unit is used for acquiring a test case to be selected;
the case generation unit is used for inputting the to-be-selected test case into a pre-constructed random forest case selection model so as to determine a current period to-be-tested case according to the attribute of the selected test case; the random forest case selection model comprises a plurality of pre-generated decision tree classifiers;
the training set test set dividing unit is used for randomly dividing the acquired historical to-be-tested cases and the historical to-be-tested cases into a training set and a test set according to a preset dividing proportion;
the information gain determining unit is used for determining information gain values of all the attributes corresponding to the test cases to be selected in the training set;
the decision tree classifier construction unit is used for constructing the decision tree classifier according to the magnitude of the information gain value; the decision tree classifier comprises attribute nodes corresponding to the attributes.
7. The test case automatic generation device of claim 6, further comprising:
the reconstruction list acquisition unit is used for acquiring a list of the current program reconstruction;
the corresponding case determining unit is used for determining whether the to-be-selected test case to be input into the random forest case selection model is the test case corresponding to the program which is reformed in the current period according to the corresponding relation between the pre-established program and the case;
And the test case corresponding unit is used for determining the test case as the current test case.
8. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the processor implements the steps of the test case automatic generation method of any one of claims 1 to 5 when the program is executed by the processor.
9. A computer readable storage medium having stored thereon a computer program, characterized in that the computer program, when executed by a processor, implements the steps of the test case automatic generation method of any of claims 1 to 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110569902.7A CN113190460B (en) | 2021-05-25 | 2021-05-25 | Automatic test case generation method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110569902.7A CN113190460B (en) | 2021-05-25 | 2021-05-25 | Automatic test case generation method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113190460A CN113190460A (en) | 2021-07-30 |
| CN113190460B true CN113190460B (en) | 2024-03-15 |
Family
ID=76985154
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110569902.7A Active CN113190460B (en) | 2021-05-25 | 2021-05-25 | Automatic test case generation method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113190460B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113760621A (en) * | 2021-09-07 | 2021-12-07 | 四川中微芯成科技有限公司 | Method and device for testing touch chip, computer equipment and storage medium |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109977028A (en) * | 2019-04-08 | 2019-07-05 | 燕山大学 | A kind of Software Defects Predict Methods based on genetic algorithm and random forest |
| CN111046931A (en) * | 2019-12-02 | 2020-04-21 | 北京交通大学 | Turnout fault diagnosis method based on random forest |
| CN113157586A (en) * | 2021-04-30 | 2021-07-23 | 中国工商银行股份有限公司 | Financial market unit test case generation method and device |
-
2021
- 2021-05-25 CN CN202110569902.7A patent/CN113190460B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109977028A (en) * | 2019-04-08 | 2019-07-05 | 燕山大学 | A kind of Software Defects Predict Methods based on genetic algorithm and random forest |
| CN111046931A (en) * | 2019-12-02 | 2020-04-21 | 北京交通大学 | Turnout fault diagnosis method based on random forest |
| CN113157586A (en) * | 2021-04-30 | 2021-07-23 | 中国工商银行股份有限公司 | Financial market unit test case generation method and device |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113190460A (en) | 2021-07-30 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108681970A (en) | Finance product method for pushing, system and computer storage media based on big data | |
| CN111815169B (en) | Service approval parameter configuration method and device | |
| CN112784112B (en) | Message verification method and device | |
| CN109543154B (en) | Type conversion method and device of table data, storage medium and electronic equipment | |
| CN113408668A (en) | Decision tree construction method and device based on federated learning system and electronic equipment | |
| CN102426567A (en) | Graphical editing and debugging system of automatic answer system | |
| CN117951547B (en) | Bid and tendered data processing method and device based on artificial intelligence | |
| CN111353891A (en) | Auxiliary method and device for identifying suspicious groups in fund transaction data | |
| CN113190460B (en) | Automatic test case generation method and device | |
| CN112163861B (en) | Transaction risk factor feature extraction method and device | |
| CN110197426A (en) | A kind of method for building up of credit scoring model, device and readable storage medium storing program for executing | |
| US20210349920A1 (en) | Method and apparatus for outputting information | |
| CN114547658A (en) | Data processing method, device, equipment and computer readable storage medium | |
| CN109213782A (en) | Configuration, display methods, device and the communication equipment of search interface | |
| CN113568888A (en) | Index recommendation method and device | |
| CN116681513A (en) | Enterprise asset data processing method and device | |
| WO2020093817A1 (en) | Identity verification method and device | |
| CN113033209B (en) | Text relation extraction method and device, storage medium and computer equipment | |
| CN113515447B (en) | Automatic testing method and device for system | |
| CN112799929B (en) | Root cause analysis method and system of alarm log | |
| CN114625894A (en) | Appreciation evaluation method, model training method, appreciation evaluation apparatus, model training medium, and computing apparatus | |
| CN114996434B (en) | Information extraction method and device, storage medium and computer equipment | |
| CN118096358B (en) | Intelligent enterprise credit report generation method and device | |
| US11934396B2 (en) | Data reconciliation for big data environments | |
| CN117453273A (en) | Intelligent program code complement method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |