CN112948274A - Test case scoring model training method and test case selection method - Google Patents
Test case scoring model training method and test case selection method Download PDFInfo
- Publication number
- CN112948274A CN112948274A CN202110432705.0A CN202110432705A CN112948274A CN 112948274 A CN112948274 A CN 112948274A CN 202110432705 A CN202110432705 A CN 202110432705A CN 112948274 A CN112948274 A CN 112948274A
- Authority
- CN
- China
- Prior art keywords
- test case
- discovery rate
- test
- test cases
- defect discovery
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/36—Preventing errors by testing or debugging software
- G06F11/3668—Software testing
- G06F11/3672—Test management
- G06F11/3684—Test management for test design, e.g. generating new test cases
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Computer Hardware Design (AREA)
- Quality & Reliability (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Stored Programmes (AREA)
Abstract
The invention discloses a test case scoring model training method and a test case selection method, wherein the test case scoring model training method comprises the following steps: determining a plurality of historical test cases for software testing; generating a training sample according to the historical test case, wherein the training sample comprises the characteristic information of the historical test case and the defect discovery rate of the historical test case; and performing model training based on horizontal federal learning by combining other participants according to the training sample to obtain a finally obtained test case scoring model, wherein the finally obtained test case scoring model is used for predicting the corresponding defect discovery rate according to the characteristic information of the multiple candidate test cases so as to select an actual test case from the multiple candidate test cases according to the predicted defect discovery rate. The model is trained based on a large amount of sample data by combining a plurality of participants, so that the model can be used for predicting the defect discovery rate of the test case, providing effective reference for selection of the test case, improving the selection efficiency of the test case and improving the software testing efficiency.
Description
Technical Field
The invention relates to the field of computers, in particular to a test case scoring model training method and a test case selecting method.
Background
The popularization of intelligent equipment, the types and the quantity of software carried by the equipment are more and more. Before software is released, functions of the software need to be tested, and test cases are used. The Test Case (Test Case) refers to the description of a Test task performed on a specific software product, and embodies Test schemes, methods, techniques and strategies. The contents of the test object, the test environment, the input data, the test steps, the expected results, the test scripts and the like are included, and finally, a document is formed.
As software functions increase, more and more test cases are required. Limited by the release time, there may not be enough time to execute all use cases. This requires that a portion of the test cases be picked out of the many test cases for execution in order to perform software testing as efficiently as possible.
For a project without enough test experience, the selection of the test case can be performed only by the experience of the tester. The selected test cases are not necessarily suitable for the current project, so that the test case selection efficiency is low, and the software test efficiency is low.
Disclosure of Invention
The invention mainly aims to provide a test case scoring model training method and a test case selection method, so that a test case scoring model is trained based on a large amount of test data, automatic selection of test cases is realized, and test case selection efficiency and software test efficiency are improved.
In order to achieve the above object, in a first aspect, the present invention provides a method for training a scoring model of a test case, including:
determining a plurality of historical test cases for software testing;
generating a training sample according to the historical test case, wherein the training sample comprises the characteristic information of the historical test case and the defect discovery rate of the historical test case;
and performing model training based on horizontal federal learning by combining other participants according to the training sample to obtain a finally obtained test case scoring model, wherein the finally obtained test case scoring model is used for predicting the corresponding defect discovery rate according to the characteristic information of the multiple candidate test cases so as to select an actual test case from the multiple candidate test cases according to the predicted defect discovery rate.
Optionally, the performing model training based on horizontal federal learning by combining other participants according to the training samples includes:
acquiring initial test case scoring model parameters from a server;
performing model training by using the training sample to obtain a loss function gradient;
sending the loss function gradient to a server so that the server can aggregate and update the loss function gradient sent by each participant;
receiving an updated loss function gradient sent by a server;
and continuing model training based on the updated loss function gradient, and iterating until the loss function is converged to obtain a trained test case evaluation model.
Optionally, the characteristics of the historical test case include: the method comprises the following steps of software form, software service, case type, module to which the case belongs, case state, whether the operation step relates to an input box, whether the operation step relates to a button, whether the operation step relates to zooming, whether a verification point comprises page display, whether the verification point comprises database verification and whether the verification point comprises log verification.
In a second aspect, the present invention provides a test case selection method, including:
obtaining a plurality of alternative test cases;
predicting the defect discovery rate of the alternative test cases based on the trained test case scoring model; the test case scoring model is trained by using the test case scoring model training method according to the first aspect;
and selecting an actual test case from the multiple candidate test cases according to the predicted defect discovery rate.
Optionally, the selecting an actual test case from a plurality of candidate test cases according to the predicted defect discovery rate includes:
sequencing a plurality of candidate test cases from high to low according to the defect discovery rate;
and selecting N candidate test cases with the highest defect discovery rate as actual test cases.
Optionally, the selecting an actual test case from a plurality of candidate test cases according to the predicted defect discovery rate includes:
setting weights for the defect discovery rate, the importance and the execution time consumption for each test case;
determining the defect discovery rate, the importance and the weighted sum value of the execution time;
sorting the multiple alternative test cases from large to small according to the weighted sum value;
and selecting the N candidate test cases with the maximum weighted sum as actual test cases.
In a third aspect, the present invention provides a training apparatus for a test case scoring model, including:
the historical test case determining module is used for determining a plurality of historical test cases for software testing;
the training sample generating module is used for generating a training sample according to the historical test case, wherein the training sample comprises the characteristic information of the historical test case and the defect discovery rate of the historical test case;
and the training module is used for performing model training based on transverse federal learning by combining other participants according to the training samples to finally obtain a test case scoring model, and is used for predicting the corresponding defect discovery rate according to the characteristic information of the multiple candidate test cases so as to select an actual test case from the multiple candidate test cases according to the predicted defect discovery rate.
Optionally, the training module is specifically configured to:
acquiring initial test case scoring model parameters from a server;
performing model training by using the training sample to obtain a loss function gradient;
sending the loss function gradient to a server so that the server can aggregate and update the loss function gradient sent by each participant;
receiving an updated loss function gradient sent by a server;
and continuing model training based on the updated loss function gradient, and iterating until the loss function is converged to obtain a trained test case evaluation model.
Optionally, the characteristics of the historical test case include: the method comprises the following steps of software form, software service, case type, module to which the case belongs, case state, whether the operation step relates to an input box, whether the operation step relates to a button, whether the operation step relates to zooming, whether a verification point comprises page display, whether the verification point comprises database verification and whether the verification point comprises log verification.
In a fourth aspect, the present invention provides a test case selection apparatus, including:
the acquisition module is used for acquiring a plurality of alternative test cases;
the prediction module is used for predicting the defect discovery rate of the alternative test cases based on the trained test case scoring model; the test case scoring model is trained by using the test case scoring model training method according to the first aspect;
and the selection module is used for selecting an actual test case from the multiple candidate test cases according to the predicted defect discovery rate.
Optionally, the selection module is specifically configured to:
sequencing a plurality of candidate test cases from high to low according to the defect discovery rate;
and selecting N candidate test cases with the highest defect discovery rate as actual test cases.
Optionally, the selection module is specifically configured to:
setting weights for the defect discovery rate, the importance and the execution time consumption for each test case;
determining the defect discovery rate, the importance and the weighted sum value of the execution time;
sorting the multiple alternative test cases from large to small according to the weighted sum value;
and selecting the N candidate test cases with the maximum weighted sum as actual test cases.
In a fifth aspect, the present invention provides an electronic device, comprising: a memory, a processor and a program stored on the memory and executable on the processor, which when executed by the processor, performs the steps of the method of the first and/or second aspect.
In a sixth aspect, the present invention provides a computer readable storage medium having a program stored thereon, which when executed by a processor, performs the steps of the method of the first and/or second aspect.
In a seventh aspect, the present invention provides a computer program product comprising a computer program that, when executed by a processor, implements the method of the first and/or second aspect.
The invention provides a test case scoring model training method and a test case selection method, wherein the test case scoring model training method comprises the following steps: determining a plurality of historical test cases for software testing; generating a training sample according to the historical test case, wherein the training sample comprises the characteristic information of the historical test case and the defect discovery rate of the historical test case; and performing model training based on horizontal federal learning by combining other participants according to the training sample to obtain a finally obtained test case scoring model, wherein the finally obtained test case scoring model is used for predicting the corresponding defect discovery rate according to the characteristic information of the multiple candidate test cases so as to select an actual test case from the multiple candidate test cases according to the predicted defect discovery rate. The scheme of the invention realizes the training of the model based on a large amount of sample data by combining a plurality of participants, can be used for predicting the defect discovery rate of the test case, can reflect the value of the test case by the predicted defect discovery rate, provides effective reference for the selection of the test case, improves the selection efficiency of the test case and further improves the software testing efficiency. Meanwhile, the data privacy of each participant can be effectively ensured based on the model training of the horizontal federal learning.
Drawings
Fig. 1 is a schematic diagram of an application scenario provided in an embodiment of the present invention;
FIG. 2 is a flowchart of a method for training a scoring model according to a test case provided in an embodiment of the present invention;
FIG. 3 is a flowchart of a test case selection method according to an embodiment of the present invention;
FIG. 4 is a schematic structural diagram of a scoring model training apparatus for a test example according to an embodiment of the present invention;
fig. 5 is a schematic structural diagram of a test case selection apparatus according to an embodiment of the present invention;
fig. 6 is a schematic structural diagram of an electronic device according to an embodiment of the present invention.
Detailed Description
Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be embodied in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.
In the software testing process, a plurality of test cases are generally required to be used for testing so as to test the completeness of software functions as much as possible, and the problem of finding out the software functions or logics is solved.
As software functions become more and more, more test cases are required. However, in some cases, the release time may be limited to allow only a certain amount of test cases to be executed. It is desirable to select test cases that can test the performance of software as efficiently as possible.
Among the indexes of test performance of test cases, the defect discovery rate is a relatively important index. The value of a test case can be represented to a certain extent by the probability of finding a defect in software by executing the test case.
If the test case is selected by using the index, a quick and accurate prediction of the defect discovery rate of each test case needs to be provided. However, currently, the prediction of the defect discovery rate is judged by the experience of the staff, the prediction accuracy is low, and the inheritance cannot be maintained.
Based on the above, the invention provides a test case scoring model training method, which is used for constructing and training a model and realizing accurate and efficient prediction of defect discovery rate. Therefore, the test cases can be selected based on the predicted defect discovery rate, and efficient software testing is further realized.
Meanwhile, some new projects may not have enough historical data to serve as a reference, and in order to enhance the applicability of the model, a federal learning training mode is introduced. In the model training process, multiple parties are combined for training. Each party provides a certain amount of historical data, so that training samples can be expanded, and the prediction accuracy of the model is improved.
In many fields, technologies such as machine learning and model training based on big data are continuously developed. By mining large data, a lot of valuable information can be obtained. With the development of technology, the source of raw data is more and more extensive, and even cross-domain union can be achieved.
In some business scenarios, multiple business platforms collect respective business data. The service data is valuable and is usually kept as the privacy information of the service platform. Each service platform does not want to share the original form of service data with other service platforms. In some needs, however, multiple business platforms desire to perform collaborative computing without exposing business data of the parties to improve business processing capabilities. Based on the above, the federal machine learning and other modes are carried out.
Federal Machine Learning (also called federal Learning), which can combine all parties to perform data use and collaborative modeling on the premise that data is not out of the local, is gradually a common method in privacy protection calculation.
For different data sets, federal Learning can be divided into Horizontal federal Learning (Horizontal federal Learning), Vertical federal Learning (Vertical federal Learning), and federal Transfer Learning (federal transferred Learning).
Wherein the horizontal federated learning may be applied to scenarios where user features of different data sets overlap more and users overlap less. For example, one bank in place a and the other bank in place B are different in location, the user overlap is less, but the data of the two banks relate to the balance and expenditure behavior, the asset data and the like of the user, and then the data collaborative calculation between the two banks can be realized based on the horizontal federal learning.
Longitudinal federated learning may be applied to scenarios where users of different data sets overlap more and user features overlap less. For example, there are two different entities in a participant, one being a bank at a location and the other being an e-commerce at that location. The intersection of users is large because the user groups possibly contain most residents of the site, but the intersection of user features is small because the data of banks relates to the balance and expenditure behaviors and the asset data of users and the data of electric companies relates to the browsing and purchasing histories of the users, and the data collaborative calculation between the banks and the electric companies can be realized based on longitudinal federal learning.
Federal transfer learning may be applied to scenarios where users of different data sets overlap less with user features. For example, there are two different organizations in the participants, one is the bank in place a, and the other is the e-commerce in place B, the user population intersection of the two participants is very small, and the degree of coincidence of the user features is also very small. In this case, it is necessary to introduce migration learning to improve the effect of the cooperative computing.
There are many machine learning algorithms that may be used for federated learning, including, but not limited to, neural networks, random forests, and the like.
In the invention, the combination of multi-party historical data relates to the data such as the characteristics of test cases, the defect discovery rate and the like, so that the method accords with the characteristics of horizontal federal learning and belongs to the scope of horizontal federal learning.
Fig. 1 is a schematic diagram of an application scenario provided in an embodiment of the present invention. As shown in fig. 1, a project (corresponding to client terminal 1) needs to test newly developed software, and the model is trained jointly with other projects (corresponding to client terminals 2, … …, k). Each client terminal conducts model training based on local training samples and exchanges model parameters with the server. And finally obtaining the test case evaluation model.
Specific implementations can be found in the following examples.
Fig. 2 is a flowchart of a test case scoring model training method according to an embodiment of the present invention. The method of this embodiment may be applied to each participant participating in model training, as shown in fig. 2, the method of this embodiment may specifically include:
s201, determining a plurality of historical test cases for software testing.
Typically, conducting software testing will use many test cases and have associated test data. The test cases used in the past software testing process are called historical test cases. Some or all of which may be selected.
S202, generating a training sample according to the historical test case, wherein the training sample comprises the characteristic information of the historical test case and the defect discovery rate of the historical test case.
And carrying out feature labeling on each test case, determining the corresponding defect discovery rate, and forming a data set of the training sample.
The defect discovery rate of a certain test case refers to the proportion of all problems discovered in the software testing process by using the software problems tested by the test case.
And S203, performing model training based on horizontal federal learning by combining other participants according to the training samples to obtain a finally obtained test case scoring model, wherein the finally obtained test case scoring model is used for predicting the corresponding defect discovery rate according to the characteristic information of the multiple candidate test cases so as to select an actual test case from the multiple candidate test cases according to the predicted defect discovery rate.
The training samples are generated by other participants by adopting the steps of S201 and S202, and the participants perform horizontal federal learning together to complete model training. Each participant may store the model parameters of the trained test case scoring model.
The test case scoring model can be used for selecting test cases, specifically, the test case scoring model can be used for predicting the defect discovery rate of the alternative test cases according to the characteristics of the alternative test cases, and then part of the alternative test cases are selected as actual test cases according to the predicted defect discovery rate to perform subsequent software tests.
The test case scoring model training method provided by the embodiment comprises the following steps: determining a plurality of historical test cases for software testing; generating a training sample according to the historical test case, wherein the training sample comprises the characteristic information of the historical test case and the defect discovery rate of the historical test case; and performing model training based on horizontal federal learning by combining other participants according to the training samples to obtain a finally obtained test case scoring model, wherein the finally obtained test case scoring model is used for predicting the corresponding defect discovery rate according to the characteristic information of the multiple candidate test cases so as to select an actual test case from the multiple candidate test cases according to the predicted defect discovery rate. The model is trained based on a large amount of sample data by combining a plurality of participants, the model can be used for predicting the defect discovery rate of the test case, the predicted defect discovery rate can reflect the value of the test case, effective reference is provided for the selection of the test case, the test case selection efficiency is improved, and the software test efficiency is further improved. Meanwhile, the data privacy of each participant can be effectively ensured based on the model training of the horizontal federal learning.
In some embodiments, the performing model training based on horizontal federal learning by using the training samples and combining with other participants may specifically include: acquiring initial test case scoring model parameters from a server; carrying out model training by using a training sample to obtain a loss function gradient; sending the loss function gradient to a server so that the server can aggregate and update the loss function gradient sent by each participant; receiving an updated loss function gradient sent by a server; and continuing model training based on the updated loss function gradient, and iterating until the loss function is converged to obtain the trained test case scoring model.
Still referring to fig. 1, the server and k client terminals together perform a model training process for horizontal federal learning. The server issues a global model to each client terminal, each client terminal trains the global model issued by the server by using local data to obtain model parameters after training and uploads the model parameters to the server, the server aggregates the model parameters uploaded by each client terminal to obtain updated global model parameters, the processes are sequentially repeated, and iteration is performed until the aggregated global model converges.
Specifically, the model parameters may include a loss function, a gradient of the loss function, and the like.
Assuming that the t-th iteration is currently performed, the specific procedure of training is as follows:
step 1, the server issues the current model parameters such as model weight, loss function gradient and the like to each client terminal Ci。
Step 2, client terminal CiAnd performing local training based on the local training sample by using algorithms such as logistic regression, linear regression and the like in combination with model parameters such as model weight and the like, recalculating parameters such as loss functions, loss function gradients and the like, and uploading the parameters such as the loss functions, the loss function gradients and the like to the server.
And 3, the server receives parameters such as loss functions, loss function gradients and the like uploaded by all the client terminals, and carries out aggregation and updating. Meanwhile, whether the loss function is converged can be judged according to the gradient state of the loss function, and if not, the steps are continuously and repeatedly executed.
In addition, an iteration number value can be preset before the model training is started, and when the iteration number reaches the preset value, the model can be determined to be in a convergence state.
In order to ensure the security of data, the client terminal and the server may encrypt the interactive data during the interaction process. The private data of the participants are protected in a parameter exchange mode under an encryption mechanism, the data can not be transmitted, the participants do not need to expose own data to other participants, the data of other participants can not be reversely deduced, and the privacy of users can be well protected and the data safety can be guaranteed.
In some embodiments, the characteristics of the historical test case may include: software form, software service, case type, module to which the case belongs, case state, whether the operation step relates to an input box, whether the operation step relates to a button, whether the operation step relates to zooming, whether the verification point comprises page display, whether the verification point comprises database verification, whether the verification point comprises log verification and the like.
The parameters corresponding to the software morphological characteristics can be web, app, api and the like; the parameters corresponding to the software service characteristics can be shopping malls, entertainment, tools and the like; parameters corresponding to the case type characteristics can be functions, performances, UIs and the like; parameters corresponding to the module characteristics can be login, home page, personal center and the like; the parameters corresponding to the case state characteristics can be normal or abnormal; whether the operation step involves inputting the corresponding parameter of the box characteristic can be yes or no; whether the operation step involves parameters corresponding to the button features may be yes or no; whether the operation step involves a parameter corresponding to the zoom feature may be yes or no; whether the verification point comprises the parameter corresponding to the page display characteristic can be yes or no; whether the verification point comprises the parameter corresponding to the database verification feature or not can be yes or no; whether the verification point includes the parameter corresponding to the log verification feature may be yes or no.
It should be noted that the above features are merely examples. Features that can be labeled in actual application may include one or more of the above-mentioned features, and may also include other features not mentioned.
By marking the characteristics of the test case in detail, the result of model training can be closer to the real corresponding relation between the characteristics and the defect discovery rate, so that the prediction of the defect discovery rate based on the characteristics is more accurate.
Fig. 3 is a flowchart of a test case selection method according to an embodiment of the present invention. The method of this embodiment may be applied to each participant participating in model training, and as shown in fig. 3, the method of this embodiment may specifically include:
s301, obtaining a plurality of alternative test cases.
The alternative test cases refer to test cases which can be used for software testing, and the test cases which are actually used for the software testing can be selected from the alternative test cases.
The related data of the alternative test case can be obtained from the designated storage position.
And S302, predicting the defect discovery rate of the alternative test cases based on the trained test case scoring model.
The test case scoring model is trained by using the test case scoring model training method of the embodiment.
And for the alternative test case, marking corresponding characteristics, inputting the test force evaluation model, and outputting a prediction result of the defect discovery rate of the alternative test case.
The labeled test case characteristics can be seen in the following table:
| example numbering | 11001 | 11002 |
| Software morphological classification | Web | Web |
| Software traffic classification | App | App |
| Type of use case | Function(s) | UI |
| …… | …… | …… |
And S303, selecting an actual test case from the multiple candidate test cases according to the predicted defect discovery rate.
The predicted defect discovery rate can reflect the value of the corresponding test case to the software test to a certain extent. And selecting an actual test case from a plurality of candidate test cases according to the predicted value of the defect discovery rate.
For example, a test case with a defect discovery rate higher than a preset value may be selected as an actual test case, and a subsequent software test may be performed.
In the embodiment, the defect discovery rate is predicted by using the transverse federal learning model, so that the method is more reliable compared with a manual judgment mode, and is wider and higher in accuracy compared with sample data referred to by a model obtained by training in other modes. The value of the test case selected by the method of the embodiment is higher, and the efficiency of software testing is further higher.
In some embodiments, the selecting an actual test case from a plurality of candidate test cases according to the predicted defect discovery rate may specifically include: sequencing a plurality of candidate test cases from high to low according to the defect discovery rate; and selecting N candidate test cases with the highest defect discovery rate as actual test cases.
And selecting a plurality of candidate test cases with higher defect discovery rate as actual test cases by taking the defect discovery rate as a test case selection standard. Therefore, the problems possibly existing in the software can be tested as much as possible in a short testing time, and high testing efficiency is obtained.
In other embodiments, the above manner of selecting an actual test case from a plurality of candidate test cases according to the predicted defect discovery rate may further be: setting weights for the defect discovery rate, the importance and the execution time consumption for each test case; determining the defect discovery rate, the importance and the weighted sum value of the execution time; sequencing a plurality of alternative test cases from large to small according to the weighted sum value; and selecting the N candidate test cases with the maximum weighted sum as actual test cases.
Wherein, the importance refers to the importance classification of the test performance of a certain test case in the software function; the execution time refers to the time length required for executing a certain test case. In addition to these two metrics, other performance metrics may be added.
And comprehensively considering multiple indexes such as defect discovery rate, importance, execution time consumption and the like, respectively setting weights for the indexes, taking weighted summation as a selection standard, and selecting a plurality of candidate test cases with higher weighted sum values as actual test cases. Therefore, the value of each test case can be evaluated more comprehensively, the problems possibly existing in the software can be tested as comprehensively as possible, and higher test efficiency is obtained.
In addition, after the software test is completed by executing the actual test case, the case and the actual defect discovery rate can be added into the sample data set so as to update the model.
The device for executing the model training and the device for selecting the test case may be the same device or different devices.
Fig. 4 is a schematic structural diagram of a test case scoring model training device according to an embodiment of the present invention, and as shown in fig. 4, the test case scoring model training device 400 according to the embodiment includes: a historical test case determining module 401, a training sample generating module 402 and a training module 403.
A historical test case determining module 401, configured to determine a plurality of historical test cases for software testing;
a training sample generating module 402, configured to generate a training sample according to a historical test case, where the training sample includes feature information of the historical test case and a defect discovery rate of the historical test case;
the training module 403 is configured to perform model training based on horizontal federal learning according to the training samples in combination with other participants, and obtain a test case scoring model finally, and is configured to predict a corresponding defect discovery rate according to feature information of a plurality of candidate test cases, so as to select an actual test case from the plurality of candidate test cases according to the predicted defect discovery rate.
Optionally, the training module 403 is specifically configured to:
acquiring initial test case scoring model parameters from a server;
carrying out model training by using a training sample to obtain new model parameters;
sending the new model parameters to a server so that the server can aggregate and update the new model parameters sent by each participant;
receiving updated model parameters sent by a server;
and continuing model training based on the updated model parameters, and iterating until the new model converges to obtain the trained test case scoring model.
Optionally, the characteristics of the historical test case include: the method comprises the following steps of software form, software service, case type, module to which the case belongs, case state, whether the operation step relates to an input box, whether the operation step relates to a button, whether the operation step relates to zooming, whether a verification point comprises page display, whether the verification point comprises database verification and whether the verification point comprises log verification.
The device provided in this embodiment can perform the test case scoring model training method in the above embodiments to achieve the same technical effects, and will not be described herein again.
Fig. 5 is a schematic structural diagram of a test case selection apparatus according to an embodiment of the present invention, and as shown in fig. 5, the test case selection apparatus 500 according to the embodiment includes: an obtaining module 501, a predicting module 502 and a selecting module 503.
An obtaining module 501, configured to obtain multiple candidate test cases;
the prediction module 502 is used for predicting the defect discovery rate of the alternative test cases based on the trained test case scoring model; the test case scoring model is trained by using the test case scoring model training method in the embodiment;
and a selecting module 503, configured to select an actual test case from the multiple candidate test cases according to the predicted defect discovery rate.
Optionally, the selecting module 503 is specifically configured to:
sequencing a plurality of candidate test cases from high to low according to the defect discovery rate;
and selecting N candidate test cases with the highest defect discovery rate as actual test cases.
Optionally, the selecting module 503 is specifically configured to:
setting weights for the defect discovery rate, the importance and the execution time consumption for each test case;
determining the defect discovery rate, the importance and the weighted sum value of the execution time;
sequencing a plurality of alternative test cases from large to small according to the weighted sum value;
and selecting the N candidate test cases with the maximum weighted sum as actual test cases.
The device provided in this embodiment can execute the test case selection method in the above embodiments to achieve the same technical effects, and details are not described here.
Fig. 6 is a schematic structural diagram of an electronic device according to an embodiment of the present invention, and as shown in fig. 6, an electronic device 600 according to the embodiment includes: the memory 601, the processor 602, and the program stored in the memory 601 and capable of running on the processor 602 may implement the test case scoring model training method in the foregoing embodiment and/or the steps of the test case selecting method in the foregoing embodiment when the program is executed by the processor 602, so as to achieve the same technical effects as the foregoing embodiment, and details are not described here.
The present invention further provides a computer-readable storage medium, in which a program is stored, and when the program is executed by a processor, the steps of the test case scoring model training method in the above embodiment and/or the test case selecting method in the above embodiment are implemented.
The present invention also provides a computer program product, which includes a computer program, and when the computer program is executed by a processor, the computer program implements the test case scoring model training method in the above embodiment and/or the test case selecting method in the above embodiment.
In the embodiments provided in the present invention, it should be understood that the disclosed apparatus and method may be implemented in other ways. For example, the above-described embodiments of the apparatus are merely illustrative, and for example, the division of modules is only one logical division, and other divisions may be realized in practice, for example, a plurality of modules may be combined or integrated into another system, or some features may be omitted, or not executed.
The integrated module implemented in the form of a software functional module may be stored in a computer-readable storage medium. The software functional module is stored in a storage medium and includes several instructions to make a computer device (which may be a personal computer, a server, or a network device) or a processor execute part of the steps of the methods according to the embodiments of the present invention.
It should be understood that the Processor may be a Central Processing Unit (CPU), other general purpose Processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), etc. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like. The steps of a method disclosed in connection with the present invention may be embodied directly in a hardware processor, or in a combination of the hardware and software modules within the processor.
The memory may comprise a high-speed RAM memory, and may further comprise a non-volatile storage NVM, such as at least one disk memory, and may also be a usb disk, a removable hard disk, a read-only memory, a magnetic or optical disk, etc.
The storage medium may be implemented by any type or combination of volatile or non-volatile memory devices, such as Static Random Access Memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic or optical disks. A storage media may be any available media that can be accessed by a general purpose or special purpose computer.
An exemplary storage medium is coupled to the processor such the processor can read information from, and write information to, the storage medium. Of course, the storage medium may also be integral to the processor. The processor and the storage medium may reside in an Application Specific Integrated Circuits (ASIC). Of course, the processor and the storage medium may reside as discrete components in an electronic device or host device.
It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.
The above-mentioned serial numbers of the embodiments of the present invention are merely for description and do not represent the merits of the embodiments.
Through the above description of the embodiments, those skilled in the art will clearly understand that the method of the above embodiments can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware, but in many cases, the former is a better implementation manner. Based on such understanding, the technical solutions of the present invention may be embodied in the form of a software product, which is stored in a storage medium (such as ROM/RAM, magnetic disk, optical disk) and includes instructions for enabling a terminal device (such as a mobile phone, a computer, a server, an air conditioner, or a network device) to execute the method according to the embodiments of the present invention.
The above description is only a preferred embodiment of the present invention, and not intended to limit the scope of the present invention, and all modifications of equivalent structures and equivalent processes, which are made by using the contents of the present specification and the accompanying drawings, or directly or indirectly applied to other related technical fields, are included in the scope of the present invention.
Claims (11)
1. A test case scoring model training method is characterized by comprising the following steps:
determining a plurality of historical test cases for software testing;
generating a training sample according to the historical test case, wherein the training sample comprises the characteristic information of the historical test case and the defect discovery rate of the historical test case;
and performing model training based on horizontal federal learning by combining other participants according to the training sample to obtain a finally obtained test case scoring model, wherein the finally obtained test case scoring model is used for predicting the corresponding defect discovery rate according to the characteristic information of the multiple candidate test cases so as to select an actual test case from the multiple candidate test cases according to the predicted defect discovery rate.
2. The method of claim 1, wherein the model training based on horizontal federal learning in conjunction with other participants according to the training samples comprises:
acquiring initial test case scoring model parameters from a server;
performing model training by using the training sample to obtain a loss function gradient;
sending the loss function gradient to a server so that the server can aggregate and update the loss function gradient sent by each participant;
receiving an updated loss function gradient sent by a server;
and continuing model training based on the updated loss function gradient, and iterating until the loss function is converged to obtain a trained test case evaluation model.
3. The method of claim 1 or 2, wherein the characteristics of the historical test cases comprise: the method comprises the following steps of software form, software service, case type, module to which the case belongs, case state, whether the operation step relates to an input box, whether the operation step relates to a button, whether the operation step relates to zooming, whether a verification point comprises page display, whether the verification point comprises database verification and whether the verification point comprises log verification.
4. A test case selection method, comprising:
obtaining a plurality of alternative test cases;
predicting the defect discovery rate of the alternative test cases based on the trained test case scoring model; the test case scoring model is trained using the test case scoring model training method according to any one of claims 1 to 3;
and selecting an actual test case from the multiple candidate test cases according to the predicted defect discovery rate.
5. The method of claim 4, wherein selecting an actual test case from the plurality of candidate test cases according to the predicted defect discovery rate comprises:
sequencing a plurality of candidate test cases from high to low according to the defect discovery rate;
and selecting N candidate test cases with the highest defect discovery rate as actual test cases.
6. The method of claim 4, wherein selecting an actual test case from the plurality of candidate test cases according to the predicted defect discovery rate comprises:
setting weights for the defect discovery rate, the importance and the execution time consumption for each test case;
determining the defect discovery rate, the importance and the weighted sum value of the execution time;
sorting the multiple alternative test cases from large to small according to the weighted sum value;
and selecting the N candidate test cases with the maximum weighted sum as actual test cases.
7. A test case scoring model training device is characterized by comprising:
the historical test case determining module is used for determining a plurality of historical test cases for software testing;
the training sample generating module is used for generating a training sample according to the historical test case, wherein the training sample comprises the characteristic information of the historical test case and the defect discovery rate of the historical test case;
and the training module is used for performing model training based on transverse federal learning by combining other participants according to the training samples to finally obtain a test case scoring model, and is used for predicting the corresponding defect discovery rate according to the characteristic information of the multiple candidate test cases so as to select an actual test case from the multiple candidate test cases according to the predicted defect discovery rate.
8. A test case selection apparatus, comprising:
the acquisition module is used for acquiring a plurality of alternative test cases;
the prediction module is used for predicting the defect discovery rate of the alternative test cases based on the trained test case scoring model; the test case scoring model is trained using the test case scoring model training method according to any one of claims 1 to 3;
and the selection module is used for selecting an actual test case from the multiple candidate test cases according to the predicted defect discovery rate.
9. An electronic device, characterized in that the electronic device comprises: memory, processor and program stored on the memory and executable on the processor, which when executed by the processor implements the steps of the method according to any one of claims 1 to 6.
10. A computer-readable storage medium, characterized in that the computer-readable storage medium has stored thereon a program which, when being executed by a processor, carries out the steps of the method according to any one of claims 1 to 6.
11. A computer program product comprising a computer program, characterized in that the computer program realizes the method of any of claims 1 to 6 when executed by a processor.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110432705.0A CN112948274A (en) | 2021-04-21 | 2021-04-21 | Test case scoring model training method and test case selection method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110432705.0A CN112948274A (en) | 2021-04-21 | 2021-04-21 | Test case scoring model training method and test case selection method |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112948274A true CN112948274A (en) | 2021-06-11 |
Family
ID=76233138
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110432705.0A Pending CN112948274A (en) | 2021-04-21 | 2021-04-21 | Test case scoring model training method and test case selection method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112948274A (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113672506A (en) * | 2021-08-06 | 2021-11-19 | 中国科学院软件研究所 | Dynamic proportion test case sequencing selection method and system based on machine learning |
| CN113778875A (en) * | 2021-09-09 | 2021-12-10 | 上海浦东发展银行股份有限公司 | System test defect classification method, device, equipment and storage medium |
| CN113837368A (en) * | 2021-09-27 | 2021-12-24 | 中国太平洋保险（集团）股份有限公司 | Control method and device for evaluating data value of each participant in federal learning |
| CN116228043A (en) * | 2023-05-05 | 2023-06-06 | 苏州德耐纳米科技有限公司 | Coating uniformity evaluation system of coating machine |
-
2021
- 2021-04-21 CN CN202110432705.0A patent/CN112948274A/en active Pending
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113672506A (en) * | 2021-08-06 | 2021-11-19 | 中国科学院软件研究所 | Dynamic proportion test case sequencing selection method and system based on machine learning |
| CN113778875A (en) * | 2021-09-09 | 2021-12-10 | 上海浦东发展银行股份有限公司 | System test defect classification method, device, equipment and storage medium |
| CN113778875B (en) * | 2021-09-09 | 2024-04-30 | 上海浦东发展银行股份有限公司 | System test defect classification method, device, equipment and storage medium |
| CN113837368A (en) * | 2021-09-27 | 2021-12-24 | 中国太平洋保险（集团）股份有限公司 | Control method and device for evaluating data value of each participant in federal learning |
| CN116228043A (en) * | 2023-05-05 | 2023-06-06 | 苏州德耐纳米科技有限公司 | Coating uniformity evaluation system of coating machine |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112948274A (en) | Test case scoring model training method and test case selection method | |
| CN109922032B (en) | Method, device, equipment and storage medium for determining risk of logging in account | |
| CN111160749B (en) | Information quality assessment and information fusion method and device | |
| CN111177473B (en) | Personnel relationship analysis method, device and readable storage medium | |
| CN103678672A (en) | Method for recommending information | |
| CN112039702B (en) | Model parameter training method and device based on federal learning and mutual learning | |
| CN113240505B (en) | Method, apparatus, device, storage medium and program product for processing graph data | |
| CN111124917B (en) | Method, device, equipment and storage medium for managing and controlling public test cases | |
| CN112035549B (en) | Data mining method, device, computer equipment and storage medium | |
| CN108932646B (en) | User tag verification method and device based on operator and electronic equipment | |
| CN111148018B (en) | Method and device for identifying and positioning regional value based on communication data | |
| CN110837577A (en) | Video recommendation method, device, equipment and storage medium | |
| CN112818162A (en) | Image retrieval method, image retrieval device, storage medium and electronic equipment | |
| CN111324370A (en) | Method and device for carrying out risk processing on to-be-on-line small program | |
| CN114818000A (en) | Privacy protection set confusion intersection method, system and related equipment | |
| CN113935788B (en) | Model evaluation method, device, equipment and computer readable storage medium | |
| CN117035082A (en) | Knowledge graph real-time operation method and system for identifying fraudulent party | |
| US20160342899A1 (en) | Collaborative filtering in directed graph | |
| CN111008873A (en) | User determination method and device, electronic equipment and storage medium | |
| CN112529102B (en) | Feature expansion method, device, medium and computer program product | |
| CN109003181A (en) | Suspicious user determination method, device, equipment and computer readable storage medium | |
| CN110414260B (en) | Data access method, device, system and storage medium | |
| CN114880637B (en) | Account risk verification method and device, computer equipment and storage medium | |
| CN116701896A (en) | Image tag determining method, image tag determining device, computer device, and storage medium | |
| CN115758271A (en) | Data processing method, data processing device, computer equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |