CN110347600A - Convolutional neural network-oriented variation coverage test method and computer storage medium - Google Patents
Convolutional neural network-oriented variation coverage test method and computer storage medium Download PDFInfo
- Publication number
- CN110347600A CN110347600A CN201910623892.3A CN201910623892A CN110347600A CN 110347600 A CN110347600 A CN 110347600A CN 201910623892 A CN201910623892 A CN 201910623892A CN 110347600 A CN110347600 A CN 110347600A
- Authority
- CN
- China
- Prior art keywords
- convolutional neural
- model
- neural networks
- variation
- test
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000010998 test method Methods 0.000 title claims abstract description 14
- 230000001537 neural effect Effects 0.000 title abstract description 4
- 238000012360 testing method Methods 0.000 claims abstract description 71
- 238000013527 convolutional neural network Methods 0.000 claims abstract description 50
- 230000035772 mutation Effects 0.000 claims abstract description 45
- 238000000034 method Methods 0.000 claims abstract description 33
- 238000012549 training Methods 0.000 claims abstract description 13
- 230000008859 change Effects 0.000 claims description 17
- 238000004590 computer program Methods 0.000 claims description 13
- 230000004913 activation Effects 0.000 claims description 10
- 238000004364 calculation method Methods 0.000 claims description 3
- 230000009467 reduction Effects 0.000 claims description 3
- 238000003062 neural network model Methods 0.000 abstract description 7
- 230000007547 defect Effects 0.000 abstract description 2
- 238000010586 diagram Methods 0.000 description 7
- 230000008569 process Effects 0.000 description 6
- 230000006870 function Effects 0.000 description 5
- 238000012545 processing Methods 0.000 description 5
- 230000000694 effects Effects 0.000 description 3
- 238000005259 measurement Methods 0.000 description 3
- 238000013528 artificial neural network Methods 0.000 description 2
- 238000002474 experimental method Methods 0.000 description 2
- 239000000203 mixture Substances 0.000 description 2
- 239000000243 solution Substances 0.000 description 2
- 230000008485 antagonism Effects 0.000 description 1
- 238000004422 calculation algorithm Methods 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 230000007812 deficiency Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000002347 injection Methods 0.000 description 1
- 239000007924 injection Substances 0.000 description 1
- 238000007689 inspection Methods 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000003058 natural language processing Methods 0.000 description 1
- 210000002569 neuron Anatomy 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000013102 re-test Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/36—Preventing errors by testing or debugging software
- G06F11/3668—Software testing
- G06F11/3672—Test management
- G06F11/3676—Test management for coverage analysis
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/36—Preventing errors by testing or debugging software
- G06F11/3668—Software testing
- G06F11/3672—Test management
- G06F11/3688—Test management for test execution, e.g. scheduling of test suites
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Computer Hardware Design (AREA)
- Quality & Reliability (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
- Debugging And Monitoring (AREA)
Abstract
The invention discloses a convolutional neural network-oriented variation coverage test method and a computer storage medium, wherein the method comprises the following steps of: 1) setting n mutation operators, and respectively injecting the n mutation operators into the convolutional neural network program P to be tested to obtain a mutation program set { P1,P2,P3,…,Pn}; 2) using a training data set D versus a variant program set { P1,P2,P3,…,PnTraining is carried out to obtain a variation model set { M }1,M2,M3,…,Mn}; 3) using test data set T to pair original model M and variation model set { M1,M2,M3,…,MnTesting is carried out; 4) and comparing the test accuracy rates of all the models, and selecting the model with the highest accuracy rate. The invention solves the defect that the traditional test method is difficult to ensure the test sufficiency of the convolutional neural network application program, can effectively improve the test sufficiency of the convolutional neural network, is more effective in testing a neural network model, and can be used for testing the neural network model according to the testThe local optimal model is found out according to the test accuracy rate, and the quality and the safety of the convolutional neural network application program are effectively guaranteed.
Description
Technical field
The present invention relates to a kind of method for testing software and computer storage mediums, more particularly to one kind towards convolutional Neural
The variation coverage test method and computer storage medium of network.
Background technique
Practical application of the convolutional neural networks in the fields such as image classification and identification, natural language processing achieve it is huge at
Function, the field much to concern safely, which is also tried to be the first, introduces convolutional neural networks.However, since nearest convolutional neural networks system occurs
Some mistakes so that everybody increasingly pays close attention to safety, the reliability of convolutional neural networks application program.Current test side
Method mainly has whitepack differential testing algorithm, for systematically generating the antagonism example of all neurons in overlay network.For
Convolutional neural networks testing adequacy, existing software test adequacy method and criterion can not directly apply to convolutional Neural
The test of network, this is because convolutional neural networks have the property that
Characteristic one: data sensitive.The control logic and operation rule of traditional software are passed through by software developer
What coding was realized, and the feature of convolutional neural networks maximum is with data sensitive.Their control logic and operation rule
It is all by being got from training dataset study, the variation of training data may cause the variation of tested software.In test process
The test case of the triggerable software operational defect of construction is likely to be used to re -training model, therefore tested software is likely to
It changes, and the change of the control logic as caused by this variation is unknown, it is also necessary to retest；
Feature two: unintelligible property.The neural network of one deep layer, each layer represents a feature, and the number of plies is more, even
Developer does not know what feature each layer represents.Although convolutional neural networks have very high in many application scenarios
Accuracy rate, but be difficult explain training data in those of characteristic play a key effect, the control that we have no way of obtaining software is patrolled
Volume and operation rule；
Feature three: tested program parametrization.In traditional software test, measurand will be understood by code.Convolution mind
It is a series of weighting parameters, offset parameter through web application test measurand.Measurand is more abstract, gives tester
Member brings more challenges.
Traditional software test adequacy criterion is mostly based on control stream, this is because the control logic and fortune of traditional software
Line discipline is all that programmer is set by programming.And for convolutional neural networks, their control logic and operation rule
All be by being got from training dataset study, it is therefore, traditional when measuring convolutional neural networks Application testing adequacy
Test Adequacy Criteria be difficult to be applicable in.
Summary of the invention
Goal of the invention: the variation towards convolutional neural networks that the technical problem to be solved in the present invention is to provide a kind of, which covers, surveys
Method for testing and computer storage medium solve conventional test methodologies and are difficult to ensure that the test of convolutional neural networks application program is abundant
Property deficiency, can effectively improve examine convolutional neural networks testing adequacy, test neural network model on more have
Effect, and local optimum model can be found out according to test accuracy rate.
Technical solution: the variation coverage test method of the present invention towards convolutional neural networks, comprising the following steps:
(1) n mutation operator is set, is injected separately into convolutional neural networks program P to be measured, variation procedure set { P is obtained1,
P2,P3,…,Pn}；
(2) using training dataset D to variation procedure set { P1,P2,P3,…,PnBe trained, obtain mutation model collection
{M1,M2,M3,…,Mn}；
(3) using test data set T to archetype M and mutation model collection { M1,M2,M3,…,MnTested；
(4) test accuracy rate of more all models selects the highest model of accuracy rate.
Further, the mutation operator in step (1) is 9, including changes activation primitive, change pond mode, reduce
The convolution number of plies, increase convolution kernel number, reduction convolution kernel number, increase convolution kernel size, reduction convolution kernel size, increase connect entirely
It connects the number of plies, reduce the full connection number of plies.
Further, the change activation primitive is that original activation primitive is changed to elu activation primitive.
Further, change pond mode is that original pool mode is changed to the pond same mode.
Further, the increase convolution kernel size is that original convolution core size is become 7*7.
Further, the increase convolution kernel size is that original convolution core size is become 3*3.
Computer storage medium of the present invention, is stored thereon with computer program, and the computer program is being counted
Calculation machine processor realizes the above-mentioned variation coverage test method towards convolutional neural networks when executing.
The utility model has the advantages that the present invention is based on variation coverage tests to propose a kind of model covering method, inspection convolution can be improved
The testing adequacy of neural network, so that it is more effective on test neural network model, to ensure convolutional neural networks application
The quality and safety of program, and be able to verify that whether the design of neural network model reasonable, i.e. the exploitation of convolutional neural networks
Whether personnel have chosen the model of a local optimum (in the model I), and are improving the abundant of such model measurement
Preferable effect can be played above property.
Detailed description of the invention
Fig. 1 is the method flow diagram of the present embodiment；
Fig. 2 is change curve of each model measurement accuracy rate with the number of iterations.
Specific embodiment
The method flow of embodiment of the present invention is as shown in Figure 1, input is convolutional neural networks program P to be measured, trains number
According to collection D and test data set T, steps are as follows:
Step 1: the n kind mutation operator of setting being injected separately into convolutional neural networks program P to be measured, will be obtained a series of
Variation program p '；
Step 2: a series of variation moulds will be obtained by being trained respectively to this serial variance program p ' with training dataset D
Type { M1,M2,M3,…,Mn}；
Step 3: with test data set T to mutation model collection M ' (by master mould and resulting mutation model { M1,M2,
M3,…,MnComposition) tested；
Step 4: contrast and experiment, whether the test accuracy rate for observing master mould is that test accuracy rate is most in the class model
Height, and select the highest model of test accuracy rate.
By the above method, to convolutional neural networks model to be measured whether be the part of the mutation model collection constructed most
Excellent to determine, i.e., whether judging nicety rate is concentrated in this this kind of mutation model constructed in highest.
This method is used in convolutional neural networks LeNet-5 model by present embodiment, and data set is common Mnist number
According to collection.We devise 6 class mutation operators (9 mutation operators) altogether for the program, are injected separately into convolutional neural networks to be measured
It in program P, then is trained with original training set, obtains 9 mutation models, we are by this 9 mutation models upper former mould again
Type is referred to as mutation model collection.Our the sort accuracys rate in this 10 models are highest to be used as local optimum model, leads to
This method is crossed to promote the testing adequacy of convolutional neural networks.
Convolutional neural networks model may be influenced by many parameters, and the present invention is directed to the sieve of convolutional neural networks model
9 mutation operators have been selected, convolutional neural networks model will be directly changed to convolutional neural networks program P to be measured injection variation.9
Mutation operator is as shown in table 1.
The mutation operator of 1 convolutional neural networks of table
| Mutation operator | Description |
| Change activation primitive (CAF) | Change original relu into elu |
| Change pond mode (CMP) | Change original vaild into same |
| It reduces the convolution number of plies (RCL) | Reduce by a convolutional layer |
| Increase convolution kernel number (ACK) | Original convolution kernel number is doubled |
| It reduces convolution kernel number (RCK) | Original convolution kernel number is reduced one times |
| Increase convolution kernel size (ECKS) | Change original convolution kernel size into 7*7 |
| Reduce convolution kernel size (CCKS) | Change original convolution kernel size into 3*3 |
| Increase the full connection number of plies (AFCL) | Increase a full articulamentum |
| Reduce the full connection number of plies (RFCL) | Reduce by a full articulamentum |
Above-mentioned different mutation operator is injected into original convolutional neural networks training program, it is available opposite
The variation CNN program { P answered1′,P2′,…,Pn′}.Then, the test set T of master mould M is run into each variation program { P1′,
P2′,…,Pn' on available corresponding mutation model { M1′,M′2,…,M′n}。
For variation convolutional neural networks model, any one test case t ∈ T can be by original convolutional neural networks
Model M is correctly classified, and cannot then say that test case t is killed by variation convolutional neural networks model M ' correctly classification
Variant M '.Traditional variation score mutation testing refers to kill variant/all variants.But the variation of traditional software
Scoring Guidelines are not suitable for the mutation testing of convolutional neural networks system.This is because becoming to convolutional neural networks model
When different test, since test set T quantity is bigger, it is very easy that test case t ∈ T, which kills variant M ',.Based on above
Reason, for the mutation testing of convolutional neural networks, obtained mutation operator is injected convolutional neural networks training program by us.
Program is executed using training set data re -training, generates corresponding variation convolutional neural networks model { M '1,M′2,…,M′n}。
Then, each mutation model model { M ' is executed according to the test set T of master mould M1,M′2,…,M′nAnd analyze it.Make
The mutation model obtained with this kind of method is more, then testing adequacy is higher.Simultaneously we can more all models test
Accuracy rate, picks out the highest model of accuracy rate, which is local optimum model.
The present invention is by { M, M '1,M′2,…,M′nIt is considered as same class model, referred to as mutation model class, and as quilt
It surveys object and improves the testing adequacy of this class model to a certain extent to promote the coverage rate of measurand.For passing
The software of system can use a large amount of test method (equivalence class, boundary value, metamorphic testing) and construct reliable test case
Collection, improves the sentence covering rate, branch covering rate, Condition Coverage Testing rate of software code, to improve testing adequacy.For convolution
Neural network model, a test case is possible to reach " covering " entire model sometimes.Therefore, for convolutional neural networks
Model, it is impossible to which there are reliable test use cases.However, the present invention uses for reference existing " reliable test use cases " theory, propose
The concept of mutation model collection, i.e., if a mutation model collection is adequately for mutation operator collection, which is collectively referred to as
Reliable mutation model collection.The survey of each target convolution neural network model can be thus improved based on reliable mutation model collection
Try adequacy.Test whether original model is local optimum, if not being local optimum, illustrates original model in model covering
It tests insufficient.By we method we specific optimal models can be selected from this class model, and pass through raising
Model covering is to improve testing adequacy.
The test accuracy rate of each model of table 2
| Corresponding model | Test accuracy rate |
| Archetype (Original) | 98.06% |
| Change activation primitive (CAF) | 98.00% |
| Change the mode (CMP) in pond | 98.01% |
| It reduces the convolution number of plies (RCL) | 96.26% |
| Increase convolution kernel number (ACK) | 98.65% |
| It reduces convolution kernel number (RCK) | 97.65% |
| Increase convolution kernel size (ECKS) | 98.00% |
| Reduce convolution kernel size (CCKS) | 98.20% |
| Increase the full connection number of plies (AFCL) | 99.03% |
| Reduce the full connection number of plies (RFCL) | 97.06% |
The test accuracy rate of each model is as shown in table 2.It can be concluded that increasing the mould obtained after the full connection number of plies (AFCL)
Type is local optimum model in this class model, illustrates that archetype test is insufficient.Original is improved by our method simultaneously
The model of beginning model covers, i.e., in archetype building process, it should which selection increases the model obtained after the full connection number of plies
Rather than archetype.Other mutation operators are compared by increasing the full connection number of plies, the accuracy rate of mutation model will increase.Respectively
Model measurement accuracy rate is with the change curve of the number of iterations as shown in Fig. 2, we can observe this 9 kinds of mutation model classes plus original
Carry out test accuracy rate of the mutation model class of LeNet-5 model composition after whole test sample iteration 20 times.It can be with from Fig. 2
It obtains, increasing the model obtained after the full connection number of plies is local optimum model in this class model, illustrates that archetype test is not filled
Point.It is covered simultaneously by the model that our method improves archetype, i.e., in archetype building process, it should select
Increase the model obtained after the full connection number of plies rather than archetype.Other mutation operators are compared by increasing the full connection number of plies,
The accuracy rate of its mutation model will increase.By above-mentioned experiment, the testing adequacy of archetype is improved, and then demonstrates mould
The validity of type coverage criterion.
Present invention could apply to can be improved in the AI software such as recognition of face, text analyzing and image classification in this process
The testing adequacy of the convolutional neural networks model guarantees the quality and peace of software to improve the accuracy of this kind of software
Entirely.Such as when testing convolutional neural networks image classification software, can effectively it be improved using method of the invention
The testing adequacy of the convolutional neural networks model, while the local optimum model of the convolutional neural networks can also be obtained, it is promoted
The accuracy and reliability of software.
If the embodiment of the present invention is realized and when sold or used as an independent product in the form of software function module,
Also it can store in a computer readable storage medium.Based on this understanding, the technical solution of the embodiment of the present invention
Substantially the part that contributes to existing technology can be embodied in the form of software products in other words, the computer software
Product is stored in a storage medium, including some instructions are used so that computer equipment (can be personal computer,
Server or the network equipment etc.) execute all or part of each embodiment the method for the present invention.And storage above-mentioned is situated between
Matter, which includes: that USB flash disk, mobile hard disk, read-only memory (ROM, Read Only Memory), magnetic or disk etc. are various, to deposit
Store up the medium of program code.It is combined in this way, present example is not limited to any specific hardware and software.
Correspondingly, being stored thereon with computer program the embodiments of the present invention also provide a kind of computer storage medium.
When the computer program is executed by processor, the aforementioned variation coverage test side towards convolutional neural networks may be implemented
Method.For example, the computer storage medium is computer readable storage medium.
It should be understood by those skilled in the art that, embodiments herein can provide as method, system or computer program
Product.Therefore, complete hardware embodiment, complete software embodiment or reality combining software and hardware aspects can be used in the application
Apply the form of example.Moreover, it wherein includes the computer of computer usable program code that the application, which can be used in one or more,
The computer program implemented in usable storage medium (including but not limited to magnetic disk storage, CD-ROM, optical memory etc.) produces
The form of product.
The application is referring to method, the process of equipment (system) and computer program product according to the embodiment of the present application
Figure and/or block diagram describe.It should be understood that every one stream in flowchart and/or the block diagram can be realized by computer program instructions
The combination of process and/or box in journey and/or box and flowchart and/or the block diagram.It can provide these computer programs
Instruct the processor of general purpose computer, special purpose computer, Embedded Processor or other programmable data processing devices to produce
A raw machine, so that being generated by the instruction that computer or the processor of other programmable data processing devices execute for real
The device for the function of being specified in present one or more flows of the flowchart and/or one or more blocks of the block diagram.
These computer program instructions, which may also be stored in, is able to guide computer or other programmable data processing devices with spy
Determine in the computer-readable memory that mode works, so that it includes referring to that instruction stored in the computer readable memory, which generates,
Enable the manufacture of device, the command device realize in one box of one or more flows of the flowchart and/or block diagram or
The function of being specified in multiple boxes.
These computer program instructions also can be loaded onto a computer or other programmable data processing device, so that counting
Series of operation steps are executed on calculation machine or other programmable devices to generate computer implemented processing, thus in computer or
The instruction executed on other programmable devices is provided for realizing in one or more flows of the flowchart and/or block diagram one
The step of function of being specified in a box or multiple boxes.
Claims (7)
1. a kind of variation coverage test method towards convolutional neural networks, which comprises the following steps:
(1) n mutation operator is set, is injected separately into convolutional neural networks program P to be measured, variation procedure set { P is obtained1,P2,
P3,…,Pn}；
(2) using training dataset D to variation procedure set { P1,P2,P3,…,PnBe trained, obtain mutation model collection { M1,
M2,M3,…,Mn}；
(3) using test data set T to archetype M and mutation model collection { M1,M2,M3,…,MnTested；
(4) test accuracy rate of more all models selects the highest model of accuracy rate.
2. the variation coverage test method according to claim 1 towards convolutional neural networks, it is characterised in that: step
(1) mutation operator in is 9, including changes activation primitive, change pond mode, reduce the convolution number of plies, increase convolution kernel
Number reduces convolution kernel number, increases convolution kernel size, reduction convolution kernel size, increase the full connection number of plies, reduce full articulamentum
Number.
3. the variation coverage test method according to claim 2 towards convolutional neural networks, it is characterised in that: described to change
Becoming activation primitive is that original activation primitive is changed to elu activation primitive.
4. the variation coverage test method according to claim 2 towards convolutional neural networks, it is characterised in that: described to change
Becoming pond mode is that original pool mode is changed to the pond same mode.
5. the variation coverage test method according to claim 2 towards convolutional neural networks, it is characterised in that: the increasing
Big convolution kernel size is that original convolution core size is become 7*7.
6. the variation coverage test method according to claim 2 towards convolutional neural networks, it is characterised in that: the increasing
Big convolution kernel size is that original convolution core size is become 3*3.
7. a kind of computer storage medium, is stored thereon with computer program, it is characterised in that: the computer program is being counted
Calculation machine processor realizes method as claimed in any one of claims 1 to 6 when executing.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910623892.3A CN110347600B (en) | 2019-07-11 | 2019-07-11 | Convolutional neural network-oriented variation coverage testing method and computer storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910623892.3A CN110347600B (en) | 2019-07-11 | 2019-07-11 | Convolutional neural network-oriented variation coverage testing method and computer storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110347600A true CN110347600A (en) | 2019-10-18 |
| CN110347600B CN110347600B (en) | 2023-04-07 |
Family
ID=68175695
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910623892.3A Expired - Fee Related CN110347600B (en) | 2019-07-11 | 2019-07-11 | Convolutional neural network-oriented variation coverage testing method and computer storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110347600B (en) |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111858340A (en) * | 2020-07-23 | 2020-10-30 | 深圳慕智科技有限公司 | Deep neural network test data generation method based on stability transformation |
| CN111881033A (en) * | 2020-07-23 | 2020-11-03 | 深圳慕智科技有限公司 | Deep learning model quality evaluation method based on operation environment error analysis |
| CN113051150A (en) * | 2019-12-27 | 2021-06-29 | 中国人民解放军陆军工程大学 | Metamorphic test method and system for image classifier |
| CN113268423A (en) * | 2021-05-24 | 2021-08-17 | 南京工业大学 | Deep learning mutation operator reduction method |
| CN113485932A (en) * | 2021-07-16 | 2021-10-08 | 深圳市网联安瑞网络科技有限公司 | Deep learning code defect detection method, system, product, equipment and terminal |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105302719A (en) * | 2015-10-26 | 2016-02-03 | 北京科技大学 | Mutation test method and apparatus |
| WO2017177661A1 (en) * | 2016-04-15 | 2017-10-19 | 乐视控股（北京）有限公司 | Convolutional neural network-based video retrieval method and system |
| CN108009525A (en) * | 2017-12-25 | 2018-05-08 | 北京航空航天大学 | A kind of specific objective recognition methods over the ground of the unmanned plane based on convolutional neural networks |
| CN108664391A (en) * | 2018-03-13 | 2018-10-16 | 北京邮电大学 | A kind of Fault Classification, mutation testing method and apparatus towards program state |
-
2019
- 2019-07-11 CN CN201910623892.3A patent/CN110347600B/en not_active Expired - Fee Related
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105302719A (en) * | 2015-10-26 | 2016-02-03 | 北京科技大学 | Mutation test method and apparatus |
| WO2017177661A1 (en) * | 2016-04-15 | 2017-10-19 | 乐视控股（北京）有限公司 | Convolutional neural network-based video retrieval method and system |
| CN108009525A (en) * | 2017-12-25 | 2018-05-08 | 北京航空航天大学 | A kind of specific objective recognition methods over the ground of the unmanned plane based on convolutional neural networks |
| CN108664391A (en) * | 2018-03-13 | 2018-10-16 | 北京邮电大学 | A kind of Fault Classification, mutation testing method and apparatus towards program state |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113051150A (en) * | 2019-12-27 | 2021-06-29 | 中国人民解放军陆军工程大学 | Metamorphic test method and system for image classifier |
| CN111858340A (en) * | 2020-07-23 | 2020-10-30 | 深圳慕智科技有限公司 | Deep neural network test data generation method based on stability transformation |
| CN111881033A (en) * | 2020-07-23 | 2020-11-03 | 深圳慕智科技有限公司 | Deep learning model quality evaluation method based on operation environment error analysis |
| CN113268423A (en) * | 2021-05-24 | 2021-08-17 | 南京工业大学 | Deep learning mutation operator reduction method |
| CN113485932A (en) * | 2021-07-16 | 2021-10-08 | 深圳市网联安瑞网络科技有限公司 | Deep learning code defect detection method, system, product, equipment and terminal |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110347600B (en) | 2023-04-07 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110347600A (en) | Convolutional neural network-oriented variation coverage test method and computer storage medium | |
| US11551036B2 (en) | Methods and apparatuses for building data identification models | |
| Mason et al. | A guide for using functional diversity indices to reveal changes in assembly processes along ecological gradients | |
| CN109408389A (en) | A kind of aacode defect detection method and device based on deep learning | |
| US20140033174A1 (en) | Software bug predicting | |
| US20190246299A1 (en) | Method and test system for mobile network testing as well as a network testing system | |
| CN105548946B (en) | For the multiple criteria screening method and device of the measurement error of intelligent electric energy meter | |
| CN110298389A (en) | More wheels circulation feature selection approach and device when training pattern | |
| CN111026664A (en) | Program detection method and detection system based on ANN and application | |
| SA520412674B1 (en) | Accounting for tool based effects in nuclear magnetic resonance logging data | |
| CN109783387A (en) | Thread test method, device, computer equipment and storage medium | |
| González et al. | Test data generation for model transformations combining partition and constraint analysis | |
| CN109933977A (en) | A kind of method and device detecting webshell data | |
| CN109582575A (en) | Game test method and device | |
| Schaaf et al. | Towards measuring bias in image classification | |
| CN107247663B (en) | Redundancy variant identification method | |
| CN104252540B (en) | A kind of game configuration method for pushing graded based on computer performance | |
| CN112527573B (en) | Interface testing method, device and storage medium | |
| CN109033564A (en) | A kind of aircraft airborne equipment testability demonstration index evaluating method | |
| Kumar et al. | Notice of Retraction: Generation of efficient test data using path selection strategy with elitist GA in regression testing | |
| CN113656279B (en) | Code odor detection method based on residual network and metric attention mechanism | |
| CN107886233A (en) | The QoS evaluating method and system of customer service | |
| Lavazza et al. | Understanding and modeling ai-intensive system development | |
| CN110889103A (en) | Method and system for slider verification and model training method thereof | |
| Mohamad | Machine learning empowered software prediction system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| CF01 | Termination of patent right due to non-payment of annual fee | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20230407 |