CN109416653A - Selecting unit test carries out network analysis - Google Patents
Selecting unit test carries out network analysis Download PDFInfo
- Publication number
- CN109416653A CN109416653A CN201780024091.4A CN201780024091A CN109416653A CN 109416653 A CN109416653 A CN 109416653A CN 201780024091 A CN201780024091 A CN 201780024091A CN 109416653 A CN109416653 A CN 109416653A
- Authority
- CN
- China
- Prior art keywords
- unit
- unit testing
- service
- test
- computer
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000012360 testing method Methods 0.000 title claims abstract description 279
- 238000003012 network analysis Methods 0.000 title description 3
- 238000000034 method Methods 0.000 claims abstract description 95
- 230000008569 process Effects 0.000 claims abstract description 23
- 230000002452 interceptive effect Effects 0.000 claims abstract description 10
- 238000013507 mapping Methods 0.000 claims abstract description 8
- 230000006870 function Effects 0.000 claims description 40
- 238000003066 decision tree Methods 0.000 claims description 27
- 238000004891 communication Methods 0.000 claims description 2
- 238000010998 test method Methods 0.000 description 25
- 238000003745 diagnosis Methods 0.000 description 7
- 230000004044 response Effects 0.000 description 5
- 230000010354 integration Effects 0.000 description 4
- 238000012552 review Methods 0.000 description 4
- 230000003993 interaction Effects 0.000 description 3
- 230000007257 malfunction Effects 0.000 description 3
- 239000011159 matrix material Substances 0.000 description 3
- 239000000700 radioactive tracer Substances 0.000 description 3
- 230000001419 dependent effect Effects 0.000 description 2
- 238000001514 detection method Methods 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000012545 processing Methods 0.000 description 2
- 230000009467 reduction Effects 0.000 description 2
- XXUZFRDUEGQHOV-UHFFFAOYSA-J strontium ranelate Chemical compound [Sr+2].[Sr+2].[O-]C(=O)CN(CC([O-])=O)C=1SC(C([O-])=O)=C(CC([O-])=O)C=1C#N XXUZFRDUEGQHOV-UHFFFAOYSA-J 0.000 description 2
- 238000004458 analytical method Methods 0.000 description 1
- 230000006399 behavior Effects 0.000 description 1
- 230000015556 catabolic process Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000012512 characterization method Methods 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000011156 evaluation Methods 0.000 description 1
- 238000007689 inspection Methods 0.000 description 1
- 238000012423 maintenance Methods 0.000 description 1
- 239000010979 ruby Substances 0.000 description 1
- 229910001750 ruby Inorganic materials 0.000 description 1
- 238000012216 screening Methods 0.000 description 1
- 239000013589 supplement Substances 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
- 238000013024 troubleshooting Methods 0.000 description 1
- 238000010200 validation analysis Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
- G06F11/0703—Error or fault processing not based on redundancy, i.e. by taking additional measures to deal with the error or fault not making use of redundancy in operation, in hardware, or in data representation
- G06F11/079—Root cause analysis, i.e. error or fault diagnosis
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
- G06F11/0703—Error or fault processing not based on redundancy, i.e. by taking additional measures to deal with the error or fault not making use of redundancy in operation, in hardware, or in data representation
- G06F11/0706—Error or fault processing not based on redundancy, i.e. by taking additional measures to deal with the error or fault not making use of redundancy in operation, in hardware, or in data representation the processing taking place on a specific hardware platform or in a specific software environment
- G06F11/0709—Error or fault processing not based on redundancy, i.e. by taking additional measures to deal with the error or fault not making use of redundancy in operation, in hardware, or in data representation the processing taking place on a specific hardware platform or in a specific software environment in a distributed system consisting of a plurality of standalone computer nodes, e.g. clusters, client-server systems
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- General Engineering & Computer Science (AREA)
- Quality & Reliability (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Computer Hardware Design (AREA)
- Debugging And Monitoring (AREA)
Abstract
The present invention provides a kind of process analyzed the system for including multiple physical entities, the multiple physical entity executes interactive software and applies to provide a variety of services.The process includes: for software application determination unit test, wherein at least partly described software application has been assigned more than one unit testing, if some unit testing for using unavailable, distributing to the application fails.The unit testing of the determination is mapped to the multiple service to establish data structure, wherein the data structure instruction is when a certain service is unavailable, which unit testing failure.Then, the subset of the unit testing of the mapping is executed to analyze the availability of the service in the system, wherein the member of the subset is in the process of implementation based in front unit test being failure or by being dynamically determined in the subset.
Description
Technical field
The present invention relates to analyzing the system with multiple physical entities, it is soft that the multiple physical entity executes interaction
Part is applied to provide a variety of services.The invention particularly relates to selecting unit tests to be analyzed.
Background technique
System with the multiple physical entities for executing interactive software application, such as cloud platform, can be by including multiple
Net computer is implemented.Every computer (alternatively referred to as " server "), which provides to the client of user's control, has various function
The service of energy.With the promotion of processing capacity, network and memory technology, these systems become stronger and stronger, also become to get over
Come more complicated.Due to the increase of complexity, the requirement to effectively diagnostic system failure is also higher and higher.
The protos test suite PROTOS including multiple unit testings can be used to carry out verification software application, these unit testings are opened as software
A part of hair process is developed, and is associated with control data it is intended that combining, targetedly independent test software application
Minimum testable unit, also referred to as unit.Execution unit test can effectively examine code in software development process
Integrated situation, also, execution unit test can verify that the modification that developer makes code, so that it is guaranteed that code quality.
Summary of the invention
Unit testing in system with multiple physical entities will not usually provide any in relation to without response or failure clothes
The information of business, wherein the multiple physical entity, which executes, provides the interactive software application of a variety of services.On the contrary, execution unit is surveyed
The inventory of a test pass through and failure can be generated after examination.The inventory can help positioning fault of software or discovery each
The problem of code module, however, the inventory itself is not used to troubleshooting service.Additionally, it is possible to which there are a large amount of available cells
Test, with being continuously increased for code library, the quantity of unit testing may be further increased.Therefore, all available to execute
Unit testing, time-consuming may be very long.
Therefore, Systems Operator would generally customize test to diagnose fault service.But if system continues to develop, frequency
It updates and modifies numerously, then the test customized usually can be out-of-date.Therefore, when such as issuing new software version every time, these
Test also needs constantly to modify/update therewith.This method cost for operator is very high, because operator needs to expend
Vast resources keeps the detailed and timely of Service Diagnostic.Therefore, it is necessary to another lower-cost methods to hold to diagnose to have
Failed services in the system of multiple physical entities of row interactive software application.
According to the first aspect of the invention, a kind of computer-readable medium, the computer-readable medium storage are provided
There is computer-readable instruction, for analyzing the system with multiple physical entities, the multiple physical entity executes one
A or multiple interactive softwares are applied to provide a variety of services, and executing the computer-readable instruction can make the system progress following
Operation: closing first service, and executes the unit testing of selection and led to the failure with determination since the first service is unavailable
First unit test；Second service is closed, and executes the selected unit testing to determine due to the second service not
Can with and the second unit that leads to the failure is tested；And the starting service, and execute first and second unit testing
Subset is to analyze the system, wherein the member of the subset in front unit in the process of implementation based on surveying in the subset
Examination is failure or by being dynamically determined.
It should be noted that term used in specification and claims " physical entity " refers in particular to join
Net computer, wherein computer may include processor, memory and network interface.In addition, it is necessary to explanation, this explanation
Term used in book and claim " interactive software application " refers in particular to the software that a certain operation is executed using other side
Using.For example, a software application indicates another software application to execute a certain operation, and the operation is waited to be performed.
In addition, it should be noted that, term used in specification and claims " service " refers in particular to process,
Client can (spontaneously) call the process, and the process for example provides data to the client or third party,
In, request of the data based on the client.In addition, it is necessary to illustrate, used in specification and claims
Term " failure " and " passing through " refer in particular to unit testing determine failure or pass through.That is, if a certain service is closed, and unit
Test passes through, then the unit testing is not used to determine whether the service can be used.On the other hand, if a certain service close and
Unit testing failure, then the unit testing can be used in determining whether the service can be used.
Therefore, available cell test (that is, the unit testing developed in software development process) can be used for diagnosing unavailable
Service.
In the first possible way of realization of the first aspect, the unit testing of the execution corresponds in decision tree
Path on node, the leaf in the decision tree indicates the unavailable service in the system.
It should be noted that term used in specification and claims " decision tree " refer in particular to it is similar
In the decision chart or model of tree and the possible outcome of the decision, wherein the decision and unit testing failure are still logical
Cross related, the result and service are assumed available or unavailable related.
Therefore, relevant unit testing need to only be executed, that is, having to carry out the service of just can determine that, whether available unit is surveyed
Examination.
In second of possible way of realization of the first aspect, the node on the path is according to the corresponding list
The ratio of the quantity of unavailable service that member test is able to detect that and stock number needed for the execution unit testing carries out
Sequence.
It should be noted that term used in specification and claims " executes the unit testing institute
The stock number needed " refers in particular to process resource amount required for a unit testing or processing time.
It generates due to preferentially executing compared with the unit testing of multi information/consuming fewer resource, it is able to ascend globality
Energy.
In the third possible way of realization of the first aspect, the computer-readable medium storage has computer can
Reading instruction, executing the computer-readable instruction can be such that the system performs the following operation: the letter called to available cell test
Number assigns relevance score；And the available cell gone to select in following situations is tested: only calling relevance score lower than threshold value
Be ignored function；Or only calling relevance score more than or equal to the function of the threshold value, the function is also described simultaneously
Other unit testings in the unit testing of selection are called.
It should be noted that term used in specification and claims " function " refers in particular to a kind of use
In the support method of execution unit test.
Therefore, unit testing can be screened to avoid incoherent or redundancy unit testing is executed.
In the 4th kind of possible way of realization of the first aspect, the relevance score is based in unit testing set
The file that is encoded of the unit testing in call the number of the function.
Therefore, the degree of correlation of function can be judged based on the use uniqueness of the function.
The unit testing in the 5th kind of possible way of realization of the first aspect, in the unit testing set
Call the number of the function more in the file encoded, the relevance score is lower.
Therefore it can be assumed that the importance of the function frequently called is less than the function seldom called.
In the 6th kind of possible way of realization of the first aspect, the computer-readable medium storage has computer can
Reading instruction, executing the computer-readable instruction can make the system go that available cell is selected to test, wherein the available cell is surveyed
The function also called by other unit testings in the selected unit testing is only called in examination.
It can thus be avoided selection redundancy unit test.
In the 7th kind of possible way of realization of the first aspect, the computer-readable medium storage has computer can
Reading instruction, executing the computer-readable instruction can make the system remove the available list for selecting performance scores to be lower than other unit testings
Member test.
It is surveyed therefore, because one group of unit testing of same (correlation) function is called to be summed up as the least unit of required resource
Examination, so being able to ascend overall performance.
In the 8th kind of possible way of realization of the first aspect, the computer-readable medium storage has computer can
Reading instruction, executing the computer-readable instruction can be such that the system performs the following operation: be tested based on redundancy unit is assigned to
Performance scores, the member of the subset is determined from the redundancy unit test in first and second unit testing,
Wherein, the redundancy unit testing needle is identical to the Failure Mode of unavailable service.
Therefore, because the one group of redundancy unit test for generating identical service availability information when being executed is summed up as institute
The least unit testing of resource is needed, so overall performance can be promoted further.
In the 9th kind of possible way of realization of the first aspect, the performance scores are tested based on the redundancy unit
The execution time.
Therefore, whole network analysis can be reduced and execute the time.
In the tenth kind of possible way of realization of the first aspect, the computer-readable medium is included in calculating equipment
In, the calculating equipment further includes processor, also, the calculating equipment is located in communication network.
According to the second aspect of the invention, a kind of side analyzed the system with multiple physical entities is provided
Method, the multiple physical entity execute interactive software and apply to provide a variety of services.The described method includes: being the software application
Determination unit test, wherein at least partly described software application has been assigned more than one unit testing, if some application can not
With the unit testing for then distributing to the application fails；The unit testing of the determination is mapped to a variety of services to build
Vertical data structure, the data structure instruction is when a certain service is unavailable, which unit testing failure；And it is reflected described in executing
The subset for the unit testing penetrated is to analyze the availability of the service in the system, wherein the member of the subset was executing
Based in front unit test being failure or by being dynamically determined in the subset in journey.
Therefore, as described above, available cell test (that is, the unit testing developed in software development process) can be used for examining
Break unavailable service.
In the first possible way of realization of the second aspect, the unit testing of the execution corresponds in decision tree
Path on node, the leaf in the decision tree indicates the unavailable service in the system.
Therefore, as described above, need to only execute relevant unit testing, that is, have to carry out whether the service of just can determine that can be used
Unit testing.
In second of possible way of realization of the second aspect, the node on the path is according to the corresponding list
The ratio of the quantity of unavailable service that member test is able to detect that and stock number needed for the execution unit testing carries out
Sequence.
Therefore, as noted previously, as preferential execute the unit testing for generating compared with multi information or expending fewer resource, so energy
Enough promote overall performance.
In the third possible way of realization of the second aspect, also wrapped for software application determination unit test
It includes: determining to the function call by available cell test execution；Relevance score is assigned to the function；Ignore and only calls correlation
Spend unit testing of the score lower than the function of threshold value；And ignore the letter for only calling relevance score to be greater than or equal to the threshold value
Several unit testings, and the function is also called by other unit testings in the unit testing of the determination simultaneously.
Therefore, as described above, can be screened to unit testing to avoid incoherent or redundancy unit testing is executed.
In the 4th kind of possible way of realization of the second aspect, the relevance score is surveyed based on the available cell
The number of the function is called in examination.
Therefore, as described above, the degree of correlation of function can be judged based on the use uniqueness of the function.
In the 5th kind of possible way of realization of the second aspect, include: for software application determination unit test
It determines to the function call by available cell test execution；And ignores and only call also by its in the unit testing of the determination
The available cell test for the function that its unit testing is called.
Therefore, as set forth above, it is possible to avoid selection redundancy unit test.
In the 6th kind of possible way of realization of the second aspect, for calling the unit testing of same functions, ignore
Performance scores are lower than the unit testing of other unit testings to carry out duplicate removal.
Therefore, as noted previously, as one group of unit testing of same (correlation) function is called to be summed up as required resource most
Few unit testing, so being able to ascend overall performance.
In the 7th kind of possible way of realization of the second aspect, the property based on the unit testing for being assigned to multiple mappings
Energy score, selects the member of the subset from the unit testing of the multiple mapping, wherein the unit of the multiple mapping is surveyed
Test point is identical to the Failure Mode of unavailable service.
Therefore, as noted previously, as generating one group of redundancy unit test of identical service availability information when being executed
It is summed up as the least unit testing of required resource, so overall performance can be promoted further.
Detailed description of the invention
Fig. 1 shows the block diagram for the exemplary system being related to when process executes；
Fig. 2 shows the flow charts of the process first stage；
Fig. 3 is according to an exemplary abstract syntax tree that will be used in the first phase；
Fig. 4 a to Fig. 4 c shows the application method that abstract syntax tree shown in Fig. 3 is directed to different target parameter；
Fig. 5 shows the example for selecting redundancy unit to test；
Fig. 6 shows the flow chart of the process second stage；
Fig. 7 shows the exemplary map between unit testing and service that second stage obtains；
Fig. 8 shows several groups of redundancy isomorphism unit testings；
Fig. 9 shows the decision tree for the relationship established based on second stage.
Specific embodiment
Following exemplary process can be realized by the mode of software, hardware or a combination of both, it is therefore intended that soft by reusing
The part development phase effectively diagnoses the system with multiple physical entities, such as cloud for the unit testing of test software application
Platform, wherein this multiple physical entity executes one or more interactive softwares and applies to provide a variety of services.
The process may include three phases, will hereafter be described in more detail in description.In the first stage, superfluous by going to select
Remaining unit testing can reduce the quantity of available cell test.It should be noted that when removal redundancy unit test, it can be true
Determine and ignores incoherent support method.In second stage, it may be determined that all clothes needed for remaining unit testing and system operation
Relationship between business, wherein remaining unit testing can be screened based on determining relationship.In the phase III, decision is produced
Tree carrys out diagnostic system failure.Test result shows that in some cases the available cell that need to only execute 4% to 5% tests just foot
To test the availability of all services.
Fig. 1 shows the exemplary system 10 being related to when the process executes.System 10 includes executing interactive software
Multiple physical entities 12 of (software, SW) application (application, APP) 14.It can be used between SW APP 14 super literary
This transport protocol (Hypertext Transfer Protocol, HTTP), remote procedure call (Remote Procedure
Call, RPC), token, message queue etc. communicated.Interaction SW APP 14 can provide service 16, these 16 pairs of services are
The operation of system 10 is most important.For example, system 10 possibly can not handle client if one of service 16 is unable to operate normally
One or more (or even any one) request at end 18.
System 10 also includes software development infrastructure 20.Software development infrastructure 20 may include mankind software development people
Member can be used to realize, update and the one or more computers and software of maintenance interaction SW APP 14.For example, software development
Infrastructure 20 may include version control database 22.Version control database 22 can be stored in computer, and may include
The modification record of software code and SW APP 14.Particularly, version control database 22 can store current with SW APP 14
In the related all codes of the implementation of physical entity 12.
In addition, software development infrastructure 20 may include code review module 24.Code review module 24 can provide evaluation
The platform of the code of SW APP 14.For example, code review module 24 can be used to accept or reject generation for developer or operator
Code is modified and provides feedback.Code review module 24 can provide web-based graphic user interface (graphical user
Interface, GUI), developer or operator can evaluate code by the GUI.Code form can be one or more people
The readable programming language of class, for example, Java, C#, HTML 5, JavaScript, R, Swift, C++, PHP, Ruby, Python,
Etc..
In addition, software development infrastructure 20 may include code integration module 26.Code integration module 26 can be used for controlling
Code is integrated into version control database (database, DB) 22.For example, all available cells can be performed and survey before integrated
Examination 28 is with the integrality of Validation Code.Unit testing 28 can be used for testing SW APP 14 most in conjunction with associated control data
Small testable unit, also referred to as unit.For example, HTTP request can be transmitted to physical entity 12 in unit testing 28, and from physics reality
Body 12 receives http response.It can based on response judging unit test 28 be by or failure.If unit testing 28 fails,
May not integrate corresponding code (and relative code), meanwhile, software developer be marked with make into
One step inspection.This can be helped in early stage test problems, so that the error-free operation of SW APP 14.
System 10 may also include failure diagnosis unit 30, for determining service 16 whether normal operation.This diagnosis can base
It is periodically carried out in the time frame that operator determines, and the test that can be executed to software development infrastructure 20 supplements,
Not found problem when to find 20 execution unit of software development infrastructure test 28.
Failure diagnosis unit 30 may include unit testing manager 32.Unit testing manager 32 can be used for selecting uncorrelated
And/or redundancy unit testing 28 (with diagnostic service 16 on the whole).Failure diagnosis unit 30 may also include testing service
Mapper 34.Testing service mapper 34 can determine data, when a certain service 16 of data instruction is unavailable, which unit testing
Failure.Before this, testing service mapper 34 can send signal, such as Unix signal to one or more physical entities 12,
So that the one or more physical entity 12 disables (stopping/closing) service 16.
Once service 16 is unavailable, testing service mapper 34 can send to one or more software applications 14 and request, example
Such as HTTP request.This can selectively unit testing 2 repeats for institute.Since each request requires one or more spies
Surely service 16 is currently running, and therefore, when some service 16 is unavailable, can determine which of selected unit testing 28
Unit testing failure.Particularly, a certain unit can be determined by checking (HTTP) response of one or more physical entities 12
Test 28 be by or failure.By unit testing 28 can be classified as be unable to test service 16 availability, and failure
Unit testing 28 is then classified as the availability for capableing of testing service 16.
Which unit testing in the unit testing 28 for determining selection being capable of (and can be used in) test special services
After 16 is unavailable, signal, such as Unix signal can be sent to one or more physical entities 12, so that the one or more object
Reason entity 12 reactivates (starting) service 16.This can be repeated for all or part of service 16.In addition, can will select
Unit testing 28 and service 16 between dependence store into data structure, such as in matrix or array.These according to
Bad relationship can indicate that unit testing 28 determines the ability of special services 16 whether normal operation.
Failure diagnosis unit 30 may also include tracer 36.Tracer 36 can be used data structure and construct and determines
Plan tree selects maximally related unit testing 28, with the service 16 in diagnostic system 10.But decision tree may be only in needs
When can just rebuild, that is, if the set of unit testing 28 of selection changes, be used multiple times identical decision tree to examine
Disconnected system service 16.For example, period (the system load lower period, for example, when night that operator can select in one day
The setting sleep period of section, system 10, or when integrating fresh code) in the diagnosis arranged properly of definition.For example, tracer
36 can send (HTTP) request to SW APP 14, and receive (HTTP) response.
After analyzing the exchange of (HTTP) message, the Trouble Report of instruction 16 (if having) of unavailable service is produced.
If system service 16 is unavailable, attended operation can be taken to it, for example, cancelling the nearest change of code.In addition, in code set
This report, can be sent to code integration module 26 by Cheng Shi, and code integration module 26 can determine that receiving or refusal code become
More.
Fig. 2 shows the flow charts of the first stage of the above process 38, can be executed by unit testing manager 32.
Unit testing manager 32 can receive a series of existing unit testings 28 being written in the development phase and take as input, and to test
The unit testing 28 of business 34 the number of output of mapper reduction, wherein the behaviour of second stage can be performed in testing service mapper 34
Make.As shown in Fig. 2, at 40, unit testing manager 32 can start the first stage by following operation: determination include with can
With source file/module of the relevant code of unit testing 28.
At 42, code analysis can be carried out to the source file/module for including to available cell tests 28 relevant codes, and
It constructs abstract syntax tree (Abstract Syntax Tree, AST).For example, code may include the unit that can call other methods
Test method, hereinafter, these other methods can be described as support method.Support method can call the various letters of SW APP 14
Number.It for determination unit test method and its supports method, AST can be constructed (referring to Kuhn, Thomas and Olivier Thomann
" abstract syntax tree " in Eclipse Corner Articles 20 (2006)).AST can be source code/module to set
Form and existing characterization, wherein the node in AST can indicate building (class, unit testing executed in source code/module
Method supports method).
AST can be traversed to carry out determination unit test method and its support method.For each unit test method, can create
It supports the set of method.AST can be constructed by compiler according to such as a series of step:
● scanning source code reads the position of character stream；
● character is grouped to form token；And
● it is parsed, wherein token is converted into syntactic structure, which can be described as analytic tree.
Then analytic tree is converted into AST.It should be noted that at least part programming language has been supported to construct
AST.For example, the library Python ast.py can be used for constructing AST when using Python.
At 44, the AST of building is screened.Method can be supported to these according to the degree of correlation of determining support method
It gives a mark.For example, if some way is only applicable to language-specific, and may not sent out in the test of the service of system 10 16
Direct effect is waved, then can go to select the support method.It can be based on the rate of falling document (Inverse Document Frequency, IDF)
Method will be supported to be labeled as " correlation " or " uncorrelated ".Therefore, the lower support method of IDF, that is, be present in multiple source files/
Support method in module, can be considered uncorrelated.For example, the IDF of support method (x) can be calculated according to following equation:
IDF (x)=log (# source file/module/# source file/module including support method x) (1)
Therefore, example as shown in connection with fig. 3 as follows is described in more detail described, is present in multiple source file/modules
In support method may have lower IDF.Fig. 3 shows four modules.Module 1 include unit test method U1, U2 with
And the definition of U3.Module 2 includes the definition of unit test method U4, U5 and U6.Module 3 includes unit test method U7, U8
And the definition of U9.Module 4 includes the definition of unit test method U10, U11 and U12.It can determine according to formula (1) following
IDF:
IDF (Assert)=log (4/4)=0
IDF (S1)=log (4/1)=log (4)
IDF (S2)=log (4/2)=log (2)
IDF (S3)=log (4/1)=log (4)
Therefore, S1, S2 and S3 can be labeled as correlations, and " Assert " can then be labeled as uncorrelated and be gone to select.Herein
It is emphasized that the threshold value under different frames may be also not identical.Therefore, it to obtain optimum, can attempt using difference
IDF threshold value and may select from a threshold value, for removing most of or all support methods just for language-specific,
But not removal only removes several usually relevant user defined methods.
With continued reference to Fig. 2, at 46, duplicate removal can be carried out to the support method for being marked as " correlation ".For example, if unit is surveyed
The subset of method for testing has invoked same support method, then it represents that there is the redundancy unit test method that can go choosing.Choosing is gone to grasp
Make to carry out based on modified set covering algorithm.Minimal number of unit testing may be selected in modified set covering algorithm
Method, so that the unit test method of selection calls all that any unit test method in source file/module is called
Hold method.For example, subset { U7, U8 } and { U7, U9 } in module 3 call all support methods in the module in Fig. 3
{ S6, S7, S8 }.It is thereby possible to select any one subset calls all support methods.In addition, set covering algorithm is optional
It selects and executes time shortest subset.
Modified set covering algorithm may relate to following parameter:
● the time: time needed for executing the unit testing in subset.This and set covering algorithm in " cost
(cost) " related, it can help to select most effective unit test method subset.For example, if having more than one unit test method
Subset calls all support methods in a source file/module, then may be selected to execute time shortest subset.
● coverage rate: the minimum vertex-covering rate (unit: %) that support method need to be realized.This refers to that operator can be based on required
The percentage of support method carrys out selecting unit test method.For example, in Fig. 3, in module 3 there are three types of the method for support S6, S7,
S8}.To reach 100% coverage rate, then selection subset { U7, U8 } or { U7, U9 } is needed, to cover all support methods.
To reach 30% coverage rate, selection { U7 } is only needed, because need to only call the support method of one third.
The mathematical definition of modified set covering algorithm can are as follows:
Input:
● support the set of method: S={ s1, s2..., sn}
● a set in m unit testing set: U={ U1, U2..., Um, so that:
●
● desired coverage rate, c ∈ [0,1]
● execute the set the time required to each unit testing: T={ t1, t2..., tm, so that tiFor UiThe execution time.
Modified set covering algorithm can be used for determining setSo that:
●∑tjIt is minimum
●
●j∈J
In addition, modified set covering algorithm can be schematically shown by following pseudocode:
1.Umin←{}
2.
A.for all unit test method U do
Find such a unit test method: for often newly-increased support method, cost α is minimum:
b.Umin←Umin∪U
3. exporting Umin。
The working method of the algorithm indicated with pseudocode is as follows:
1.UminIndicate the set of unit testing.Initial value is empty set.
2. when being unsatisfactory for coverage rate condition (that is, the quantity of the support method in set is less than desired value):
A. each support method of such a unit test method U:U is found, cost α is minimum, wherein α is unit survey
The ratio of method for testing executed between time and the quantity of newly-increased support method.
B. unit test method U is added to UminIn.
3. exporting Umin。
In this regard, Fig. 4 a shows the AST of the module 3 after removing uncorrelated support method (Assert).The AST includes three kinds
Unit test method: U7, U8 and U9.Hereinafter, it is assumed that the execution time of U7 is 15 seconds, the execution time of U8 is 8 seconds, U9's
Executing the time is 9 seconds.If modified set covering algorithm is applied to the AST in Fig. 4 a, to remove redundancy unit test side
Method and reach 100% expectation coverage rate, then all support methods must all occur.In this case, as shown in Figure 4 b, select
U7 and U8, because U7 and U8 cover all different support methods and cost is minimum.Therefore, modified set covering algorithm
Eliminate U9.
If modified set covering algorithm is applied to the AST in Fig. 4 a, to remove redundancy unit test method and reach
To 50% expectation coverage rate, then any two kinds for need to only executing in three kinds of support methods can be (because of 2/3 > 50%).U8 and
U9 includes two different support methods.Since U8 cost is minimum, U8 is selected, goes to select U7 and U9.It needs to illustrate herein
, selected percent value is exemplary only above, and coverage rate can be arranged in operator according to actual needs.
Still referring to Figure 2, at 48, it can remove the redundancy unit test 28 in source file/module.For example, such as Fig. 5 institute
Show, when being related to module 2 and module 4 (in Fig. 3), the unit test method U5 in module 2 calls support method { S4, S5 }.
Similarly, module 4 includes unit test method U10, U11 and U12, calls support method set { S4 }, { S10, S4 } respectively
And { S12, S5 }.Support method goes Baryon-system to ensure that the subset of the unit test method in module 4 calls support method
{ S4, S5, S10, S12 }.This means that module 4 has covered all support methods (that is, { S4, S5 }) of U5 calling.Therefore,
U5 is redundancy, can go to select.This method can be applied to all unit test methods, to remove in entire source file/module
Redundancy unit test method.
The flow chart of the second stage of the above process can be completed by testing service mapper 34, as shown in Figure 6.Testing service
Mapper 34 can receive the data about the unit testing 28 after relevant and duplicate removal from unit testing manager 32.52
Place, testing service mapper 34 can determine the relationship between the unit testing 28 received and service 16.The relationship can be based on single
Member test detects the ability of special services 16 whether normal operation to determine.Following step can be repeated for all services 16
It is rapid:
● disabling (stopping/closing) services s.The case where this analog service 16 is broken down.
● the set of the unit testing 28 of operation quantity reduction.Unit testing 28 dependent on service s will will fail.
Unit testing 28 independent of service s will pass through.It can record this result.
● reactivate (starting) service s.
It may can only once disable a service 16.If once disabling more than one service 16 and a unit testing mistake
It loses, then can not judge which service 16 causes unit testing 28 to fail.Therefore, it can rise and close for the operation of all pairs of systems 10
The service 16 of key effect repeats above-mentioned steps.Can based on test run as a result, establish unit testing 28 and service 16 it
Between relationship, and indicated with matrix, as shown in Figure 7.
In Fig. 7, each grid indicates the relationship between a unit testing 28 and a service 16.The relationship can pass through
(Ui, service j) indicate that wherein Ui indicates unit testing 28 to expression formula R, and service j indicates service 16.(Ui services j)=0 table to R
Show the Ui failure when service j is unavailable.If executing Ui in system 10, and successfully pass, it may be considered that service j is normally transported
Row.This means that Ui is dependent on service j.But if Ui fails, servicing j may be unavailable, although can not be absolutely
Determine that service j is unavailable in this case.
(Ui, service j)=1 indicate that Ui passes through after servicing j disabling to R.Therefore, Ui is not rely on service j.In other words,
It is not available Ui and comes whether testing service j can be used.Therefore, executing the unit testing 28 can not clearly judge that servicing j whether may be used
With.For example, if U5 fails, service 2, service 3, service 4 or service 5 may be unavailable, but nothing as shown in the matrix of Fig. 7
Method judges whether service 1 can be used.But if U5 passes through, then it represents that service 2, service 3, service 4 and service 5 are available.These are closed
System can be used for constructing decision tree in the phase III.Before this, in the 54 of Fig. 6, list can be further reduced by screening
The quantity of member test 28.
As shown in fig. 7, each unit testing 28 can be indicated with a kind of binary-coded character string pattern, the binary-coded character
String pattern is from indicating that each row (except the row for indicating the execution time) of service 16 derives from.For example, U1 is indicated with 01010, U2
It is indicated, etc. with 11101.The identical unit testing 28 of string of binary characters can be described as isomorphism test.That is, isomorphism test and service
Relationship between 16 is identical, therefore can detect the availability of identical service 16.Therefore, isomorphism test is redundancy testing.In order to
These redundancy testings are removed, isomorphism unit testing 28 can be classified as one group, and therefrom select a unit testing 28 (for example, choosing
Select and execute time least unit testing 28), as shown in Figure 8.
For example, the execution time of unit testing U1 can be 10 seconds, the execution time of unit testing U2 can be 9 seconds, and unit is surveyed
The execution time for trying U3 can be 13 seconds, and the execution time of unit testing U4 can be 19 seconds, and the execution time of unit testing U5 can be 3
Second.Therefore, unit testing U1, U5 and U2 may be selected.
Selected cell test just can be used for constructing decision tree in the phase III only in second stage.In the first two
The quantity in stage, unit testing 28 is reduced, and is established between unit testing 28 and the service 16 of the offer of system 10 to be diagnosed
Relationship.In the phase III, these relationships can be used for constructing decision tree.It decision tree can be used to carry out detection system 10 operating normally
In failed services 16, that is, the failed services 16 concurrent with the service request of client 18.
The tree construction of decision tree can be that each of which internal node tests an attribute, and each branch corresponds to an attribute value,
Each leaf node distributes a classification or label.Path from root node to leaf node can presentation class rule.In such case
Under, each internal node can indicate select in second stage unit testing 28.Each branch can indicate that a unit is surveyed
Examination 28 as a result, every leaf can indicate one or more unavailable services 16.
Decision tree can be constructed by sequential fault diagnosis.This can be based on attribute information Gain generating decision tree.Each list
The information gain of member test can be calculated by the following formula:
Information gain=unit testing testability/unit testing execution time (2)
The quantity for the non-detection service that testability=unit testing of unit testing can test out
Relationship shown in Fig. 9 is given, formula (2) can be used to calculate information gain, as follows:
Information gain: U1=1/8；
Information gain: U2=2/7；
Information gain: U3=2/5；
Information gain: U4=2/4.
The highest unit testing 28 of information gain may be selected, and it (is herein U4) institute that node can be added to unit testing 28
Decision tree in.Then, node corresponding with the result of the unit testing 28 can be added in decision tree.Such as second stage
Associated description, the relationship of unit testing Ui be R (Ui, service j)=0, if Ui passes through, then it is assumed that service j is operated normally.The pass
System is for constructing decision tree.After selecting the unit testing, two branches can be created in decision tree.U4's can be indicated by branch
Service 1 and service 2 operate normally.Therefore, for next node, service 3 is considered as the service not detected.For passing through on U4
Branch, the information gain of U1, U2 and U3 are respectively 1/8,1/7 and 1/5.
Due to the information gain highest of U3, so U3 may be selected as next node.Based on above-mentioned relation, if U3 passes through,
Then it is believed that service 1 and service 3 are available.Previous node has determined that service 1 and service 2 operate normally.Therefore, the node is logical
It crosses branch and determines that all services operate normally.But if the test crash, any one of service 16 service can be indicated
It is likely to unavailable.Since previous node has determined that service 1 and service 2 are normal, so the failure branch of the node indicates
Service 3 is in malfunction.The repeatable process that executes is with the other parts in decision tree shown in structure figures 9.The decision tree packet
Include four attribute, these four attribute representatives are shown as four unit testings 28 of internal node.Road from root node to leaf node
Diameter indicates testing sequence, wherein leaf node indicates failed services 16.It, can be based on unit testing 28 to diagnose operating system 10
As a result decision tree is traversed, until reaching leaf node.
For example, the decision tree in Fig. 9 is since U4.If execution unit test U4 simultaneously passes through, move on to including the next of U3
A node, execution unit test U3.If U3 fails at this time, service 3 can be indicated in malfunction；If unit testing U3 is logical
It crosses, then can indicate that all services 16 all operate normally.However, as previously mentioned, perhaps more than one may service in certain situations
16 are in malfunction.For example, if U4 fails, the node possible breakdown including U3.U3 failure then may cause including U2's
Node failure.Similarly, U2 failure may cause U1 failure.If U1 passes through, can indicate service 1 or service 2 it is unavailable or
Service 1 and service 2 are all unavailable.
In the example depicted in fig. 9, under normal circumstances, it is only necessary to which running log (n) (approximate number) a unit testing can be detected
Failed services, without running n unit testing, n is the unit testing sum after removing isomorphism unit testing.Therefore, decision
Tree allows users to detect unavailable service 16 without executing all available cell tests 28.
Claims (19)
1. a kind of computer-readable medium, which is characterized in that storage is for analyzing the system with multiple physical entities
Computer-readable instruction, the multiple physical entity executes one or more interactive softwares and applies to provide a variety of services, hold
The row computer-readable instruction can be such that the system performs the following operation:
First service is closed, and executes the unit testing of selection and is led to the failure with determination since the first service is unavailable
First unit test；
Second service is closed, and execute the selected unit testing to cause to lose since the second service is unavailable with determination
The second unit test lost；And
Start the service, and execute the subset of first and second unit testing to analyze the system, wherein the son
The member of collection is in the process of implementation based in front unit test being failure or by being dynamically determined in the subset.
2. computer-readable medium according to claim 1, which is characterized in that the unit testing of the execution corresponds to certainly
The node on path in plan tree, the leaf in the decision tree indicate the unavailable service in the system.
3. computer-readable medium according to claim 2, which is characterized in that the node on the path is according to described right
The ratio of stock number needed for the quantity and the execution unit testing of the unavailable service that the unit testing answered is able to detect that
To be ranked up.
4. computer-readable medium as claimed in any of claims 1 to 3, which is characterized in that be stored with computer
Readable instruction, executing the computer-readable instruction can be such that the system performs the following operation:
The function called to available cell test assigns relevance score；And
The available cell gone to select in following situations is tested:
Relevance score is only called to be lower than the function that is ignored of threshold value；Or
Relevance score is only called to be greater than or equal to the function of the threshold value, the function is also surveyed by the selected unit simultaneously
Other unit testings in examination are called.
5. computer-readable medium according to claim 4, which is characterized in that the relevance score is based on unit testing
The number of the function is called in the file that the unit testing in set is encoded.
6. computer-readable medium according to claim 5, which is characterized in that the list in the unit testing set
Call the number of the function more in the file that member test is encoded, the relevance score is lower.
7. computer-readable medium as claimed in any of claims 1 to 6, which is characterized in that be stored with computer
Readable instruction, executing the computer-readable instruction can be such that the system performs the following operation:
It goes that available cell is selected to test, wherein available cell test is only called also by its in the selected unit testing
The function that its unit testing is called.
8. computer-readable medium according to any one of claims 4 to 7, which is characterized in that be stored with computer
Readable instruction, wherein executing the computer-readable instruction can be such that the system performs the following operation:
It goes that performance scores is selected to test lower than the available cell of other unit testings.
9. computer-readable medium as claimed in any of claims 1 to 8, which is characterized in that be stored with computer
Readable instruction, wherein executing the computer-readable instruction can be such that the system performs the following operation:
Based on the performance scores for being assigned to redundancy unit test, from the redundancy unit test in first and second unit testing
The member of the middle determination subset, wherein the redundancy unit testing needle is identical to the Failure Mode of unavailable service.
10. computer-readable medium according to claim 9, which is characterized in that the performance scores are based on the redundancy
The execution time of unit testing.
11. a kind of communication network, which is characterized in that including calculating equipment, the calculating equipment includes processor and according to power
Benefit require any one of 1 to 10 described in computer-readable medium.
12. a kind of method analyzed the system with multiple physical entities, wherein the multiple physical entity executes friendship
Mutual software application is to provide a variety of services, which is characterized in that the described method includes:
For software application determination unit test, wherein at least partly described software application has been assigned more than one unit
Test, if some using unavailable, distributes to the unit testing failure of the application；
The unit testing of the determination is mapped to a variety of services to establish data structure, wherein the data structure refers to
Which unit testing failure when a certain service is unavailable shown；And
The subset of the unit testing of the mapping is executed to analyze the availability of the service in the system, wherein the subset
Member in the process of implementation based in front unit test being failure or by being dynamically determined in the subset.
13. according to the method for claim 12, which is characterized in that the unit testing of the execution corresponds in decision tree
Node on path, the leaf in the decision tree indicate the unavailable service in the system.
14. according to the method for claim 13, which is characterized in that the node on the path is according to the corresponding unit
The ratio for the quantity of the unavailable service and stock number needed for the execution unit testing being able to detect that is tested to be arranged
Sequence.
15. method described in any one of 2 to 14 according to claim 1, which is characterized in that determined for the software application single
Member test further include:
It determines to the function call by available cell test execution；
Relevance score is assigned to the function；
Ignore and only calls relevance score lower than the unit testing of the function of threshold value；And
Ignore only call relevance score be greater than or equal to the threshold value function unit testing, and the function simultaneously also by
Other unit testings in the unit testing of the determination are called.
16. according to the method for claim 15, which is characterized in that the relevance score is tested based on the available cell
The middle number for calling the function.
17. method described in any one of 2 to 16 according to claim 1, which is characterized in that determined for the software application single
Member is tested
It determines to the function call by available cell test execution；And
Ignore and the available cell of the function also called by other unit testings in the unit testing of the determination is only called to test.
18. method described in any one of 5 to 17 according to claim 1, which is characterized in that for calling the list of same functions
Member test ignores unit testing of the performance scores lower than other unit testings to carry out duplicate removal.
19. method described in any one of 2 to 18 according to claim 1, which is characterized in that based on being assigned to multiple mappings
The performance scores of unit testing select the member of the subset, wherein the multiple from the unit testing of the multiple mapping
The unit testing of mapping is identical for the Failure Mode of unavailable service.
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| PCT/EP2017/060360 WO2018202275A1 (en) | 2017-05-02 | 2017-05-02 | Unit test selection for system analysis |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109416653A true CN109416653A (en) | 2019-03-01 |
| CN109416653B CN109416653B (en) | 2020-11-27 |
Family
ID=58668888
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201780024091.4A Active CN109416653B (en) | 2017-05-02 | 2017-05-02 | Selecting unit tests for system analysis |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN109416653B (en) |
| WO (1) | WO2018202275A1 (en) |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9032361B2 (en) * | 2011-12-15 | 2015-05-12 | Tata Consultancy Services Limited | Agile unit and regression testing framework for domain specific languages |
| CN104899141A (en) * | 2015-06-05 | 2015-09-09 | 北京航空航天大学 | Test case selecting and expanding method facing network application system |
-
2017
- 2017-05-02 CN CN201780024091.4A patent/CN109416653B/en active Active
- 2017-05-02 WO PCT/EP2017/060360 patent/WO2018202275A1/en active Application Filing
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9032361B2 (en) * | 2011-12-15 | 2015-05-12 | Tata Consultancy Services Limited | Agile unit and regression testing framework for domain specific languages |
| CN104899141A (en) * | 2015-06-05 | 2015-09-09 | 北京航空航天大学 | Test case selecting and expanding method facing network application system |
Non-Patent Citations (1)
| Title |
|---|
| 王小丽: "软件回归测试用例选取方法与研究", 《空间控制技术与应用》 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109416653B (en) | 2020-11-27 |
| WO2018202275A1 (en) | 2018-11-08 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Ma et al. | Using service dependency graph to analyze and test microservices | |
| US10146676B1 (en) | Multi-version regression tester for source code | |
| US7480602B2 (en) | System verification test using a behavior model | |
| Yoo et al. | Regression testing minimization, selection and prioritization: a survey | |
| US7458064B2 (en) | Methods and apparatus for generating a work item in a bug tracking system | |
| US20090271351A1 (en) | Rules engine test harness | |
| CN110765018B (en) | Automatic interface testing method and equipment | |
| WO2006115937A2 (en) | System review toolset and method | |
| Porter et al. | Skoll: A process and infrastructure for distributed continuous quality assurance | |
| TWI626538B (en) | Infrastructure rule generation | |
| CA3165869A1 (en) | Onboarding of monitoring tools | |
| Zoppi et al. | Context-awareness to improve anomaly detection in dynamic service oriented architectures | |
| Olianas et al. | MATTER: A tool for generating end-to-end IoT test scripts | |
| CN114579467A (en) | Smoking test system and method based on release subscription mechanism | |
| CN112559343A (en) | Test path generation method and related equipment | |
| Fagerström et al. | Verdict machinery: On the need to automatically make sense of test results | |
| Malik et al. | An analysis of performance testing in distributed software applications | |
| Babac et al. | AgentTest: A specification language for agent-based system testing | |
| US8341610B2 (en) | Method and apparatus for authoring and optimizing flowcharts | |
| CN109416653A (en) | Selecting unit test carries out network analysis | |
| Jebbar et al. | Architecture for the automation of live testing of cloud systems | |
| Badih et al. | Static Analysis Framework Based on Multi-Agent System | |
| Kanstrén | A framework for observation-based modelling in model-based testing | |
| Gupta et al. | Hybrid regression testing technique: based on requirement priorities, fault and modification history | |
| Dobuneh et al. | Clustering test cases in web application regression testing using self-organizing maps |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| TR01 | Transfer of patent right | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20220211 Address after: 550025 Huawei cloud data center, jiaoxinggong Road, Qianzhong Avenue, Gui'an New District, Guiyang City, Guizhou Province Patentee after: Huawei Cloud Computing Technologies Co.,Ltd. Address before: 518129 Bantian HUAWEI headquarters office building, Longgang District, Guangdong, Shenzhen Patentee before: HUAWEI TECHNOLOGIES Co.,Ltd. |