WO2023041187A1

WO2023041187A1 - System for testing functionalities of a computer application

Info

Publication number: WO2023041187A1
Application number: PCT/EP2021/075832
Authority: WO
Inventors: Hamza SAYAH; Bastien BOUFFAUT
Original assignee: Circle Internet Services Inc.
Priority date: 2021-09-20
Filing date: 2021-09-20
Publication date: 2023-03-23

Abstract

The invention relates to a system (1) for testing functionalities of a computer application, comprising: -a digital processing device (2) for retrieving a source code, and for: a) generating a list of functions to be tested by: -defining a graph of calls of all of the functions and by generating a call-frequency database; -generating a call probability score; -classifying each function by semantic theme; -generating an interlinking score for each of said functions, depending on scope; -determining a list of most commonly used words; -generating a combined score for each function; -selecting some of the functions of the source code using their combined score; -b) generating a list of test values for the listed functions, by: -analysing operations carried out on the parameters; -listing exemplary values of the parameters; -extrapolating other parameters by applying variations to the exemplary values.

Description

Title of the invention: System for testing the functionalities of a computer application

[0001] [The invention relates to computer developments, and in particular tools for detecting malfunctions of a computer application under development.

[0002] With a view to anticipating development problems, it is known to carry out unit tests of a computer application. To this end, parameter values are injected as input data into functions of the computer application. It is then determined whether the execution of the application or the returned values are causing problems.

[0003] In practice, and in particular for complex computer applications developed as a team, the unit tests can relate only to part of the application and only to a limited set of parameter values. Indeed, it turns out to be unrealistic to carry out unit tests for the entire application and for exhaustive parameter values. Therefore, developers perform unit tests on really limited samples of the application and settings. Moreover, the choice of the functions tested and the values of test parameters proves to be truly empirical. Unit tests are therefore not optimal both in terms of reliability and in terms of implementation time.

[0004] Moreover, the evolutions of a computer application are sometimes made starting from a healthy application, with new interlocutors who can introduce harmful mutations of the application. Anticipating such changes and their consequences is tricky.

The invention aims to solve one or more of these drawbacks. The invention thus relates to a system for testing the functionalities of a computer application as defined in claim 1.

[0006] The invention also relates to the variants of the dependent claims. Those skilled in the art will understand that each of the features of the dependent claims or of the description may be combined independently to the above characteristics, without however constituting an intermediate generalization.

Other characteristics and advantages of the invention will emerge clearly from the description which is made of it below, by way of indication and in no way limiting, with reference to the appended drawings, in which:

[0008] [Fig.1] is a schematic representation of an example of a system for implementing the invention;

[0009] [Fig.2] is a schematic representation of major categories of steps implemented in a system for implementing the invention;

[0010] [Fig.3] is an example of a graph, the graph in Figure 3 represents a call graph for a registration service for an identification email and a password.

[0011] Figure 1 schematically illustrates a functionality test system of a computer application 1, according to an example implementation of the invention. The test system 1 is configured for the implementation of unit tests of a computer application, during development or before its commissioning.

The test system 1 here comprises a processing device 2 and a database 20. The processing device 2 typically be implemented in the form of a computer server accessible by a computer network 3, typically the Internet. The database 20 is accessible by the processing device 2. Examples of the content of the database 20 will be detailed below.

[0013] The processing device 2 can be implemented in the form of a computer server accessible by the network 3. For this purpose, the user here has a terminal 4 connected to the processing device 2 by the intermediary of the computer network 3. The terminal 4 is configured to display a certain amount of information on a screen of the user. The terminal 4 can for example have an Internet browser communicating with the processing device 2 and displaying a user interface 6. The user can have an input interface such as a mouse 5, in order to provide answers At Internet browser, then transmitted to the processing device 2. The terminal 4 also has access to a computer application under development or to be tested, stored on a storage medium 7. The storage medium 7 can be a local storage of the terminal 4 or remote storage, for example on storage servers accessible online. The processing device 2 can be configured to send display information to the terminal 4, for example in the form of HTML content.

[0014] The digital processing device 2 is configured to recover the source code of a computer application. The source code of the computer application can also be stored in the database 20, so that the device 2 can be configured to both provide access to the source code and implement unit tests.

[0015] FIG. 2 illustrates the main categories of steps that can be implemented by the processing device 2 during tests or preparations for tests of the source code of the computer application. Category 100 corresponds to processes for generating a list of functions to be tested. Category 200 corresponds to processes for generating a list of test values for the listed functions. Category 300 corresponds to the testing processes of the source code itself.

The digital processing device 2 is configured to analyze the source code of the computer application to be tested in order to define an appropriate list of functions to be tested, which corresponds to the category 100 mentioned above. An appropriate list of functions corresponds to a selection of source code functions to be tested, i.e. a limited number of functions among all the source code functions. To this end, according to the invention, the processing device 2 performs the following operations to generate the list of functions to be tested:

[0017] -generate a call probability score for each function;

[0018] -generate a failure impact score for each function;

[0019] to generate a nesting score for each function; [0020] to generate an atypicality score.

[0021] The generation of the call probability score for each function can be performed as follows:

[0022] -defining a call graph of all the functions of the computer application and generation of a database of the frequency of calls for each of the functions of the call graph. For each function 'function', we can associate the call frequency 'fafonction' in such a database. The calling frequency database may be stored in database 20;

[0023] to generate a call probability score for each function, the score depending on the call graph and the call frequency stored in the database for this function. A call probability score for a function 'function' will be referenced 'pafonction' thereafter. A corresponding example is illustrated with reference to FIG. 3. In the example, the graph of FIG. 3 represents a graph of calls for a service for registering an identification email and a password. Function 101 corresponds to reading input data from a user. Function 102 is a function for verifying the validity of an email entered. Function 103 is a password strength validation function. Function 104 is an input data conformity validation function. Function 105 is a registration function in an identifier database. Functions 102 and 103 are executed only if the function for reading input data 101 has been carried out correctly. For a given frequency 'fa10T of call of function 101, the respective call probabilities 'pa102' and 'pa103' of functions 102 and 103 is less than 'fa10T. The conformity validation function 104 is executed only if the functions 102 and 103 have been carried out correctly. Therefore, the probability of calling 'pa104' is lower than the probabilities of calling 'pa102' and 'pa103' of functions 102 and 103. The registration function 105 is only executed if the function 104 has been carried out properly. Therefore, the call probability 'pa105' is less than the call probability 'pa104'; The generation of the failure impact score of each function can be carried out as follows: each function is classified by semantic theme. To this end, a semantic analysis of each function is implemented to generate a failure impact score for this function as a function of its semantic theme. For example, a function whose semantic analysis reveals semantic themes of authentication, certificate management or encryption will be considered as having a very high failure impact score for a computer application processing bank payments. For another example, for a computer application of a service provider, the semantic analysis of the source code can bring out for each function, a context linked to the service, a context linked to invoicing, a context linked to the support and a display context. According to a criticality which can be defined by the developer or the user for each context, the semantic analysis can generate a criticality score for each of the functions according to the identified semantic contexts.

The generation of the nesting score for each function is carried out for each function according to the scope or level of nesting of this function (nesting level in English). An example of nesting score calculation for a reference function is detailed at "https://gist.github.com/avmnu-sng/790f995091d209c00dcf2170352bebbd". The score is detailed here as a source code cognitive complexity score. The nesting score is representative of the difficulty of understanding a function due to its level of nesting.

[0026] The generation of the atypicality score makes it possible to take the difficulty of debugging for functions which are outside the usual context of the application. The generation of the atypicality score can be implemented as follows. A list of the most used words in the whole of the source code is determined, then a list of words used for each of the functions is generated. An atypicality score is generated, depending on the semantic mismatch between words in a function's wordlist compared to the list of most used words in the entire source code. For example, in a 3D management application, the majority vocabulary is related to graphics (shapes, light, camera angle, etc.). However, such applications most often include a module related to the laws of physics in order to be able to generate a realistic animation. Such a module can include a lot of vocabulary related to the theme of mechanics, which is a significant shift from the theme of 3D.

In order to be able to simply and objectively make a selection to be tested from among all the functions of the source code, the processing device 2 generates a combined score for each function of the source code. The combined score notably combines the call probability score, the failure impact score, the nesting score and the atypicality score, for example by simply adding these scores or by applying weightings.

Thus, a score is digitally generated for each function, allowing a digital processing system to establish a hierarchy of functions to be tested, and thus a reduced list of functions to be tested.

The invention makes it possible to optimize the efficiency of the unit tests carried out, that is to say that for a given number of unit tests carried out, the level of criticality of the errors detected will be optimal. With limited test resources, optimal test reliability can thus be achieved.

[0030] Advantageously, the processing device 2 is configured to generate the list of functions to be tested, by:

- identifying the presence of documentation of each function in the source code and generating a documentation score for the function based on the identification of the presence of such documentation. The presence of documentation for a function is indeed a factor favoring the resumption of the computer application by another developer or by the initial developer himself;

-by generating the combined score of each source code function taking into account the documentation score.

Advantageously, the generation of the atypicality score is implemented as follows. The selection of names of the most used words in the source code includes the listing of all the words of the source code and the elimination of the names of usual operators of a computer language. The usual operators are therefore not taken into account to calculate a semantic shift with the words of the different functions.

The processing device 2 is further configured to generate a list of test values of the listed functions. The generation of the list of test values corresponds to the category 200 mentioned previously. The generation of a list of test values according to the invention can be implemented as follows:

[0033] the operations carried out on the parameters of the functions of the source code are analyzed, by identifying the structures of parameters compatible with these operations, and by constructing examples of parameter values corresponding to the structures identified.

[0034] - examples of source code parameter values are listed;

[0035] - other parameter values are extrapolated by applying variations to example parameter values in the source code.

For example, for the following function:

[0037] defcheck(email):

[0038] if(re.match(regex, email)):

[0039] print("Valid Email")

[0040] else:

[0041 ] print("lnvalid Email")

The function re. match only applies to character strings and therefore we deduce that the 'email' parameter is a character string. We have therefore identified a parameter structure compatible with the 'email' parameter.

[0043] By a semantic analysis, it is also determined that 'email' is undoubtedly information in the form of an email. Then, other values of the parameters are extrapolated by applying variations to the sample parameter values in the source code. For example, the following values can be extrapolated for the 'email' parameter: user@domain.com, user123@domain.com, User@domain.com, user@domain.co.uk, @domain.com, or user. domain.com The most common and most used compatible parameter structures can for example be integers, real numbers, character strings, dates, arrays, dictionaries or instances of classes.

Advantageously, the identification of compatible parameter structures comprises the association of a weighting with each compatible parameter structure. The number of values for each compatible structure in the list of values is thus determined according to this weighting, so that the test values correspond statistically to the most probable data structures.

Advantageously, the identification of compatible parameter structures is based on a semantic analysis of the terms close to each call to a function. Compatible parameter structures are thus determined by taking into account the terms used near the calls of each function in the source code.

Prior to the generation of values, we can detect, thanks to the semantics, the internal calls which must be the subject of mocks and we can then treat them as inputs. In object-oriented programming, mocks (simulacra or mock object) are simulated objects that reproduce the behavior of real objects in a controlled way. A programmer creates a mock in order to test the behavior of other objects, real, but related to an inaccessible or unimplemented object. The latter is then replaced by a mock.

[0048] In a unit test, mocks can simulate the behavior of real and complex objects and are useful in this respect when these real objects are impossible to use. The following cases can be cited:

- Replace non-deterministic behavior (time or ambient temperature);

-If the object has states that are difficult to reproduce (a network error for example); - If the initialization of the object is long (for example: creation of a database);

-If the object does not exist or if its behavior can still change; -If it is necessary to include attributes and methods for testing purposes only.

The processing device 2 is also configured to implement a process for testing the source code itself. The implementation of the tests corresponds to the category 300 mentioned above. The computer application tests are implemented with execution of the source code to test the list of generated functions, with the list of parameter values generated, and thus benefit from the selections made to have maximum efficiency of these tests. The processing device 2 is advantageously configured to memorize the output values of the listed functions for the listed input values and to memorize the internal calls. Such storage makes it possible to carry out an a posteriori analysis of the execution of the computer application.

[0050] Advantageously, the source code testing process includes a mutation module. The mutation module is configured to generate mutations of the source code to be tested. The processing device 2 is then configured to execute the mutated source code for the list of functions and the list of listed values. The processing device 2 is further configured to compare the output values of the listed functions of the original source code and the mutated source code. Such a mutation test makes it possible to observe the influence of mutations of the code on its execution, its operation and its results.

[0051] Advantageously, it is possible to set up a filtering of the assertions and of the tests at the end of the process before carrying out the unit tests. For example, we can keep the minimal subset of test cases and assertions that cover all lines of code and eliminate all generated mutants. This filtering optimizes test execution costs and the readability of unit tests.

Claims

[Claim 1] [System (1) for testing the functionalities of a computer application, characterized in that it comprises:

-a digital processing device (2) configured to retrieve source code from a computer application, and to: a) generate a list of functions to be tested by:

- defining a call graph of all the functions of the application and generating a database of the frequency of calls for each of the functions of the call graph;

-generating a call probability score for each function, the score depending on the call graph and the call frequency stored in the database for this function;

- classifying each function by semantic theme by semantic analysis of this function, and by generating a failure impact score for this function according to its semantic theme;

-generating a nesting score for each of said functions, depending on the scope or level of nesting of this function;

-determining a list of the most used words in the whole of the source code, by generating a list of the words used for each of said functions, and by generating an atypicality score depending on the semantic shift between the words of the list of words of each function in relation to the list of the most used words;

-generating a combined score for each function of the source code, the combined score combining the call probability score, the failure impact score, the nesting score and the atypicality score;

- selecting part of the source code functions by their combined score to generate the list of selected functions;

-b) generate a list of test values for each of the listed functions, by: -analyzing operations performed on function parameters in the source code, identifying parameter structures compatible with these operations, and building example parameter values corresponding to the identified structures;

- listing examples of parameter values in the source code; - extrapolating other parameters by applying variations to example parameter values in the source code.

[Claim 2] System for testing the functions of a computer application according to claim 1, in which the digital processing device is configured to generate the list of functions to be tested by: - identifying the presence of documentation of each function in source code and generating a feature documentation score based on the identification of the presence of documentation;

-generating said combined score for each function of the source code by combining the documentation score.

[Claim 3] A computer application functionality testing system according to claim 1 or 2, wherein selecting most used words in the source code includes listing all source code words and eliminating names of usual operators of a computer language.

[Claim 4] A computer application functionality testing system according to any preceding claim, wherein identifying compatible parameter structures includes associating a weight to each compatible parameter structure, the number of values for each compatible structure in the list of values being determined based on this weighting.

[Claim 5] A computer application functionality testing system according to any preceding claim, wherein the identification of compatible parameter structures includes a semantic analysis of terms near each function, the compatible parameter structures being determined according to the terms used near each function in the source code.

[Claim 6] A computer application functionality testing system according to any preceding claim, wherein the digital processing device is configured to execute source code to test said list of functions with said list of values, the digital processing device being further configured to store output values of listed functions for listed input values, and configured to log internal calls.

[Claim 7] A computer application functionality testing system according to claim 6, wherein the digital processing device is configured to generate mutations of said source code, the digital processing device being further configured to execute the source code mutated for said list of functions with said list of values, and further configured to compare the output values of the listed functions of the source code and the mutated source code.

[Claim 8] A computer application functionality testing system according to claim 6 or 7, wherein the digital processing device is configured to select from said list of values based on a system optimization of linear programming.

[Claim 9] A computer application functionality testing system according to any preceding claim, wherein said compatible parameter structures include integers, real numbers, character strings and dates.