WO2012049376A1

WO2012049376A1 - Automation of application tests for mobile telephones

Info

Publication number: WO2012049376A1
Application number: PCT/FR2010/052195
Authority: WO
Inventors: Youssef Ridene; Jean-Gilles Hoursiangou
Original assignee: Neomades
Priority date: 2010-10-15
Filing date: 2010-10-15
Publication date: 2012-04-19

Abstract

The subject matter of the invention is a method of automating tests of an application on several mobile telephones (12) grouped together in a test bench (10), characterized in that it consists in creating at the level of the test bench a database cataloguing the mobile telephones (12) of the test bench and certain of their properties, in generating at the level of a computer (18) a single script comprising at least one interaction and generally a series of interactions, at least the verification of a reaction and of the points of variability specifying a type of mobile telephones to which certain interactions will be applied, in transmitting the unique script to the test bench (10), the test bench transmitting to each mobile telephone to be tested a dedicated script while deleting or not deleting the interaction or interactions contained in the unique script while correlating the information rearding the points of variability and those contained in the database, and finally in inputting and transmitting to the computer (18) for verification the reaction or reactions.

Description

AUTOMATION OF APPLICATION TESTS FOR MOBILE PHONES

The present invention relates to a method for automating tests of an application intended for mobile telephony.

There are a large number of mobile phone variants with quite disparate commands from one model to another. Some may only include a touch screen, others may combine a touch screen with a small number of keys, others include a traditional phone keypad with a dozen keys, still others a keypad-like keyboard. 'computer.

In the same way, the means for moving a pointer at the screen are also disparate from one model to another. Thus, a touch screen, keys in four directions or scrolling systems can be used to move the pointer.

In another aspect, the mobile phones may include a large number of variants concerning the output interfaces such as screens, for example, which may have different dimensions and / or resolutions. Unlike computers with limited operating systems, there are more operating systems, especially for "smartphones," which make up a significant portion of the telephony market.

In addition, mobile phones, apart from the traditional telephony functions, can be equipped with a wide variety of hardware devices e † / or software, for example image capture, video, sound, proximity communication or other devices making the variability wider.

According to another aspect of this variability, a mobile phone model may have different characteristics from one operator to another, operators may require that certain applications are implemented from the outset for phones offered to their customers.

Finally, a mobile phone may have a different behavior for some applications depending on the features used.

Given this great variability of mobile phones, the deployment of a new application on many of these terminals requires a heavy investment heavily impacted by the tests that must be performed to validate that the application works properly on a consistent list of mobile phones.

To limit these costs, there are simulators to validate certain parts of the application code, such as for example the simulator "Wireless Toolkit" (WTK) proposed by the company Sun.

However, given the large number of variants, these simulators may have shortcomings and not perform well for some tests.

Thus, the manual tests on all the real mobile phones targeted remain the most convincing tests.

According to one method, to carry out a test of an application, the developer must buy the mobile phones, write the different scripts corresponding to the same test according to the mobile phones and their environments, then perform on each mobile phone the test or tests manually .

To keep costs down, there are companies in countries with low-cost labor that have mobile phones on which the application needs to be tested and that offer developers the ability to manually run the tests. In this case, the developer indicates by transmitting a test plan to the third party tester the action to be performed on the mobile phone and the latter tells him in return the result obtained.

This solution reduces costs because it is no longer necessary to invest locally in a wide range of phones.

However, it is not fully satisfactory in terms of quality.

Indeed, performing tests on different mobile phones is a repetitive and tedious task that is not very motivating and that leads quickly to a decrease in attention of the tester.

According to another problem, the language barrier between the developer and the tester can cause interpretation problems and therefore errors during testing. In the same way, the verification of a displayed text namely the detection of spelling or grammatical errors can be problematic and be erected by the language skills of the tester.

According to another point, it can be difficult for the developer to describe in a precise and exhaustive manner an action to be performed, especially if the action consists in touching a part of the screen.

Finally, the evaluation of the result that the tester must transmit to the developer may in some cases not be obvious and subjective. For example, whether an image is correctly displayed or the sound is melodious is subjective and depends on the tester.

Also, there is a real need to improve the quality and reduce the costs of testing an application for telephony.

For this purpose, the invention proposes a method for automating tests of an application on several mobile phones grouped together in a test bench, characterized in that it consists in creating at the level of the test bench a database listing mobile phones test bench and some of their properties, to generate at the computer level a single script with at least one interaction and usually a series of interactions, at least the verification of a reaction and points of variability specifying a type of mobile phones to which certain interactions will be applied, to transmit the unique script to the test bench, the test bench transmitting to each mobile phone to test a dedicated script by deleting or not the interaction or interactions contained in the unique script by correlating the information of the points of variability and those contained in the database, and finally to enter and transmit to the computer for verification the reaction (s).

Other features and advantages will emerge from the following description of the invention, a description given by way of example only, with reference to the appended drawings in which:

FIG. 1 is a diagram illustrating an embodiment of a test bench for applications intended for telephony,

FIG. 2 is a diagram of part of the test bench of FIG. 1,

FIG. 3 is a diagram illustrating a meta-model used to define test scripts,

FIG. 4 is a screen preview illustrating the definition of a script,

FIG. 5 is an overview of a menu making it possible to specify the actions that can be performed during a test,

FIG. 6 is a preview of a menu making it possible to specify the type of results of a verification point,

FIG. 7 is a preview of a menu making it possible to choose the type of verification points,

FIG. 8 is a preview of a menu making it possible to choose a type of interrupts that may be generated by the user,

FIG. 9 is an overview of a menu making it possible to act on settings relating to the mobile telephone or to the application,

FIG. 10 is an overview of a menu making it possible to choose a type of interrupts that may be generated by another mobile, and - Figure 11 is an overview of a menu to manage variability. In Figure 1, there is shown a test bench comprising in at least one cabinet 10 several different mobile phones 12 connected to at least one central unit or server 14 connected via a network 16 to at least one remote computer 18.

The network 16 may be the Internet network.

The computers 18, the central unit 14 and the means of communication between these different elements are not further described because they are known to those skilled in the art.

The central unit comprises means of connection to the different mobile phones 12.

For the rest of the description, it is considered that two phones are different in the case of two different models or in the case of two mobile phones of the same model with different environments (operating system, network, operator, ...). .

Thus, the cabinet 10 may comprise several different mobile phones either by their operating system or by their components and devices.

As illustrated in FIG. 2, the computer 18 comprises the following peripherals: a screen 20, a keyboard 22, a loudspeaker 24 and preferably a microphone 26.

For each mobile phone 12, the cabinet includes means for capturing the image produced at the screen of the mobile phone, means for capturing the sound transmitted by the mobile phone, means for transmitting data to the mobile telephone and possibly means for generating a sound near the microphone of the mobile phone.

Regarding the audio part, the tester can transmit from a microphone 26 an audio signal which will be returned to the microprocessor of the phone via its microphone or via a cord 28 connected to a plug of the phone for the headset.

If the tester wishes to hear a sound emitted by the telephone, the audio signal generated by the microprocessor of the telephone is transmitted directly to the central unit 14 or transmitted to the speaker of the telephone arranged near a microphone connected to the unit. central 14. The audio signal received by the central unit is returned to the tester via the loudspeaker 24.

The audio exchanges between the tester and the telephone on which a test is performed are not described any more as different solutions within the reach of those skilled in the art could be envisaged.

Regarding the video signal visible on the screen, a first solution consists in recovering the video signal transmitted by the microprocessor of the telephone at the output of the microprocessor and then transmitting it via the central unit and the network to the computer. the tester that returns it at the screen. However, according to this implementation, there may be distortions between the image obtained on the screen of the mobile phone from the video signal emitted by the microprocessor of the telephone and the image restored on the screen of the computer. from the same video signal.

To limit this risk of distortions, the test bench includes for each mobile phone a camera 30 capable of filming the screen of the phone. The video signal of the camera 30 is transmitted via the central unit and the network to the computer of the tester and then restored on the screen 20.

This solution has the advantage of showing the tester the image actually displayed on the screen of the mobile phone.

The transmission of a video signal from a camera 30 to a remote computer 18 via the Internet is not more detailed because it is known to those skilled in the art.

Regarding the emulation of the phone, several solutions are possible. Software can be installed on the computer 18 for viewing at the screen 20 the front of the mobile phone to be tested. With the help of a pointer that can be moved around the screen, for example with a mouse, the tester can simulate an action on the phone keypad. The software generates a signal or a code according to the action simulated by the tester. This signal (or code) is transmitted to the test phone present in the test bench via the network and the central unit.

According to a first approach, this signal (or code) is transmitted to the appropriate inputs of the microprocessor of the telephone.

Thus, the code or signal generated by the software when the tester simulates an action on the "1" key is transmitted to the input of the microprocessor of the tested telephone connected to the "1" key of said telephone.

According to another preferred embodiment, the test bench comprises for each mobile phone an electronic card 32 interposed between the central unit 14 and the mobile phone, said card 32 having an output dedicated to each key of the mobile phone to close the contactor of the mobile phone associated with said key.

Thus, the code or signal generated by the software when the tester simulates an action on the "1" key is transmitted to the electronic card 32 of the telephone under test. The electronic card then generates an electric current between the terminals of the contactor associated with the "1" key of the tested mobile phone which simulates a pressing on the "1" key of said telephone. The input of the microprocessor of the tested mobile phone connected to the "1" key receives a signal identical to the signal corresponding to a real pressure on the "1" key.

This solution provides a more reliable simulation and limits the risk of distortions. The software installed on the computer of the tester for generating a code or a signal according to the action simulated by the tester is not more described because it is within the reach of the skilled person.

In addition, this software may not be limited to the simulation of an action on a key. In an improved variant, it can also simulate other actions of a user on a mobile phone such as pivoting to adjust the orientation of the image on the screen and generate a code or a signal based of this action which is transmitted to the appropriate input of the microprocessor of the mobile phone to be tested.

The test bench of the invention makes it possible, on the one hand, to remotely emulate a set of mobile phones without the intermediary of a third party, and on the other hand, to restore to the tester the sound emitted by the tested telephone or the image displayed on the tested phone without the intermediary of a third party.

This solution reduces the risk of errors related to the presence of an intermediary which tends to improve the quality of the tests.

However, even using this test bench, the testing of an application on a set of different mobile phones is long and tedious considering that it is necessary for the tester to define as many tests as variants of mobile phones tested.

Therefore, the invention also proposes a method of automating tests of an application intended for mobile telephony.

This method consists in generating from a model illustrated in FIG. 3 a script 34 unique for a group of mobile phones to be tested.

This script 34 comprises at least one interaction and at least one reaction of the mobile phone. It usually includes a sequence of interactions with one or more reactions interspersed between the interactions. According to the invention, the interactions are in the form of actions 36, and possibly disturbances generated by the user 38 and / or disturbances generated by another mobile phone 40.

In Figure 5, there is shown a list of icons each associated with an action 36. Thus, the first icon may correspond to the movement of the pointer on the screen of the mobile phone to be tested, the second icon may correspond to the test of the monkey. (Monkey test), the third icon may correspond to the pressing of two keys determined simultaneously, the fourth icon may correspond to the support in two points on a touch screen simultaneously, the fifth icon may correspond to the the sixth icon can correspond to the support at a given point on a touch screen, the seventh icon can correspond to the press on the red key or end of communication, the eighth icon can correspond to pressing the green key or communication start, the ninth icon can correspond to the launch of the application, the tenth icon can correspond to the insertion of a text.

This list of actions is not exhaustive, other icons relating to other actions that can be created.

Each action 36 comprises several attributes, namely an identifier, a source, a destination, if necessary a complement relating to the description of the action when it is necessary, for example, to define one or more specific keys or one or more points. on a touch screen for example.

In Figure 8, there is shown icons each associated with a disturbance generated by the user 38. Thus, the first icon may correspond to a change of orientation of the mobile, the second icon may correspond to the closure of a keyboard mobile, the third icon may correspond to the opening of a mobile keyboard, the fourth icon may correspond to the connection of the cord dedicated to charging the mobile phone battery, the fifth icon may correspond to the disconnection of the cord dedicated to charging the mobile phone battery.

This list of disturbances generated by the user 38 is not exhaustive, other icons relating to other disturbances generated by the user can be created.

Each disturbance generated by the user 38 comprises several attributes, namely an identifier, a source, a destination.

In Figure 10, there is shown icons each associated with a disturbance generated by another telephone 40. Thus, the first icon may correspond to the end of a communication, the second icon may correspond to the beginning of a communication, the third icon can correspond to the sending of an SMS type message, the fourth icon can correspond to the rejection of a call, the fifth icon can correspond to the sending of a message of the MMS type, the sixth icon can correspond to when accepting a call, the seventh icon may correspond to a "Bluetooth" type connection.

This list of disturbances generated by another telephone 40 is not exhaustive, other icons relating to other disturbances generated by another telephone can be created.

Each disturbance generated by another mobile has several attributes, namely an identifier, a source, a destination.

In an improved version, the tester will be able to define other types of interactions, for example interactions relating to a parameterization of the mobile 42, such as for example a parameter setting relating to the clock or a parameter setting relating to the communication modes of the type " Wif i "or" GPRS ", or interactions relating to a parameterization of the tested application.

Moreover, he will be able to define more or less long breaks between two successive interactions. As a reaction, the tester may define verification points 44 which will each generate a result 46.

According to a simplified variant, a reaction may consist in verifying the correct display of a text or an image on the screen or the correct transmission of an audio signal.

In Figure 7, there is shown icons each associated with a verification point, such as a screenshot, a sound recording or a video recording.

Each verification point has several attributes, namely an identifier, a source, a destination, a generated result.

These verification points will generate results which will each be associated with an icon represented in FIG. 6, namely a screen preview, a sound sequence or a video sequence.

Each result comprises several attributes, namely an identifier, a result defined by the verification point, an expected result and a validation attribute of validating or not the result defined by the verification point with respect to the expected result.

The model also manages the variability of mobile phones.

Thus, to manage the variability, the tester has icons illustrated in FIG. 11 associated respectively with the resolution of the screen, with the models of mobile telephones or with certain characteristics of the mobile phone, for example whether or not the mobile has a touch screen, a GPS module, a "Bluetooth" module, an alphanumeric keyboard, an orientation detection system.

Thus, it is possible to define several points of variability relating to the resolution of the screen 48, each of them comprising several attributes, namely an identifier, a resolution, the interactions or the identifiers of the interactions associated with this point of variability. . For example, the test will perform a given interaction only if the mobile phone has a given resolution.

In the same way, it is possible to define several points of variability relating to mobile phone models 50, each of them having several attributes, namely an identifier, a list of at least one mobile model, the interactions or the identifiers of interactions associated with this point of variability.

For example, the test will perform a given interaction only for one or more specific models.

It is also possible to define several points of variability relating to certain characteristics of the telephone 52, each of them having several attributes, namely an identifier, the characteristic necessary to perform an interaction, the interactions or the identifiers of the interactions associated with this point. of variability.

For example, the test will perform a given interaction only if the mobile tested has a specific characteristic.

The model makes it possible to define a unique script for a set of mobile phones to be tested without having any competence in a programming language. In a first step, the tester defines nodes, a first node 54 corresponding to the tester, a second node 56 corresponding to the mobile to be tested and possibly a third node 58 corresponding to another mobile. It specifies the actions 36 between the first node 54 and the second node 56. It can specify for each action the type of mobile phones on which said action must be performed.

It can then define other interactions applied to the phone to be tested, such as interactions originating from node 58, ie from another mobile or originating from node 54, namely the tester. As before, it can specify the type of mobile phones to which these interactions will be applied.

Finally, it defines the verification points and the associated results.

It can also define points of variability to associate certain interactions with one or more mobile phones.

The tester can then launch from a single script tests on a set of mobile phones present in at least one test bench.

In addition, the test bench also includes a database listing the mobile phones present in the bench, associating them with certain properties such as the resolution of their screens and / or certain characteristics such as the presence or absence of a touch screen. , of a module

GPS, a "Bluetooth" module, an alphanumeric keyboard, an orientation detection system.

Thus, the central unit 14 which stores the database receives the unique script and generates for each mobile to be tested a dedicated script, whether or not the interaction is maintained as a function of the points of variability defined by the tester in the script. unique and information contained in the database.

Thus, if an interaction concerns only mobile phones incorporating a touch screen, the central unit 14 transmits a script incorporating said interaction only mobile phones incorporating a touch screen and removes this interaction in the script transmitted to mobile phones without touch screen. The test bench, and more particularly the central unit 14, comprises at least one conversion means for defining for each interaction a code or a signal understandable by the mobile phone that receives it.

The central unit may include several conversion means each specific to at least one mobile phone. The database can associate to each mobile phone conversion means of its own. Thus, the unit central chooses the right conversion means depending on the phone to which the interaction is intended.

In parallel with the interactions, the central unit 14 manages the verification points and triggers screen shots, audio or video recordings. Advantageously, the test bench uses the cameras 30 to take screenshots or video recordings.

The results (screen previews, audio or video sequences) can be transmitted in real time to the tester.

Depending on the case, the script may comprise several sequences, the passage from one sequence to another being subject to control by the tester of the result generated by the verification point. Thus, the tester can stop the test if a result defined by the test bench is not consistent with that expected.

According to the invention, the method of automating the testing of an application intended for a set of mobile phones present in a test bench consists in defining at the level of a remote computer of the bench a single script comprising at least one interaction and usually a sequence of interactions, at least the verification of a reaction, usually the verification of several reactions through checkpoints, and points of variability, then to transmit this unique script to the test bench.

From its database, the test bench transmits to each mobile phone to test a dedicated script by deleting or not the interaction or interactions contained in the unique script by correlating the information points of variability and those contained in the database. data. From the database, the test bench uses the right conversion means for starting from the dedicated script to transmit to the mobile phone a signal or code function interactions, understandable by said mobile phone. The test bench captures and transmits to the remote computer for verification the reaction (s) that the unique script requests to enter. In an improved variant, the test bench uses the cameras 30 to capture the reaction (s).

Advantageously, the electronic card 32 dedicated to each mobile phone can be used to convert the interaction or interactions of the dedicated script into a signal or code understandable by the phone associated with said card.

This method makes it possible to simplify the test phase insofar as the tester only writes a single script. After having downloaded by all appropriate means the application on the mobile phones to be tested, it starts the execution of the script. The test bench automatically generates a dedicated test for each mobile phone to be tested and gives feedback to the tester who can analyze them directly without the need of a third party.

Claims

A method for automating the testing of an application on a plurality of mobile phones (12) grouped in a test bench (10), characterized in that it consists in creating at the level of the test bench a database listing the mobile phones (12) of the test bench and some of their properties, to generate at a computer level (18) a single script (34) having at least one interaction and generally a sequence of interactions, at least the verification of a reaction and points of variability (48, 50, 52) specifying a type of mobile phones to which certain interactions will be applied, to transmit the unique script (34) to the test bench (10), the test bench transmitting to each mobile phone to test a dedicated script by deleting or not the interaction (s) contained in the unique script by correlating the information of the points of variability and those contained in the database, and finally to seize e t transmit to the computer (18) for checking the reaction (s).

2. Method according to claim 1, characterized in that the database of the test bench associates with each mobile phone conversion means and in that the interaction or interactions transmitted to the mobile phone to be tested are converted using the means of adequate conversion.

3. Method according to claim 1 or 2, characterized in that it consists in using a camera (30) associated with the mobile phone to capture the reaction or reactions.