WO2020225635A1

WO2020225635A1 - Method and information system to evaluate an interaction between a user and a device

Info

Publication number: WO2020225635A1
Application number: PCT/IB2020/053796
Authority: WO
Inventors: Andrea Basilico
Original assignee: Baso Labs Sagl
Priority date: 2019-05-09
Filing date: 2020-04-22
Publication date: 2020-11-12
Also published as: IT201900006686A1

Abstract

The present invention relates to a method (600; 700) for evaluating an interaction between a user and a device (20). The method comprises the steps of: - running (601) a software application (50) on said device (20), the software application (50) requesting at least one interaction (I) from the user through a user interface (29) of the device (20); - identifying an observation time interval (Δ), said observation time interval (Δ) being comprised between an initial instant of time in which the software application (50) requests a first interaction from the user, and a final time instant in which a final interaction is recorded in response to a final interaction request by the software application (50), during said observation time interval (Δ), the method comprises: - recording (603) a sequence of user interactions (I); - acquiring (605) a plurality of measurements (S) through at least one sensor (27, 47) associated with said device (20). Furthermore, the method comprises the steps of: - comparing the sequence of interactions (I) of the user with one or more sequences of interactions, so as to identify an interaction anomaly when the sequence of interactions of the user deviates with respect to at least one sequence of expected interactions beyond a deviation threshold; - checking (615) whether during the observation time interval a disturbance event has occurred, said disturbance event being checked in the case in which at least one from among said plurality of measurements (S) deviates from an expected value beyond a disturbance threshold, and - assessing (617, 619) the interaction between the user and the device (20), providing an interaction result as a function of the interactions recorded and the occurrence or not of the disturbance event.

Description

METHOD AND INFORMATION SYSTEM TO EVALUATE AN

INTERACTION BETWEEN A USER AND A DEVICE

DESCRIPTION

TECHNICAL FIELD

The present invention relates to the field of information technology. In particular, the invention relates to a method and related computer system for evaluating an interaction between a user and a device.

BACKGROUND

Interaction between humans and machines - in particular with devices that implement one or more software applications - can be influenced by events that take place during the interaction between an individual and a machine, with strongly negative effects.

For example, in the field of aptitude tests - i.e. the evaluation of physical, psychophysical or innate or acquired psychological abilities - computer systems are proposed for the purpose of reducing the time scales and professionals needed for performing the aptitude test, in particular in the case of selection of personnel or for identifying a more suitable course of studies and/or a more satisfying professional career. Furthermore, computer systems can eliminate ethnic or other prejudices that could arise, in case of evaluation performed by a human.

An example of such computer systems is described in US 2015/379454 which describes a computer system for the identification of "talents", which is designed to assist the process of hiring new employees. The system uses a series of tests in digital format based on neuroscience for evaluating the career inclinations of a user, after which the system can provide career recommendations for the user or report on the suitability of the user's occupation to a company.

Although the system described by US 2015/379454 enables the examination of a large number of subjects that have the necessary resources for that purpose, the Applicant has identified that the computerization of aptitude tests does not enable the checking of an environment in which the aptitude tests of users are performed or the identification of whether an external event has influenced the performance of a user during the implementation of an aptitude test. Furthermore, WO 2017/037445 proposes a network monitoring system for monitoring user interactions with one or more computer systems. Such system comprises receiving metadata from one or more devices of a monitored computer system and identifying through the metadata events corresponding to a plurality of user interactions. Event data associated with this plurality of user interactions are memorized by the monitoring system and used to determine normal user behaviour, which is in turn memorized to be used as a reference.

Considering the different sector of remote assistance services - in particular in remote assistance services that comprise the aid of augmented reality - the Applicant has identified that unfavourable environmental conditions - such as background noise, mechanical strain, lighting, presence of gases or vapours - lead a user to enter data, maintain a position and / or perform incorrect or dangerous movements, complicating or unduly extending the assistance process.

This scenario is frequent in the event of assistance services for repairing vehicles, machinery and/or industrial systems in which a technician is operating in situ in a generally disadvantageous environment subject to numerous disturbance events possibly due in part to the vehicle, machine and/ or industrial system to be repaired.

OBJECTS AND SUMMARY OF THE INVENTION

An object of the present invention is to overcome the disadvantages of the prior art.

In particular, it is an object of the present invention to present a method and a related computer system able to evaluate an interaction between a user and a device identifying and considering any disturbances that arose during the interaction between the user and the device. The term "interaction" is used to indicate one or more user commands, or inputs, provided by the user by acting on the device. These and other objects of the present invention are achieved by a device incorporating the features of the accompanying claims, which form an integral part of the present description.

In an embodiment, a method for evaluating an interaction between a user and a device. The method comprises the steps of:

- executing a software application on said device, the software application requesting at least one interaction from the user through a user interface of the device;

- identifying an observation time interval, said observation time interval being comprised between an initial instant of time in which the software application requests a first interaction from the user, and a final time instant in which a last interaction is recorded in response to a last interaction request by the software application,

during said observation time interval, the method comprises:

- recording a sequence of user interactions;

- acquiring a plurality of measurements through at least one sensor associated with said device.

Furthermore, the method comprises the steps of:

- comparing the sequence of interactions of the user with one or more sequences of interactions, so as to identify an interaction anomaly when the sequence of interactions of the user deviates with respect to at least one sequence of expected interactions beyond a deviation threshold;

- verifying whether during the observation time interval a disturbance event has occurred, said disturbance event being verified in the case in which at least one from among said plurality of measurements deviates from an expected value beyond a disturbance threshold, and

- evaluating the interaction between the user and the device, providing an interaction result as a function of the interactions recorded and the occurrence or non-occurrence of the disturbance event.

Thanks to such a solution, it is possible to identify the occurrence of a disturbance event during the interaction of the user with the software application and adjust the result of such interaction based on the alteration of the interactions due to such disturbance event.

For example, in the event of aptitude tests the method makes it possible to identify whether during an aptitude test a disturbance event has taken place which may have influenced the responses provided by the user and therefore able to distort the aptitude evaluation derived from the test. Furthermore, thanks to the method described it is possible to automatically compensate for the effect of such disturbance event, guaranteeing the reliability of the aptitude test provided.

Likewise, in the event of remote assistance services, the method makes it possible to identify whether during the assistance a disturbance event has taken place that has led the user to provide data or incorrect instructions associated with an operation performed and therefore discard such data or incorrect instructions and request data or instructions to be re-entered and/or identify a correct response.

In an embodiment, the step of comparing the sequence of user interactions with one or more sequences of interactions, so as to identify an interaction anomaly comprises:

- identifying at least one selected from among:

- at least one interaction of the sequence of user interactions that is different from a corresponding interaction of the at least one sequence of expected interactions;

- an interaction frequency with which the user enters the interactions of the different sequence of interactions from a threshold frequency of interactions in the at least one sequence of expected interactions, and

- a delay between at least one interaction request by the software application and a corresponding interaction of the sequence of interactions greater than a threshold delay.

In this way it is possible to identify an anomaly - possibly, due to a disturbance event - starting from user interactions only with the software application.

In an embodiment, the step of verifying whether during the observation time interval a disturbance event has occurred, comprises:

- detecting at least one selected among:

- a crossing of the disturbance threshold by at least one measurement of said plurality of measurements;

- an exceeding of the disturbance threshold by a value variation between two consecutive measurements of said plurality of measurements,

- a deviation of a sequence of measurements of said plurality of measurements with respect to a corresponding sequence of expected values.

Thanks to such solution it is possible to identify a disturbance event connected with the interaction anomaly detected in a particularly simple, effective and reliable way.

In an embodiment, the step of evaluating the interaction between the user and the device comprises:

- selecting a disturbance coefficient between a plurality of predetermined disturbance coefficients based on one selected among:

- a value of the measurement of said plurality of measurements that exceeds the disturbance threshold;

- a value of the variation between the two measurements of said plurality of measurements that exceeds the disturbance threshold, and

- a deviation value (punctual, average, maximum or minimum) of the sequence of measurements of said plurality of measurements with respect to the corresponding sequence of expected values;

- processing the sequence of interactions according to an evaluation criterion, and

- combining the result of said processing with the disturbance coefficient to obtain the interaction result.

This makes it possible to quickly and efficiently select the correct disturbance coefficient and apply it to the result of the interaction so as to dynamically compensate for the effect of the disturbance event.

Alternatively, instead of being selected from among a plurality of predetermined disturbance coefficients, the disturbance coefficient can be calculated starting from one selected between:

- a value of the measurement of said plurality of measurements that exceeds the disturbance threshold, and

- a value of the variation between the two measurements of said plurality of measurements that exceeds the disturbance threshold.

This is particularly useful in the case in which predetermined disturbance coefficients are not available - for example, during an initial implementation step of the method, where there is a lack of connection with a remote database or an area of the memory module which stores the disturbance coefficients and/or any other situation in which the specific contingencies require a direct calculation of the disturbance coefficient.

In an embodiment, the step of processing the sequence of interactions according to an evaluation criterion comprises:

- selecting an interaction result between a plurality of predetermined interaction results based on a sequence of interactions, and

wherein the step of combining the result of said processing with the disturbance coefficient comprises:

- obtaining a modified interaction result by weighing the interaction result against the disturbance coefficient. This enables the effect of the disturbance event on the result of the interaction between the user and the software application to be numerically quantified particularly simply and efficiently manner.

In an embodiment, the method also comprises the steps of:

- comparing a recorded sequence of interactions in the absence of a disturbance event with a recorded sequence of interactions in the presence of a disturbance event with the same interaction requests by the software application;

- modifying at least one predetermined disturbance coefficient on the basis of said comparison, and

- associating the modified disturbance coefficient with at least a corresponding measurement acquired during the observation time interval in which the sequence of interactions is recorded in the presence of a disturbance event.

Additionally or alternatively to the step of modifying at least one disturbance coefficient, the method can comprise the step of determining at least one new predetermined disturbance coefficient on the basis of said comparison.

Thanks to such solutions it is possible to dynamically and completely automatically improve the precision and, possibly, the sensitivity of the method in recognizing and evaluating the effect of disturbance events to which the user can be subject, as the number of users interacting with the software application increases.

This makes the use of one or more machine learning algorithms particularly advantageous and efficient in order to perform the comparison between interactions and the modification and/ or addition of predetermined disturbance coefficients. In fact, such algorithms allow an initial set of predetermined disturbance coefficients to be adjusted and optimized as the interactions and disturbance events detected increase.

In an embodiment, the step of acquiring a plurality of measurements through at least one sensor associated with said device comprises:

- monitoring a plurality of measurements provided by at least one sensor between:

- an environmental sensor;

- a movement sensor;

- a biometric sensor, and

- a position sensor comprised in the user device.

Thanks to such a solution, it is possible to identify the occurrence of a disturbance event reliably, exploiting sensors present in the user device used for implementing the software application.

Additionally or alternatively, the step of acquiring a plurality of measurements through at least one sensor associated with said device comprises:

- an environmental sensor;

- a movement sensor;

- a biometric sensor, and

- a position sensor,

comprised in a satellite device operatively connected to the user device, but separate from it.

Thanks to such solution it is possible to identify the occurrence of a disturbance event reliably, exploiting separate sensors from the user device, therefore not necessarily requiring the user device to comprise sensors and / or to allow it to acquire measurements through sensors not present on the user device or having greater resolution. Furthermore, in the event in which the satellite device is of the wearable type, it is possible to monitor measurements of environmental magnitudes substantially as perceived by the user him/herself and it is also possible to easily acquire biometric measurements related to the user.

In an embodiment, the step of verifying whether during the observation time interval a disturbance event has taken place, comprises identifying at least one selected from between an interaction of the user with the user interface of the device not required by the software application, and an interaction request made by a further software application through the user interface of the device.

Thanks to this solution, it is possible to identify disturbance events connected with other software applications performed by the device which can compromise the user interactions in response to requests of the software application.

A different aspect of the present invention proposes a system comprising at least one user device and a remote processing unit adapted to establish between them a communication channel through a data communication network, the user device comprising a user interface and at least one sensor, and implementing a software application adapted to submit an aptitude test to a user. Advantageously, the software application and the remote processing unit are configured to implement the method according to any one of the embodiments described above.

Thanks to such solution, the software application and the remote processing unit cooperate to provide a reliable interaction result also in case of disturbance events. Furthermore, it is possible to select which part of the operations are performed by the software application and which ones by the remote processing unit according to the particular specific contingencies, e.g. based on the amount of data collected by the user devices and/ or the processing capacities of the user devices and the remote processing unit. Therefore, the calculation capacity necessary for the user device to perform the software application is extremely contained, allowing the requirements of the system on the user device to be relaxed.

Advantageously, implementing one or more processing steps of the interactions and measurements in the remote processing unit enables greater control and safety of the data and the algorithms exploited in the system.

Further features and advantages of the present invention will be more apparent from the description of the accompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

The invention will be described below with reference to some examples, provided for explanatory and non-limiting purposes, and illustrated in the accompanying drawings. These drawings illustrate different aspects and embodiments of the present invention and, where appropriate, reference numerals illustrating similar structures, components, materials and/ or elements in different figures are indicated by similar reference numbers.

Figure 1 is a block diagram of a system configured to implement the method according to an embodiment of the present invention;

Figure 2 is a flow diagram of a method according to an embodiment of the present invention; and

Figure 3 is a flow diagram of a dynamic updating procedure according to an embodiment of the present invention.

DETAILED DESCRIPTION OF THE INVENTION

While the invention is susceptible to various modifications and alternative constructions, certain preferred embodiments are shown in the drawings and are described hereinbelow in detail. It is in any case to be noted that there is no intention to limit the invention to the specific embodiment illustrated, rather on the contrary, the invention intends covering all the modifications, alternative and equivalent constructions that fall within the scope of the invention as defined in the claims.

The use of "for example", "etc.", "or" indicates non-exclusive alternatives without limitation, unless otherwise indicated. The use of "includes" means "includes, but not limited to" unless otherwise stated.

With reference to Figure 1, a system 1 is described according to an embodiment of the present invention.

The system 1 comprises a remote processing unit 10 and one or more user devices 20, 4 user devices 20 in the example considered. The remote processing unit 10 and each of the user devices 20 is able to establish a connection with a data communication network 30. In particular, the remote processing unit 10 is configured to establish a communication channel with each of the user devices 20 through data communication network 30 and / or vice versa, each user device 20 is configured to establish a communication channel with the remote processing unit 10 through the data communication network 30.

In detail, the remote processing unit 10 comprises a processing module 11 configured to implement one or more data processing algorithms, a memory module 13 configured to memorize data, and a transceiver module 15 configured to establish and manage one or more connections with the data communication network 30.

For example, the processing module 11 can comprise one or more processors, microprocessors, microcontrollers, ASIC, FPGA, DSP and the like. The memory module 13 can comprise one or more non-volatile and volatile memory elements adapted to memorize data, preferably in binary format. Finally, the transceiver module 15 can comprise one or more from among modems, switches, gateways, firewalls, etc. Naturally, the remote processing unit 10 or one or more of its modules 11, 13 and 15 can be provided as a single device, as a distributed network of devices and/ or as one or more virtual machines.

The user device 20 also comprises a processing module 21 configured to implement one or more data processing algorithms, a memory module 23 configured to memorize data, a transceiver module 25 configured to establish and manage one or more connections with the data communication network 30, one or more sensors 27, each configured to measure a respective physical magnitude and a user interface 29 configured to receive commands and provide information to a user (not illustrated).

In the event of the user interface 20, the processing module 21 can comprise one or more processors, microprocessors, microcontrollers, ASIC, FPGA, DSP or the like. The memory module 23 can comprise one or more non-volatile and volatile memory elements adapted to memorize data, preferably in binary format. The transceiver module 15 can comprise a modem for cabled and/or radio communications (Wi-Fi, bluetooth, GSM, UMTS, LTE, 5G, etc.).

The sensors 27 can comprise movement sensors - such as one or more from among accelerometers, gyroscopes, gravity sensors, etc. - position sensors - such as one or more from among magnetometers, a GNSS detection system, etc. - and environmental sensors - such as one or more from barometers, photometers, thermometers, microphones, photo cameras, proximity sensors, etc., and biometric sensors - such as a fingerprint reader.

Finally, the user interface 29 can comprise output elements - such as a screen, a loudspeaker, etc., input elements - such as a keyboard, joystick, physical and/ or virtual joypads, a microphone - and/ or mixed input/ output elements - such as a haptic interface.

Examples of user devices 20 comprise smartphones, tablets, personal computers and the like.

Optionally, the system 1 can also comprise one or more satellite user devices 40 configured to establish a direct communication channel with a respective user device. Preferably, satellite user devices 40 are wearable electronic devices - such as smartwatches, visors for augmented or virtual realities, etc. Additionally or alternatively, the satellite device 40 can comprise other types of device, such as domotic devices, voice assistants, an electronic on-board system of a vehicle and the like.

Like the user device 20, the satellite user device comprises a processing module 41 configured to implement one or more data processing algorithms, a memory module 43 configured to memorize data, a transceiver module 45 configured to establish and manage one or more direct connections (e.g. via Bluetooth) and/or indirect connections (e.g. through a cloud platform) with the corresponding user device, one or more sensors 47, each configured to measure a respective physical magnitude and optionally a user interface (not illustrated).

Preferably, the satellite user device 40 can comprise one or more biometric sensors, each configured to detect a vital sign of a respective user (not illustrated), such as a heart beat, a body temperature, etc.

In the embodiments according to the present invention, each user device 20 is configured to instantiate a software application 50 designed for the aptitude test of a user.

Preferably, the software application 50 comprises submitting one or more aptitude tests of the logical, psychological and/or psychophysical type that comprise interaction by the user through the user interface 29 of the user device used by the user.

The system 1 described enables a method 600 to be implemented for the aptitude test of a user according to an embodiment of the present invention described below with reference to Figure 2.

Initially, the software application 50, once instantiated in the user device 20 of the user, is configured to submit one or more aptitude tests to the user (block 601). For example, the software application 50 submits a series of aptitude tests to the user through the user interface 29 of the user device 20. In an embodiment of the present invention the aptitude tests may be substantially similar to the tests described in US 2015/379454.

During the performance of each aptitude test, interactions I of the user in performing each test submitted by the software application 50 (block 603) are monitored. In the present description, the term "interaction" indicates a user command, or input, provided by the user to the software application 50 through the user interface 29 of the user device 20. For example, the software application 50 records a sequence of one or more interactions I of the user with the user interface 29 - like an activation of a mechanical element of the user interface (a button, a joystick), contact with a particular portion of a capacitive screen, a movement of the user device 20, a voice command, etc. Additionally, the software application 50 can also identify a completion, a failed completion of each aptitude test and/or a final score, and a time dedicated by the user to the performance of each aptitude test.

Furthermore, an observation time interval D is identified (605) during which the interactions I and the measurements S are monitored. For example, the software application 50 is configured to determine the observation time interval D as the time interval comprised between an initial instant of time tO in which the software application 50 requests a first interaction from the user, and a final time instant tf in which a final interaction is recorded in response to a final interaction request by the software application 50. In a preferred embodiment, the software application 50 is configured to determine a respective observation time interval D for each of the aptitude tests submitted to the user. In other embodiments, the observation time interval D can be defined differently and / or can comprise an initial time instant W prior to the request for a first interaction or a final time instant tf after the recording of the final interaction of the user.

Advantageously, during the performance of each aptitude test - i.e. during the observation time interval D - measurements S generated by at least one of the sensors 27 and / or 47 of the user device 20 and / or by the satellite user device 40 (block 607) are also monitored. For example, the software application 50 monitors and, possibly, records measurements generated by one or more of the sensors 27 and/ or 47 during the interaction between the user and the software application 50.

Preferably, although not necessarily, the interactions I and the measurements S when recorded comprise or are associated with a respective timestamp, which indicates the generation, or acquisition time instant, of the specific interaction I or corresponding measurement S.

The interactions I and measurements S are, therefore, transferred to the remote processing unit 10 (block 609). For example, the software application 50 is configured to establish a communication channel with the remote processing unit 10 and to transmit the interactions I and the measurements S periodically, continuously or once the user has finished each individual aptitude test or after having detected the end of the interactions of the user with the software application 50.

Advantageously but not necessarily, the interactions I and the measurements S comprise, or are transmitted together with, an indication of the aptitude test to which they refer.

Subsequently, the interactions I are analysed to identify any interaction anomaly (decision block 611). Preferably, the remote processing unit 10 is configured to identify a trend of the interactions I that deviates from an expected trend of the interactions between a user and software application 50. Even more preferably, the interaction anomaly is identified when at least one sequence of interactions I detected deviates with respect to a sequence of expected interactions beyond a deviation threshold.

For example, the remote processing unit 10 can be configured to determine a deviation of one or more interactions I of the sequence considered with respect to corresponding interactions of a sequence of expected interactions - e.g. one or more incorrect commands provided by the user through the user interface 29. Alternatively or additionally, the remote processing unit 10 can be configured to determine the exceeding of a frequency - punctual, average, maximum or minimum - in which the interactions I of the sequence of interactions I are entered with respect to a corresponding threshold frequency. Again, the remote processing unit 10 can be configured to determine the exceeding of a frequency time delay - punctual, average, maximum or minimum - expected between an interaction request by the software application 50 and a corresponding interaction - e.g. in the event in which the user interacts with the software application 50 particularly slowly.

Preferably, the remote processing unit 10 memorizes a plurality of sequences of expected interactions in the memory module 13, advantageously organized in a table or database (e.g. based on experimental tests and/ or statistics). Even more preferably, one or more sequences of expected interactions can be defined and memorized as one or more sets of rules - e.g. a variation in the frequency of interactions of a sequence greater than a threshold, a number of incorrect interactions following a number of correct interactions, etc.

In the event that no abnormality is identified in the interactions I (exit branch N of the block 611), the interactions I are processed to determine a corresponding interaction result, an aptitude test of the user in the considered case (block 613). For example, the remote processing unit 10 is configured to implement an aptitude test algorithm that receives interactions I as the sole input and generates the aptitude score A as the output according to a predefined criterion (e.g. similar to what is described in US 2015/379454).

In the example considered, the remote processing unit 10 selects as the output a corresponding aptitude score A from a plurality of predetermined score values on the basis of the interactions I - e.g. high scores are associated with a larger number of interactions I corresponding to the expected predetermined interactions. Preferably, the remote processing unit 10 memorizes the plurality of predetermined scores in the memory module 13, advantageously organized in a table or database, where each of such predetermined scores is associated, for example, with a level of correspondence between the interactions I and the expected interactions (e.g. based on experimental tests and/ or statistics). After this, the method 600 therefore continues to the block 621 described below.

Returning to the decision making block 611, in the event that an interaction anomaly I is identified (output branch Y of block 611), the measurements S provided by the sensors 27 and / or 47 are therefore analysed in order to identify the occurrence of a disturbance event able to influence the interaction of the user with the software application 50 (decision making block 615). Advantageously, the occurrence of a disturbance event is identified in the event in which a trend of the measurements S deviates from an expected trend.

In the present description, the term "disturbance event" indicates a physical phenomenon that can influence the interaction between the user and the software application, e.g. due to environmental factors such as noises and/or excessive / inconsistent light intensities, interaction between the user and other individuals/apparatuses, accentuated mechanical strain, such as a high speed movement of the user and/ or a falling device, etc.

In a preferred embodiment, the remote processing unit 10 can be configured to identify a deviation of one or more measurements S with respect to an expected trend according to one of the following criteria. The deviation can be identified by detecting the crossing of a threshold value by a measurement S and / or the exceeding of a threshold variation by two or more preferably consecutive measurements S. Additionally or alternatively, such deviation can be identified by detecting an average/minimum/maximum deviation of a sequence of measurements S with respect to a corresponding sequence of expected values.

For example, the remote processing unit 10 can be configured to detect an acceleration value greater than a limit acceleration value that can be associated with a fall of the user device 20 or, more generally, a sudden displacement of the user device 20 from an enjoyment position indicative of an, at least temporary, interruption of the performance of the test. Additionally, the measurements of the proximity sensor can be used in a similar way. Furthermore, the algorithm that identifies disturbance events can be configured to detect whether the sounds acquired by the microphone exceed an acoustic threshold value, whether the temperature measured by the thermometer exceeds a limit temperature, whether the light intensity measured by the photometer exceeds or drops below a limit light intensity, etc.

In the event of variations between measurements, the remote processing unit 10 can be configured, for example, to compare a sequence of two or more measurements provided by the magnetometer, by the gyroscope, by the accelerometers, by the GNSS positioning system, heart rate variations or by a pedometer to identify that the user has undertaken a movement with a speed greater than a threshold speed and/or has moved by a greater distance than a threshold distance during the interaction with the software application 50. Again, the identification of sudden light variations can identify an unfavourable light condition to concentration.

Finally, in the event of deviation from an expected trend, the remote processing unit 10 can be configured, for example, to identify whether and for how long the user has looked away from the user device 20 through images acquired by the photo camera - identifying a variation in the position of the eyes or a pattern of the position of the eyes and/ or the face of the user. Furthermore, sequences of sounds acquired by a microphone can be exploited for identifying a conversation between the user and an interlocutor or the onset of particularly intense background noise.

In the case of analysing measurements S not indicated, the occurrence of a disturbance event (output branch N of block 615), the interactions I are processed to determine the corresponding interaction result, an aptitude test of the user in the case considered (as described in relation to the previous block 613). The method 600 therefore continues to the block 621 described below. Returning to the decision-making block 615, in the event that the analysis of the measurements S indicates the occurrence of a disturbance event (output branch Y of block 615), a disturbance coefficient D is selected, indicative of the influence that the disturbance event has had on the interaction between user and software application 50, between a plurality of predetermined disturbance coefficients on the basis of the analysed measurements S (block 617).

In the example considered, the remote processing unit 10 can be configured to select the disturbance coefficient D between a plurality of predetermined disturbance coefficients memorized in the memory module 13 of the remote processing unit 10, e.g. in a corresponding table or database wherein each of such predetermined disturbance coefficients is associated with one or more corresponding reference values - or intervals of values - referring to one or more of the measurements S (e.g. based on experimental tests and/ or statistics). In other words, the disturbance coefficient D substantially acts as a weight that indicates an influence of the disturbance detected through the measurements S on the interaction between user and software application 50.

In that case, the disturbance coefficient D is selected from the predetermined disturbance coefficients on the basis of an intensity and/ or type of disturbance event through the measurements S.

For example, the remote processing unit 10 is configured to base the selection of the disturbance coefficient D on the values of the measurements S that have led to the identification of the disturbance event, on a difference between the values of such measurements S or the variations between such measurements S with respect to the corresponding threshold values, threshold variations, or expected values in the event of deviation from an expected sequence of measurements.

Additionally, the remote processing unit 10 can be configured to adjust the selection of the disturbance coefficient D obtained through the measurements S on the basis of the type of disturbance event identified. For example, a conversation between a user and an interlocutor can lead to the selection of a greater disturbance coefficient D than the disturbance coefficient D selected if a background noise is identified that is greater than the selected threshold value.

Additionally or alternatively, the disturbance coefficient D can be selected on the basis of the interaction anomaly identified through the interactions I. Again, the disturbance coefficient D can be calculated, preferably in real time, on the basis of the measurements S provided by the sensors 27 and/or 47. For example, the remote processing unit 10 is configured to determine the disturbance coefficient by implementing a function that receives as input one or more values of the measurements S provided by the sensors 27 and / or 47.

Subsequently, the interactions I are processed and combined with the disturbance coefficient D selected to determine a corresponding adjusted interaction result. In the example considered, the interaction result corresponds to an adjusted aptitude score A⁺ of the user related to an aptitude characteristic on the basis of the specific test submitted by the software application 50 (block 619).

For example, the remote processing unit 10 is configured to implement an aptitude test algorithm that receives as input only the interactions I and selects as output a corresponding aptitude score A between a plurality of predetermined scores, e.g. memorized in the memory module 13, on the basis of interactions I (similar to what is described above the block 613) - without considering the effect due to the disturbance event. Therefore, the aptitude score A is combined with the disturbance coefficient D. For example, the remote processing unit 10 is configured to obtain the adjusted aptitude test A⁺ by multiplying the aptitude test score A and the disturbance coefficient D (i.e., A⁺ = A x D). In other words, the disturbance coefficient D is used as a weight applied to the aptitude test A.

Optionally, the interaction results related to each test are combined to obtain an overall interaction result (block 621). In the example considered, the aptitude scores A and/ or the adjusted aptitude scores A⁺ associated with one or more different tests can be combined to obtain a total aptitude score A_T. For example, the interaction results are compared with one or more models memorized in the memory module 13 for defining the total interaction result.

Finally, the interaction results are made available for viewing and / or used by an entity - e.g. a different computer system, a software application, by the user - preferably through the software application 50 - and/ or third parties (block 623).

For example, the remote processing unit 10 is configured to memorize the aptitude scores A, A⁺ and/or A_T in the memory module 13. Alternatively or additionally, the remote processing unit 10 can generate a report containing one or more of the aptitude scores A, A⁺ and/or A_T related to the user and memorize it in the memory module and / or transmit it to third parties, possibly, only in the event in which one or more from among the aptitude scores A, A⁺ and / or A_T exceeds a corresponding threshold value. For example, the interaction results are compared with one or more interaction models and/or results associated with other subjects, memorized in the memory module 13, to determine whether the total interaction result of the considered user defines a profile corresponding to a predefined/ desired profile. In the example considered, the aptitude scores A, A⁺ and/or A_T can be made available to an HR selection department for guiding/ assisting a selection process of subjects suitable for a duty.

According to an embodiment of the present invention, the system 1 is configured to implement an automatic learning and adjustment procedure 700 configured to update the disturbance coefficients D memorized and, preferably, the association between the disturbance coefficients D and the measurements S.

In the example illustrated by the flow diagram in Figure 3, the procedure 700 comprises acquiring the interactions I and the measurements S, associated with the implementation of each aptitude test provided by each user device 20 of the system 1 (block 701).

Therefore, for each group of interactions I and measurements S it is identified whether during the interaction of the user with the software application 50 a disturbance event has occurred (decision-making block 703). For example, the remote processing unit 10 is configured to identify the presence or absence of disturbances on the basis of the contents of the measurements S - as described above in relation to the method 600 - or for monitoring the outcome of steps 609 - 615 of the method 600.

In the event in which a disturbance event is identified during the interaction between the user and the software application 50 (output branch Y of block 703), the interactions I and the measurements S are memorized in a first register, or database (block 705); whereas in the case in which a disturbance event (output branch N of the block 703) is identified, the interactions I and the measurements S are memorized in a second register, or database (block 707). Preferably, the interactions I and the measurements S are memorized together with an indication of the aptitude test with which they are associated.

For example, the remote processing unit 10 is configured to memorize interactions I and measurements S in the first or the second database - defined in the memory module 13 - once the occurrence of a disturbance event has been detected or excluded during the implementation of the method 600.

Subsequently, at least part of the interactions I that take place in the presence of a disturbance are compared with corresponding interactions I that take place in the absence of a disturbance (block 709).

For example, the remote processing unit 10 is configured to correlate one or more interactions I that take place in the presence of a disturbance event and interactions I that take place in the absence of a disturbance event associated with the same sequence of interaction requests by the software application 50, i.e. the same aptitude test in the example considered.

On the basis of the result of such comparison, it is possible to modify the value of one or more corresponding predetermined disturbance coefficients and/or add one or more predetermined disturbance coefficients (block 711).

Each modified, or added, disturbance coefficient D, is therefore associated with one or more corresponding measurements S detected during the interaction anomaly identified in the interactions I that take place in the presence of a disturbance (block 713).

This dynamic update of the plurality of selectable disturbance coefficients - e.g. stored in the memory module 13 of the remote processing unit 10 - increases the reliability and precision of the plurality of selectable disturbance coefficients in the evaluation of the effect of disturbance events.

For example, the remote processing unit 10 is configured to analyse such measurements S and identify one or more reference measurements comprised in the measurements S and associate them with the corresponding disturbance coefficient D calculated in the previous step 705. Also in this case, it is possible to configure the remote processing unit 10 to determine new disturbance events - each associated with a respective set of reference measurements S. In an embodiment, the reference measurements S can be used to modify or add the disturbance threshold values used in the method 600.

Finally, the disturbance coefficients D, thus calculated, and the related reference measurements S are made available to be used in the method 600 (block 715). For example, the remote processing unit 10 is configured to dynamically update the predetermined disturbance coefficients memorized in the memory module 13 and the related reference measurements S. Advantageously, the automatic procedure 700 can comprise implementing machine learning algorithms in order to implement in a completely automated way the steps of comparing the interactions I, modifying the existing disturbance coefficient D and/or defining the disturbance coefficient D and associating the disturbance coefficient D with the corresponding disturbance or combination of disturbances identified. In particular, the use of machine learning algorithms allows the precision and reliability of the interaction result provided by the method 600 to be increased as the users interacting with the software application 50 increase.

Preferably, the procedure 700 is reiterated in parallel to the method 600 whenever the remote processing unit 10 receives new interactions I and measurements S from a user device 20 of the system 1.

The invention thus conceived is susceptible to numerous further modifications and variants all falling within the scope of the present invention according to the appended claims.

For example, instead of a dedicated software application 50 the system 1 can provide access to a computer platform by means of a third party application such as a web browser, possibly supported by a software agent or implementing a dedicated plug-in. In this case, the submission of the tests and the acquisition of interactions I and measurements S are run by the computer platform through the browser instantiated on the user device 20, e.g. by exploiting appropriate APIs. Such computer platform can be implemented by the remote processing unit 10 or by another dedicated unit (not illustrated) accessible via the data communication network 30.

Furthermore, nothing prevents from detecting information or other outputs provided by other software applications and / or exploiting an interaction with the user interface 29 of the user device 20. Advantageously, the software application can be configured to identify an interaction request sent by a different software application instantiated on the device and/ or an interaction of the user not requested by the software application 50 and associate it with the occurrence of a disturbance event during the implementation of the aptitude test. For example, it is possible to determine the occurrence of a disturbance event by identifying a suspension of the software application 50, due to the arrival of a telephone call, the notification generated by another software application instantiated on the user device 20 - such as an instant messaging application - etc. Again, the measurements S can be acquired according to different criteria. For example, the data generated by the sensors 27 can be acquired by the software application 50 periodically according to various sampling criteria. Alternatively, the software application 50 can be configured to monitor the data generated by the sensors 27 and only record data that respond to a predetermined criterion - for example, the exceeding of a threshold, a variation of interest between two consecutive measurements of a physical magnitude measured by one of the sensors 27 or 47.

Furthermore, the occurrence of a disturbance event can be identified through a combined analysis of two or more measurements S, preferably, provided by different sensors 27 or 47.

In alternative embodiments (not illustrated), nothing prevents from determining a plurality of partial disturbance coefficients, each associated with a specific measurement S and, therefore, defining a total disturbance coefficient provided by the combination of such partial disturbance coefficients.

As will be clear to a person skilled in the art, the interactions I and measurements S can be exploited in a similar way to what is described above to determine a concentration capacity of the user during the implementation of a test in the presence of disturbances.

Furthermore, for each predetermined disturbance coefficient it is possible to define a corresponding interval of values of the measurements S associated thereto, as well as a punctual value.

Furthermore, nothing prevents configuring the software application 50 to perform at least part of the processing of the interactions I and/or of the measurements S, at the limit up to the calculation of one or more of the aptitude scores A, A⁺ and/or A_T, and transmit only the result of such operation to the remote processing unit 10.

For example, in an embodiment, the software application 50 implements substantially the entire method 600 up to determining the interaction result for each aptitude test performed by the user. In this case, the software application 50 transmits the interaction results determined by the remote processing unit 10 that combines them to determine the total interaction result and determine whether the total interaction result of the considered user defines a profile corresponding to a predefined/desired profile.

Advantageously, the processing unit 10 can be configured to determine the value of the disturbance coefficient D according to the procedure 700, as described above, and transmit an updated list (e.g. in the form of a table or a database) of disturbance coefficients to each software application 50 installed on a user device 20 of the system 1. For example, the list of disturbance coefficients comprises for every disturbance coefficient D, one or more respective values or intervals of values of one or more measurements S provided by the sensors 27 and / or 47.

As will be clear to a person skilled in the art, the embodiments of the present invention are not limited to systems for the aptitude test of users such as in the example case described above.

For example, embodiments of the present invention can be applied in the case of remote assistance, in particular in the case of using augmented reality. In that case, disturbance events can be identified that compromise the interaction between the user and the user interface of the user device (e.g. sustained mechanical vibrations, notable light variations, high background noise, etc.) and, if an error or inconsistency is identified in the interactions, incorrect commands provided by the user can be discarded and requests for interaction by the software application 50 can be reiterated or confirmation of the command provided by the user can be requested and/or the acquisition of interaction requests through the user interface of the user device can be modified in order to compensate for the disturbance event (e.g. by varying the luminosity of a screen or stabilizing an image, by applying suppression of background noise, etc.).

Moreover, all the details can be replaced by other technically equivalent elements. In particular, one or more steps of methods and procedures described above can be omitted, performed in a different order, performed in series and/or in parallel according to the specific implementation needs without departing from the scope of protection of the following claims.

In particular, as will be clear to a person skilled in the art, it is possible first to detect the occurrence of a disturbance event and then check the presence of an abnormal user interaction - i.e. invert the order of steps 615 and 611 described above - and obtain the same interaction result in a similar way to what is described above. Furthermore, one or more steps of the method 600 and/ or the process 700 can be implemented by the software application 50 instantiated on the user device 20 alternatively or additionally to the steps implemented by the remote processing unit 10.

Claims

1. Method (600; 700) for evaluating an interaction between a user and a device (20), the method comprising the steps of:

executing (601) a software application (50) on said device (20), the software application (50) requesting to the user at least one interaction (I) through a user interface (29) of the device (20);

identifying an observation time interval (D), said observation time interval (D) being comprised between an initial time instant in which the software application (50) requests a first interaction from the user, and a final time instant in which a last interaction is recorded in response to a last interaction request by the software application (50),

during said observation time interval (D), the method comprises:

recording (603) a sequence of user interactions (I);

acquiring (605) a plurality of measures (S) by means of at least one sensor (27, 47) associated with said device (20), and

wherein the method also comprises the steps of:

comparing the sequence of user interactions (I) with one or more sequences of interactions, so as to identify an interaction anomaly when the sequence of user interactions deviates with respect to at least one sequence of expected interactions beyond a deviation threshold;

verifying (615) whether a disturbance event has occurred during the observation time interval, said disturbance event being verified in the case in which at least one of said plurality of measures (S) deviates from a value expected beyond a disturbance threshold, e

evaluating (617, 619) the interaction between user and device (20) by providing an interaction result as a function of the recorded interactions and of the occurrence or non-occurrence of the disturbance event.

2. Method (600; 700) according to claim 1, wherein the step of comparing the sequence of user interactions (I) with one or more sequences of interactions, so as to identify an interaction anomaly comprises:

identifying at least one selected among:

at least one interaction of the sequence of user interactions (I) differing from a corresponding interaction of the at least one sequence of expected interactions;

an interaction frequency with which the user enters the interactions of the sequence of interactions (I) different from a threshold frequency of interactions in the at least one sequence of expected interactions, and a delay between at least one interaction request by the software application and a corresponding interaction of the sequence of interactions greater than a threshold delay.

3. Method (600;700) according to claim 1 or 2, wherein the step of verifying (615) whether a disturbance event has occurred during the observation time interval, comprises:

detecting at least one selected among:

a crossing of the disturbance threshold by at least one measure of said plurality of measures (S);

an exceeding of the disturbance threshold by a value variation between two consecutive measures of said plurality of measures (S), and a deviation of a sequence of measures of said plurality of measures (S) with respect to a corresponding sequence of expected values.

4. Method (600; 700) according to claim 3, wherein the step of evaluating (617, 619) the interaction between user and device (20) comprises:

selecting (617) a disturbance coefficient (D) among a plurality of predetermined disturbance coefficients based on one selected among:

a value of the measure of said plurality of measures (S) that exceeds the disturbance threshold;

a value of the variation between two measures of said plurality of measures (S) which exceeds the disturbance threshold, and

a deviation value of the sequence of measurements of said plurality of measures (S) with respect to the corresponding sequence of expected values;

processing (619) the sequence of interactions (I) according to an evaluation criterion, and

combining (619) the result of said processing with the disturbance coefficient (D) to obtain the interaction result.

5. Method (600; 700) according to claim 3, wherein the step of evaluating (617, 619) the interaction between user and device (20) comprises:

calculating a disturbance coefficient (D) starting from one selected among: a value of the measure of said plurality of measures (S) that exceeds the disturbance threshold;

a value of the variation between the two measures of said plurality of measures (S) which exceeds the disturbance threshold, and

a deviation value of the sequence of measures of said plurality of measures (S) with respect to the corresponding sequence of expected values; processing (619) the sequence of interactions (I) according to an evaluation criterion, and

6. Method (600; 700) according to claim 4 or 5, wherein the step of processing (619) the sequence of interactions (I) according to an evaluation criterion comprises:

selecting an interaction result (A) between a plurality of predetermined interaction results based on a sequence of interactions (I), and

wherein the step of combining (619) the result of said processing with the disturbance coefficient (D) comprises:

obtaining a modified interaction result (A+) by weighing said interaction result (A) with the disturbance coefficient (D).

7. Method (600;700) according to claim 4, 5 or 6, further comprising the steps of:

comparing (709) a sequence of interactions (I) recorded in absence of a disturbance event with a sequence of interactions (I) recorded in presence of a disturbance event in response to the same interaction requests from the software application (50);

modifying (711) at least one disturbance coefficient (D) predetermined on the basis of said comparison, and

associating (713) the modified disturbance coefficient to at least one corresponding measure (S) acquired during the observation time interval (A) in which it is recorded the sequence of interactions (I) in presence of a disturbance event.

8. Method (600; 700) according to claim 4, 5 or 6, further comprising:

comparing (709) a sequence of interactions (I) recorded in absence of a disturbance event with a sequence of interactions (I) recorded in presence of a disturbance event in response to the same interaction requests by the software application (50);

determining (711) at least one new disturbance coefficient (D) based on said comparison, and associating (713) the new disturbance coefficient (D) to at least one corresponding measure (S) acquired during the observation time interval (D) in which it is recorded the sequence of interactions (I) in presence of a disturbance event.

9. Method (600; 700) according to any one of the preceding claims, in which the step of acquiring (605) a plurality of measures (S) by means of at least one sensor (27, 47) associated with said device (20) comprises:

monitoring a plurality of measures provided by at least one among:

an environmental sensor;

a motion sensor,

a biometric sensor, and

a position sensor

comprised in the user device (20).

10. Method (600; 700) according to any one of the preceding claims, in which the step of acquiring (605) a plurality of measures (S) by means of at least one sensor (27, 47) associated with said device (20) comprises:

monitoring a plurality of measures provided by at least one among:

an environmental sensor;

a motion sensor,

a biometric sensor, and

a position sensor,

comprised in a satellite device (40) operatively connected to the user device (20), but distinct therefrom.

11. Method (600; 700) according to any one of the previous claims, wherein the step of verifying (615) whether a disturbance event has occurred during the observation time interval, comprises identifying at least one selected among an interaction the user with the user interface (29) of the user device (20) not requested by the software application (50), and an interaction request performed by a further software application through the user interface (29) of the device (20).

12. System (1) comprising at least one user device (20) and a remote processing unit (10) configured for establishing communication channel between them through a data communication network (30), the user device (20) comprising a user interface (29) and at least one sensor (27), and implementing a software application (50) configured for submitting an aptitude test to a user, in which the software application (50) and the remote processing unit are configured for implementing the method according to any one of the previous claims.