WO2016146536A1

WO2016146536A1 - Method and apparatus for controlling a user's interaction with a non-medical application

Info

Publication number: WO2016146536A1
Application number: PCT/EP2016/055338
Authority: WO
Inventors: Hanne Storm
Original assignee: Hanne Storm
Priority date: 2015-03-13
Filing date: 2016-03-11
Publication date: 2016-09-22

Abstract

A method and an apparatus for controlling a user's interaction with a non-medical application, such as a computer game, have been disclosed. The method comprises steps of providing a skin conductance signal measured at an area of the user's skin through a time interval, counting the number of fluctuation peaks in the skin conductance signal through said time interval, and using the number of fluctuation peaks in the skin conductance signal as a control parameter for controlling the user's interaction with the non-medical application.

Description

METHOD AND APPARATUS FOR CONTROLLING A USER'S INTERACTION WITH A NON-MEDICAL APPLICATION

TECHNICAL FIELD

The invention relates in general to biofeedback, and in particular to a method and an apparatus for controlling a user' s interaction with a non-medical application, such as a computer game.

BACKGROUND OF THE INVENTION

Computer games which make use of biofeedback information are known in the background art, e.g. from WO-2004/104763 and US-20110009193. WO-03/94726 discloses a method and an apparatus for monitoring the autonomous nervous system of a sedated user. In the method, a skin conductance signal is measured at an area of the user' s skin. Certain characteristics, including the average value of the skin conductance signal through a time interval and the number of fluctuation peaks through the interval, is calculated. Based on these characteristics, two output signals are established, indicating pain discomfort and awakening in the user, respectively. The awakening signal is established based on, i.a., the number of fluctuation peaks in the skin conductance signal through the interval.

SUMMARY OF THE INVENTION

An object of the present invention is to provide an improved method and an apparatus for controlling a user' s interaction with a non-medical application, such as a computer game.

According to the invention, the above objects are achieved by a method and an apparatus as defined in the appended claims.

Further advantages and characteristics of the invention are indicated in the dependent claims.

BRIEF DESCRIPTION OF THE DRAWINGS The invention will be described by example with reference to the drawings, wherein

Figure 1 is a block diagram illustrating a preferred embodiment of an apparatus according to the invention,

Figure 2 is a flow chart illustrating a method according to the invention, and Figures 3-8 are schematic figures illustrating further aspects of the invention.

DETAILED DESCRIPTION OF THE INVENTION

Figure 1 illustrates a block diagram for an embodiment of an apparatus according to the invention. The apparatus is particularly arranged for controlling a user' s interaction with a non-medical application, such as a computer game. On an area 2 of the skin on a body part 1 of the user, sensor means 3 are placed for measuring the skin's conductance. The body part 1 is preferably a hand, wrist or a foot, and the area 2 of the skin on the body part 1 is preferably the palmar side of the hand or wrist or the plantar side of the foot. The sensor means 3 comprise contact electrodes where at least one electrode is placed on the skin area 2. In an embodiment the sensor means 3 consist of three electrodes: a signal electrode, a measuring electrode and a reference voltage electrode, which ensures a constant application of voltage over the stratum corneum (the surface layer of the skin) under the measuring electrode. The measuring electrode and the signal electrode are preferably placed on the skin area 2. The reference voltage electrode may also be placed on the skin area 2, but it is preferably placed in a nearby location, suitable for the measuring arrangement concerned.

The contact electrodes may be non-disposable and preferably made of metal, e.g. steel or silver. The measuring electrode should be about 1 cm to give the best assessment for the 3 electrode system, the C and R electrode can be smalle,r e.g. 0.5 cm or less. Various electrode types, materials and sizes may be used. In an embodiment an alternating current is used for measuring the skin's conductance. The alternating current advantageously has a frequency in the range of up to 1000 Hz, corresponding to the area where the skin's conductance is approximately linear. A frequency should be selected which ensures that the measuring signal is influenced to the least possible extent by interference from, e. g., the mains frequency. In a preferred embodiment the frequency is 88 Hz. A signal generator, operating at the specified frequency, applies a signal current to the signal electrode.

In the case of alternating current the conductance is identical to the real part of the complex admittance, and therefore not necessarily identical with the inverse value of the resistance. An advantage of using alternating current instead of direct current in conductance measurement is that by this means one avoids the invidious effect on the measurements of the skin's electrical polarizing properties.

The resulting current through the measuring electrode is conveyed to a measurement converter 4. This comprises a current to voltage converter, which in a preferred embodiment is a transresistance amplifier, but in its simplest form may be a resistance, which converts the current from the measuring electrode to a voltage.

The measurement converter further comprises a decomposition circuit, preferably in the form of a synchronous rectifier, which decomposes the complex admittance in a real part (the conductance) and an imaginary part (the susceptance). However, it is sufficient if the decomposition circuit only comprises means for deriving the conductance. The synchronous rectifier multiplies the measured voltage with the voltage from the signal generator. The two signals are in-phase. After

multiplication, the result is according to the cosine (2u) equation, where the result is a DC component and one component at 2u frequency. In the preferred embodiment, this becomes 176 Hz. In the preferred embodiment, this synchronous rectifier is realized as an analog circuit with the required accuracy.

The measurement converter 4 may also comprise amplifier and filter circuits. In the preferred embodiment the measurement converter contains low-pass filters, both at the input and at the output. The object of the input low-pass filter is to attenuate high-frequency noise, for instance coming from other medical equipments, and also to serve as anti-aliasing filter to prevent high frequency components from being received by subsequent circuits for time discretization. The output low-pass filter shall attenuate the 2u components that result from the multiplication operation in the synchronous rectifier so that only the signal near DC is used for further processing. By means of the choice of components and design details, moreover, the measurement converter may be designed with a view to obtaining high sensitivity and a low noise level. The control unit 5 comprises a time discretization unit 51 for time discretization of the signal from the measurement converter. The time discretization takes place at a sampling rate, which may advantageously be in the order of 20 to 200 samplings per second. The control unit further comprises an analog-digital converter 52, which converts measurement data to digital form. The choice of circuits for time discretization and analog-digital conversion implies technical decisions suitable for a person skilled in the art. In the preferred embodiment, time discretization is done in an integrated circuit, which combines oversampling, filtering and discretization. It should be understood that the above description of skin conductance measuring equipment, included in the control unit 5, is an illustrative example, and that numerous other skin conductance equipment may be envisaged by a skilled person.

The arrangement and number of electrodes may e.g. be varied. For instance, a two electrode system could be used. An example of a suitable, alternative arrangement that may substitute the three-electrode sensor means 3 and measurement converter 4 described above with reference to figure 1 , has been disclosed in Tronstad,

Martinsen: "Embedded instrumentation for skin admittance measurement", 30th Annual International IEEE EMBS Conference, Vancouver, British Columbia, Canada, August 20-24, 2008, in particular in Secton II Methods, subsection A, "AC skin admittance measurement". According to the approach used in this publiacation, an admittance measurement can either be done using constant current or constant voltage applied to the skin. For simplicity of the design with lesser amount of components, and because parallel admittance is measured, constant voltage may be used. The publication' s figure 1 shows the main components needed for a constant voltage skin admittance measurement using a typical analog lock-in amplifier design. A signal generator (Ve) generates an AC excitation signal at the desired measuring frequency which is applied to the skin through the electrodes E and M. The current passing through M is then equal to Ve- Yskin, and is converted to a voltage signal by the current to voltage converter (I-V). This signal is then demodulated by phase- sensitive detectors (PSD) into inphase and quadrature components. These signals are then lowpass filtered to obtain DC voltages proportional to the skin conductance at I and skin susceptance at Q.The control unit may advantageously comprise additional analog and possibly also digital inputs (not illustrated), in addition to the input from the measurement converter 4. In this case the control unit 5 can either be equipped with a plurality of analog-digital converters 52, or it can employ various multiplexing techniques well- known to those skilled in the art in order to increase the number of analog inputs.

These additional analog inputs may, for example, be arranged for additional electrodermal measurements, or for other physiological measurements which may advantageously be performed simultaneously or parallel with the electrodermal measurement. The control unit 5 also comprises a processing unit 53 for processing the digitized measurement data, storage means in the form of at least one store for storing data and programs, illustrated as a non-volatile memory 54 and a random access memory 55. The control unit 5 further comprises an interface circuit 61 , which provides at least one output signal 71. Another output signal 72 may also be provided.

Preferably, the control unit 5 further comprises a further interface circuit 81, which is further connected to an output unit 8 such as a display. The control unit 5 may also advantageously comprise a communication port 56 for digital communication with an external unit, such as a personal computer 10, possibly via a with a wired or wireless network (not shown). Such communication is well-suited for loading or altering the program which is kept stored in the memory 54, 55 in the control unit, or for adding or altering other data which are kept stored in the memory 54,55 in the control unit. Such communication is also well suited for read-out of data from the memory 54,55 in the apparatus, thus enabling them to be transferred to the external computer 10 for further, subsequent analysis or storage. The communication port 56 may be advantageously designed in accordance with established communication standards, such as USB, Ethernet, etc. Alternatively, the communication port may include a wireless transceiver, operating in accordance with WiFi or Bluetooth specifications, for instance.

In an embodiment the non-volatile memory 54 comprises a read-only storage in the form of programmable ROM circuits, containing at least a program code and permanent data, and the random access memory 55 comprises a read and write storage in the form of RAM circuits, for storage of measurement data and other provisional data.

The control unit 5 also comprises an oscillator (not shown), which delivers a clock signal for controlling the processing unit 53. The processing unit 53 also contains timing means (not shown) in order to provide an expression of the current time, for use in the analysis of the measurements. Such timing means are well-known to those skilled in the art, and are often included in micro controllers or processor systems which the skilled person will find suitable for use with the present invention. The control unit 5 may be realized as a microprocessor-based unit with connected input, output, memory and other peripheral circuits, or it may be realized as a micro controller unit where some or all of the connected circuits are integrated. The time discretization unit 51 and/or analog-digital converter 52 may also be included in such a unit. The choice of a suitable form of control unit 5 involves decisions, which are suitable for a person skilled in the art. An alternative solution is to realize the control unit as a digital signal processor (DSP).

The control unit 5 is arranged to read time-discrete and quantized measurements for the skin conductance from the measurement converter 4, preferably by means of an executable program code, which is stored in the non-volatile memory 54 and which is executed by the processing unit 53. It is further arranged to enable measurements to be stored in the read and write memory 55. By means of the program code, the control unit 5 is further arranged to analyze the measurements in real time, i. e. simultaneously or parallel with the performance of the measurements. In this context, simultaneously or parallel should be understood to mean simultaneously or parallel for practical purposes, viewed in connection with the time constants which are in the nature of the measurements. This means that input, storage and analysis can be undertaken in separate time intervals, but in this case these time intervals, and the time between them, are so short that the individual actions appear to occur concurrently. The processing unit 53, the memories 54,55, the analog/digital converter 52, the communication port 56, the interface circuit 81 and the interface circuit 61 are all connected to a bus unit 59. The detailed construction of such bus architecture for the design of a microprocessor-based instrument is regarded as well-known for a person skilled in the art.

The interface circuit 61 is a digital port circuit, which derives digital output signals 71 ,72 from the processing unit 53 via the bus unit 59 when the interface circuit 61 is addressed by the program code executed by the processing unit 53.

The first output signal 71 is a control parameter that represents the number of fluctuation peaks in the skin conductance signal through the interval.

The second output signal 72, if present, may represent another parameter derived entirely or partially from the measured skin conductance signal. The apparatus further comprises a power supply unit 9 for supplying operating power to the various parts of the apparatus. The power supply may e.g. be a battery or a mains supply.

The control unit 5 may be embodied as a configurable and/or programmable portable communication and processing device, such as a portable computer, a tablet computer or a smartphone.

The control unit 5 is configured to provide a skin conductance signal measured at an area of the user' s skin through a time interval, e.g. by means of the sensor means 3.

The control unit 5 is further configured to count the number of fluctuation peaks in the skin conductance signal through said time interval. To this end, the control unit 5 is configured to identify the fluctuations in the time- discrete, quantized measuring signal, by means of a program code portion which is stored in the nonvolatile memory 54 and which is executed by the processing unit 53.

The control unit 5 is further configured to output the number of fluctuation peaks through the interval as a control parameter, as the first output signal. 71. The control unit is also configured to use the number of fluctuation peaks in the skin conductance signal as a control parameter for controlling the user' s interaction with the non-medical application.

The non-medical application may e.g. be a computer game, implemented as a computer program. In an aspect, the computer program may be executed on a separate computer, including a tablet computer, a computer watch, an arm wrist band, or a smartphone. In this case, the control parameter represented by the output signal 71 may be transferred and input to the separate computer, e.g. through a communication network. In another aspect, the non-medical application, such as the computer game, may be held in the memory 54, 55 and executed by the processing unit 53 in the control unit 5.

In the case of the non-medical application being a computer game, the computer game may involve movement of movable objects, e.g. a ball, a game participant, a virtual vehicle such as a car, an aeroplane, a character, etc., on a display. The control parameter represented by the output signal 71 may, in an aspect, control one, two or three dimensional movement of such a moveable object in the game. In particular, the movement of the moveable object may be controlled forwardly or backwardly in dependency if the control parameter.

In a simple example, a virtual ball may be moved forwardly or backwardly on the display in dependency of the control parameter represented by the output signal. In another example, a speed of the moveable object, as displayed, may be controlled in dependency of the control parameter.

In an aspect, the computer game may have an educational learning objective, which may be changeable. In particular, the educational learning objective may be changed in dependency of the control parameter.

The computer game may be a one-player game or a multi-player game. In the latter example, multiple players may be equipped with separate apparatus according to the invention, and their skin conductance measurement may result in multiple control parameters which may be used to control individual objects in separate computer games or the same object in one single computer game. For instance, in a two- player game, the movement of a displayed object may be controlled along two axes, by means of two respective control parameters resulting from skin conductance measurements performed on the respective users. In other examples, the movement of a displayed object may be controlled along one, two or three axes in either a one- player game or a two-player game. Figure 2 is a flow chart illustrating a method for controlling a user' s interaction with a non-medical application, according to the invention. The non-medical application may e.g. be a computer game, as described above with reference to Figure 1. The method starts at the initiating step 31.

In the measurement step 32, a skin conductance signal measured at an area of the user' s skin through a time interval is provided. The measurement step 32 may include measuring a skin conductance on a palmar side of a hand of the user, or on a palmar side of a wrist of the user. Alternatively, it may include measuring a skin conductance on a plantar side of a foot of the user.

The skin conductance signal, or EDR (electrodermal response) signal, may be measured, time-quantized and converted to digital form using e.g. equipment of the type described with reference to fig. 1. A time-series of a certain duration, typically between 5 seconds and 40 seconds, and more preferably between 5 and 20 seconds, e.g. about 15 seconds, containing skin conductance data, may be acquired during this step. At 15 seconds, with a sampling rate of 20 - 200 samples per second, the time-series may contain 300 - 3000 samples. In the counting step 33, the number of fluctuation peaks in the skin conductance signal through the time interval is counted. A sliding interval may be used for the acquired time series, in such a way that a present number of fluctuation peaks in the skin conductance signal

corresponds to the number of fluctuation peaks for instance 15 seconds ago up to the time present.

To decrease the need of power use, the sampling time may be as described, but the duration of the time series may be 15 seconds, e.g. stored for about each 15 or 30 seconds.

The existence of a valid peak of the skin conductance signal may e.g. be established by checking that the derivative of the signal changes sign through a small period in the interval. The derivative of the signal is calculated as the difference between two subsequent sample values. In addition, it is possible to use a simple digital filter that needs to see two or more subsequent sign changes before the sign change is accepted. It may be useful to establish additional criteria for when a peak should be considered as valid. In their simplest form such criteria may be based on the fact that the signal amplitude has to exceed an absolute limit in order to be able to be considered a valid fluctuation. Such a reference value for the conductance may e.g. be between 0.01μ8 and 0.10 μ8, for instance 0.05 μ8.

Alternatively or in addition, the criteria may be based on the fact that the signal actually has formed a peak that has lasted a certain time. The criteria may also be based on the fact that the increase in the skin conductance signal value as a function of time must remain below a certain limit, typically 20 μ8/$, if the maximum value is to be considered valid.

Another possible condition for establishing a valid peak may be that the absolute value of the change in the conductance signal from a local peak to the following local valley exceeds a predetermined value, such as a value in the range [0.00 luS, 0.02uS], e.g. 0.005 μβ. Also, a maximum value appearing at the border of the interval, i.e. the starting point or ending point of the interval should preferably not be considered as a valid peak. The object is thereby achieved that artifacts, which can occur in error situations such as, e.g., electrodes working loose from the skin, or other sources of noise or disturbances, does not lead to the erroneously detection of peaks. In the control parameter use step 34, the counted number of fluctuation peaks in the skin conductance signal is used as a control parameter for controlling the user' s interaction with the non-medical application.

The control parameter may control a two or three dimensional movement of a moveable object in a computer game, as described above with reference to Figure 1. The movement of the moveable object may be controlled forwardly or backwardly in dependency if the control parameter, also as described above with reference to Figure 1. Alternatively, a speed of the moveable object may be controlled in dependency of the control parameter. In an aspect of the method, an educational learning objective of the computer game may be changed in dependency of the control parameter, in the same way as has already been described above with reference to Figure 1.

The computer game may be a single-player or a multi-player game, as has already been described above with reference to Figure 1.

As illustrated, the process ends at the terminating step 35. However, after step 34 the process may instead be repeated from step 32, if required.

Figures 3-8 are schematic figures illustrating further possible aspects of the invention. Figure 3 illustrates the invention used with a computer game in the form of a soccer game, wherein the posision of a moveable element, e.g. a virtuall ball, may be controlled horizontally by means of the control parameter derived from skin conductance measurements of a user. The skin conductance signal is measured at an area of the user' s skin through a time interval, a number of fluctuation peaks are counted during the time interval, and the number of fluctuation peaks in the skin conductance signal is used as the control parameter which controls the position of the moveable element, e.g., the virtuall ball. Hence, figure 3 may illustrate aspects of a single-user game.

Figure 4 illustrates the invention used with a computer game in the form of a soccer game, wherein the posision of a moveable element, e.g. a virtuall ball, may be controlled horizontally by means of the control parameter derived from skin conductance measurements of a user. Also, the position of the moveable element, e.g. the virtual ball, may be controlled vertically by means of a control parameter derived from the skin conductance measurements of another user, Hence, figure 4 may illustrate aspects of a multi-user game.

Figure 5 illustrates the invention used with a computer game in the form of a flight simulator, wherein the position of a moveable element, e.g. a virtual aeroplane, may be controlled horizontally and vertically by means of the control parameter derived from skin conductance measurements of a user. Figure 6 illustrates the invention used with a computer game in the form of a ping- pong game, wherein the posision of a moveable element, e.g. a virtuall ball, may be controlled in three dimensions by means of the control parameter derived from skin conductance measurements of a user. Figure 7 illustrates the invention used with a computer game in the form of a car racing game, wherein the position of a moveable element, e.g. a car, may be controlled in two dimensions by means of the control parameter derived from skin conductance measurements of a user. In such an example, influences from random outside factors may be overcome by the user' s effort, which may result in stress management training of the user.

Figure 8 illustrates the invention used with a computer game in the form of an astronomy game, wherein the position of a moveable element, e.g. a virtual spaceship among virtual celestial bodies, may be controlled in two or three dimensions by means of the control parameter derived from skin conductance measurements of a user.

Figure 9 illustrates the invention used with a computer game in the form of a princess game for children, wherein the position of a moveable element, e.g. a princess character, may be controlled in one, two or three dimensions by means of a control parameter derived from skin conductance measurements of a user.

Claims

1. Method for controlling a user' s interaction with a non-medical application, the method comprising

providing a skin conductance signal measured at an area of the user' s skin through a time interval,

counting the number of fluctuation peaks in the skin conductance signal through said time interval,

using the number of fluctuation peaks in the skin conductance signal as a control parameter for controlling the user's interaction with the non-medical application.

2. Method according to claim 1,

wherein the non-medical application is a computer game.

3. Method according to claim 2,

wherein the control parameter controls one, two or three dimensional movement of a moveable object in the game.

4. Method according to claim 3,

wherein the movement of the moveable object is controlled forwardly or backwardly in dependency if the control parameter.

5. Method according to claim 3,

wherein a speed of the moveable object is controlled in dependency of the control parameter.

6. Method according to claim 2,

wherein an educational learning objective of the computer game is changed in dependency of the control parameter.

7. Method according to claim 2,

wherein the computer game is a multi-player game.

8. Method according to claim 1,

wherein the step of providing a skin conductance signal includes measuring a skin conductance on a palmar side of a hand of the user.

9. Method according to claim 1,

wherein the step of providing a skin conductance signal includes measuring a skin conductance on a palmar side of a wrist of the user.

10. Method according to claim 1,

wherein the step of providing a skin conductance signal includes measuring a skin conductance on a plantar side of a foot of the user.

11. Apparatus configured to control a user' s interaction with a non-medical application, the apparatus comprising measuring equipment for measuring skin conductance at an area of the user' s skin, communicatively connected to a processing device, the processing device being configured to perform the following steps:

counting the number of fluctuation peaks in the skin conductance signal through said time interval, and

12. Apparatus configured to control a user' s interaction with a non-medical application, the apparatus comprising measuring equipment for measuring skin conductance at an area of the user' s skin, communicatively connected to a processing device, the processing device being configured to perform a method as set forth in one of the claims 1- 10.