WO2005051181A1

WO2005051181A1 - Pupillometric method

Info

Publication number: WO2005051181A1
Application number: PCT/EP2004/013548
Authority: WO
Inventors: Florian Kerkau
Original assignee: Freie Universität Berlin
Priority date: 2003-11-25
Filing date: 2004-11-25
Publication date: 2005-06-09
Also published as: DE10355880A1; DE112004002270D2

Abstract

The invention relates to a pupillometric method in which a brightness-corrected value of a person's pupil diameter (PDkorr) is obtained. According to the inventive method, the diameter of at least one pupil of a person is measured and a first pupil diameter signal (PDmess-Signal) is generated which represents the actual pupil diameter (PDmess). The ambient brightness is measured in the vicinity of the pupil that is to be measured, and a brightness signal (LD) is generated which represents said ambient brightness. A brightness pupil diameter value (PDhell) created by the brightness is determined in accordance with the determined brightness value (LD), and a corresponding brightness pupil diameter signal (PDhell-Signal) is generated. The first pupil diameter signal (PDmess-<->Signal) and the brightness pupil diameter signal (PDhell-Signal) are fed to a signal processing device (22) for generating a second pupil diameter signal (PDkorr-Signal) from the relation PDkorr = PDmess - PDhell, wherein PDhell = a + b⋅exp(-k⋅LD) with the provisions that a, b, k ∈ R, b <> 0, and k> 0, said second pupil diameter signal (PDkorr-Signal) corresponding to the brightness-corrected pupil diameter value (PDkorr). The parameters a, b, and k are determined from estimated values or by experiment.

Description

Patent application

Free University of Berlin Kaiserswerther Str. 16-18. 14195 Berlin

Pupillometric method

Technical field

The invention relates to a pupillometric method according to the preamble of claim 1 and a device according to the preamble of claim 7.

The invention further relates to certain applications and uses of the method and the device.

In particular, the invention relates to determining the physiological activation potential of a person.

The physiological activation potential ("arousel") of a person is interpreted as a correlate of the mental or emotional stress (mental workload) of the person and thus provides valuable information about the mental state of stress

Person. The mental stress level of a person is a quantity that is of great importance in terms of the person's performance. This information can be used for various purposes, e.g. in diagnostics or in process control using biofeedback signals.

Underlying state of the art

There are a variety of different methods for determining the physiological activation potential. Some of these methods are based on the measurement of the pupil width or the change in the pupil width of the eye (pupillometry), it being assumed that the pupil width changes with increasing physiological

Activation potential increased. In all pupillometric methods, the diameter of the pupil is measured, an actual value of the pupil diameter being determined. There are a variety of different measuring devices for this. Preferred such measuring devices contain a camera or video camera that works either in the visible or in the infrared spectral range. As an example here is the infrared video

Pupillography mentioned, in which the pupil is filmed by means of a video camera sensitive in the near infrared spectrum and the image is evaluated with computer support (see H.Wilhelm, B.Wilhelm, H.Lüdke: "Pupillography - Principles and Applications", Z.prakt. Ophthalmology (1996) 17: 327-336). Another device for measuring the pupil reaction of an eye is described in DE 44 19 489 AI.

However, since the pupil not only reacts to changes in the autonomic nervous system caused by emotional and mental stress, but also to other factors, a correction must be made in pupillometric methods to determine the physiological activation potential. The strongest factor influencing the pupil size is the prevailing luminance. The maximum change in pupil diameter caused by the change in luminance is approximately 8 mm, while the change in mental stress causes a change in pupil diameter of only approximately 2 mm. To date, however, it has been assumed that the sensitivity of the pupil diameter to changes in luminance in the environment prevents the pupilometric method from being used widely.

In the book by Peter Roessger "The Development of Pupillometry to a Method of Mental Stress in Ergonomics" (1997, ISBN 3-932490-03-7) the development of pupillometry as a method of measuring mental stress in. documented work science contexts. It is described that it is necessary to free the measured pupil diameters from the influence of brightness in such a way that the pupil diameter values determined are comparable with one another in order to obtain the measurement values for mental stress that can be used from the pupillometry. For this purpose, the measured pupil diameter is in one corrected pupil diameter converted, which is the theoretical value that the diameter would have with a luminance of 0 cd / m ² . It is assumed that the pupil diameter decreases linearly with increasing luminance, so that the following applies:

(1) PD _mess = PD _he ι ₁ + PD _k0IT ,

in which

PD _m ess the measured pupil diameter,

PD _or r the corrected pupil diameter at x = 0 cd / m ² , and PD _he ιι the pupil diameter caused by the brightness ("brightness

Pupil diameter value "), where

(2) PD _light = A-LD,

in which

LD is the current luminance, and A is a correction factor.

This results in

(3) PD _corr = PD _meSs - A-LD.

The correction factor A is the slope of the straight line that results when the pupil diameter is plotted against the brightness.

The correction factor A differs individually and must be determined again for each test person. To this end, the test subjects are presented with two luminance levels (calibration point luminances) LD _d un and LD _he i and the pupil diameters (PDdun and PDh _e i) that occur are measured. The correction factor is then determined from the change in pupil diameter between the two calibration point luminances:

(4) A = (PD _dun - PD _hel ) / (LD _dun - LD _hel ) It is mentioned that the calibration point luminance should not be too far from the luminance that occurs during the actual measurement, since the linearity assumption between pupil diameter and luminance is only approximate and applies to narrow luminance ranges.

A device for determining the vigilance state of a person is known from DE 198 03 158 CI. The device contains an image recording system with a CCD video camera and infrared lighting for recording images of the area of at least one eye of the person and an image evaluation system which evaluates the images recorded by the image recording system. The image evaluation system has means for determining the pupil diameter. The vigilance state is inter alia determined by an evaluation device. depending on the pupil diameter state information obtained by the pupil diameter determining means. The image evaluation system contains an image processing unit, to which the images recorded by the video camera are transmitted. The image processing unit works with two operating states, between which it is constantly switched back and forth. In a first operating state, when the infrared lighting is switched off, the ambient light, i.e. Extraneous light, the average brightness produced are measured and this information is output as a brightness signal. In a second operating state, the infrared lighting is switched on and the image processing unit evaluates the image obtained from the monitored eye. In this evaluation, the pupil diameter of the observed eye is determined by the pupil diameter determination means. The image evaluation unit outputs the information about the current pupil diameter in the form of a pupil diameter signal. Furthermore, a correlation unit correlates the time profile of the continuously measured

Pupil diameter with the information about the ambient brightness obtained via the ambient brightness signal. This correlation is intended to eliminate the normal pupil response to fluctuations in the ambient brightness as a disturbing influence. The correlation unit emits a corresponding correlation signal, which contains information about the extent to which changes in the pupil diameter are caused by a changing ambient brightness or not. However, it is not specified how this brightness signal is determined. Many conventional interfaces for human-machine interaction (in particular computer interfaces) are designed in such a way that a request for action by the machine is answered by a human action. Input devices such as a mouse and keyboard play a major role in this form of interaction. Entries are therefore subject to cognitive control by the user and are accordingly imprecise in certain areas.

However, it is known to carry out process controls on the basis of biofeedback signals attributable to the physiological activation potential of a person. US 5,896,164 discloses a bio-feedback control of an image output device. The image of the image output device is changed by a signal changing unit. The signal change unit is acted upon by a control signal which is obtained from the psychophysiological parameters (e.g. electrodermal activity) of a user.

Disclosure of the invention

The invention has for its object to provide a method and an apparatus of the type mentioned by which a reliable determination of a value of the pupil diameter of a person corrected in terms of brightness is made possible, in particular in the case of people viewing a screen.

The invention is also based on the object of creating a method or a device of the type mentioned at the outset, by means of which the determined state of a person provides a reliable measure of the person's physiological activation potential.

According to the invention these objects are achieved by a method according ^'to claim 1 or a device according to claim solved. 7

The invention is also based on the object of a new method or a new one

To provide device for process control of a device. According to the invention, this object is achieved by a method or a device according to claim 9.

In the present invention, the pupillometric measurement of the body activation establishes a direct connection to the autonomous nervous system of the user. Physical activation as an indicator of mental processes is not subject to arbitrary control of the cognitive apparatus and is therefore very reliable.

So far it has been assumed that the problem of eliminating the influence of brightness in pupillometric methods leads to pupillometric

Methods in certain applications, especially in the case of strongly fluctuating ambient brightness, cannot be used to determine the physiological activation potential. This is due to the fact that a linear relationship between the pupil diameter (PDheii) caused by the brightness and the current ambient luminance (LD) has been assumed to this day. The greater the brightness fluctuations in the environment, the greater the problem, e.g. in front of a screen or a television.

Surprisingly, it has now been shown by experimental investigations that pupillometric methods provide usable values for the pupil diameter corrected for the brightness or the physiological activation potential if one exponentially relates the pupil diameter caused by the brightness (PD _h eii) and the current ambient luminance ( LD) assumes.

According to the invention, the brightness pupil diameter value PD _he ii is thus determined on the basis of an essentially exponential dependency PD _he ii = a + b-exp (-k-LD) between the brightness value LD and the brightness pupil diameter value PD _he ii, where a, b and k are real numbers (ie a, b, ke R), b is non-zero (ie b ≠ ö) and k is greater than zero (ie k> 0). In the simplest case, the parameters a, b and k in the exponential dependence PDheii = a + b-exp (-k-LD) can be formed from estimated values. Achieved better accuracy however, if these parameters a, b and k are determined from experimentally determined values. The values used to determine parameters a, b and k can be determined experimentally at a specific location (for example at a workstation in front of a screen) and / or at a single person. This person is then preferably the person whose value of the corrected for the brightness

Pupil diameter or its physiological activation potential to be determined later in certain activities. The values used to determine the parameters a, b and k can, however, also be determined experimentally on several persons, in which case mean values of the parameters a, b and k can be formed.

The present invention can be used not only for determining the brightness of the pupil diameter corrected for the brightness or the physiological activation potential, but also for process control of a device in the sense of a human-machine interface ("bio-feedback"), in particular for controlling multimedia learning systems , For this purpose, a control signal is generated from the determined corrected pupil diameter value PD _corr and applied to the device, with process control of the device taking place as a function of this control signal. (Such bio-feedback process controls are known per se, in particular using other measured variables (for example skin conductance) as a basis.) By the assumption according to the invention of an exponential relationship between the pupil diameter caused by the brightness (PD _h eii) and the current ambient luminance ( LD), however, the present invention provides a human-machine interaction with wide-ranging fields of application.

Embodiments of the invention are the subject of the dependent claims.

Exemplary embodiments of the invention are explained in more detail below with reference to the accompanying drawings. Brief description of the drawings

Fig. 1 is a schematic representation and shows a user at a workplace with a Arbeitsmoήitor, as well as parts of a pupillometric device.

FIG. 2 is a schematic block diagram and illustrates a sample embodiment of a pupillometric device.

3 shows an experimentally recorded measurement curve for determining the parameters a, b and k in the equation PD eii = a + b-exp (-k-LD).

A preferred embodiment of the invention in the exemplary embodiment shown here is the pupillometric method or the pupillometric device for determining the value of the pupil diameter or the physiological value corrected for the brightness

Activation potential of a person 10 (“user”) is used in a work station with a work computer (“subject PC”) (not shown in the figures) with work monitor 12. (The keyboard 14 of the work computer is also indicated in FIG. 1.) The work monitor 12 generates a strongly fluctuating ambient luminance which has to be taken into account by the pupillometric method.

In the exemplary embodiment of the pupillometric method or the pupillometric device shown here, two commercially available computers are used. The first computer 16 ("control PC") is a commercially available infrared camera 18 (REM π from SensoMotoric Instruments GmbH, Teltow) for measuring the

Pupil reaction of an eye 20 of the user 10 connected. The eye 20 of the user 10 is filmed by the infrared camera 18 and the video data obtained in this way are fed to the first computer 16, in the first computer 16 these video data are evaluated by means of a video card ^• and the current actual pupil diameter (PD _measurement ) of the eye 20 is determined , From this we get a corresponding one

Pupil _diameter signal PD _meSs signal generated. A commercially available luxmeter 24 (Testo 545 from Testo AG, Lenzkirch) is connected to the second computer 22 (“stearing PC”). The luxmeter 24 is used for light measurement in the vicinity of the pupil to be measured and supplies data about the ambient brightness in the form of brightness values LD to the second computer 22. In this computer, these brightness values are converted into brightness pupil diameter values

(PD _he ii) converted and corresponding brightness pupil diameter signals PDheir signal are generated.

The second computer 22 is also supplied with the pupil diameter data determined in the first computer 16 in the form of pupil diameter signals PD _measurement signal. In the second computer 22 these pupil diameter _signals PD _meSS -

Signal and the ambient light brightness data (LD) processed by means of an analysis program described later. From the ambient light brightness data

(LD), the corresponding brightness pupil diameter values (PDheii) and corresponding brightness pupil diameter signals PD _he ii signal are determined using a function described in more detail later. From the pupil diameter _signals PD _raeSs -

Signal and the brightness pupil diameter signals PD _he ii signal are corrected

Pupil diameter values (PD _kor r) and corresponding pupil diameter signals

PDkorr signal determined, where PDkorr = PD _mess - PDheii applies (see Eq. (1)). This corrected pupil diameter value (PD _corr ) can then be used as a correlate of the physiological

Activation potential of the user. From this can be mental and emotional

Stresses are derived.

In the analysis program, the two data records (PD _mess and LD) are read in as tables. When calculating the brightness pupil diameter values (PDheii), it is assumed that the brightness pupil diameter values (PD _he ii) decrease exponentially with increasing luminance (LD). The brightness pupil diameter values (PD _he ii) can be calculated from the read brightness values (LD) according to the following equation:

(5) PD _he ii = + b-exp (-k-LD), where a, b and k are experimentally determined parameters.

Here a corresponds to the pupil diameter in extreme brightness (LD = infinite) and a + b the pupil diameter in absolute darkness (LD = 0). The parameter k describes how quickly the pupil diameter changes with increasing luminance

Minimum value a approaches.

The parameters a, b and k in equation (5) are determined experimentally by suitable measurements of the luminance (LD) at the workplace shown in FIG. 1 and the associated brightness pupil diameter values (PD _he ii) of users. The calibration for determining a and b is carried out using a black (dark) monitor and a white (light) monitor. The measurements can be carried out on several users and the parameters a, b and k can be determined from these measurements as mean values. It is also possible to carry out the measurements directly only on the user whose physiological activation potential is to be determined later. In the exemplary embodiment shown here, the parameters a, b and k are calculated with a nonlinear least-square fit with data from several users.

The analysis program ("mental workload analyzer") takes over the measured values of the pupil diameter and cleans them up for any measurement errors (such as those caused by blinking) by eliminating these extreme values. For this purpose, an average and the associated standard deviation are determined. All values that are more than two standard deviations away from the mean are ignored for the further calculations. The remaining values are then around the

Contamination by light influences using the in Eq. (5) function described cleaned up. The remaining net values are saved by the program and can be processed graphically or in tabular form, but they can also be passed on directly to process control.

The values of the pupil diameters PD _meS s, PDheii and PDkorr as well as the values of the parameters a and b are given in suitable units of length, which, however, depend on the Depending on the measurement method, in the exemplary embodiment shown, these values are given in number of pixels of the sensor (camera) used.

As an example of a series of experiments to determine parameters a, b and k, the measured pupil values (unit: number of pixels) are plotted against the luminance (unit: cd / m ² ) in FIG. 3. From 2500 measurements on 250 people at a typical computer workstation, the following (rounded) values were determined from the curve: a = 623; b = 987 and k = 0.24.

The evaluation of the data can either in

In table form or as online processing in a graphic. With the direct online processing of the data, the mental state of the user can be recorded in real time. This database can be used to control adaptive computer systems. For example, a learning environment can be continuously adapted to the performance of the user to ensure the best possible learning process. The learning process can experience a significant increase in efficiency through the targeted coordination of user ability and system requirements.

The control of adaptive computer systems may be mentioned as a first example of an area of application of the method and the device according to the invention. Since the pupil size serves as a physical correlate of a mental stress situation, the current level of the mental stress of the user is possible in real time by measuring the pupil. Using the data from the pupillometric method, the work computer can control the difficulty of the application in such a way that the load level of the user is as favorable as possible.

By coordinating the user ability with the load of the program, a good learning performance can be achieved through flow experience.

Media analysis may be mentioned as a second example of an area of application of the method and the device according to the invention. By measuring the mental and emotional demands on the recipient, important analysis data can be provided for media research, for example at Commercials, thrillers or computer applications. In this area, a bio-feedback device based on the method according to the invention can advantageously replace the previously used method of measuring the skin conductance, since the pupil is the faster and more accurate indicator of the activity of the autonomic nervous system. Furthermore, no annoying wiring by the user is necessary.

Claims

claims

1. Pupillometric method, in which a value of the pupil diameter (PD _corr ) of a person corrected for the brightness is achieved, with the method steps:

(a) measuring the diameter of at least one of the person's pupils,

(b) generating a first pupil _diameter signal (PD _meSs signal) which _reflects the actual pupil diameter (PD _meS s) from this measurement,

(c) measuring the ambient brightness in the vicinity of the pupil to be measured, (d) generating a brightness signal (LD) reflecting this ambient brightness from this measurement, characterized in that (e) a brightness pupil diameter value caused by the brightness (PD _he ii) determined in dependence on the determined brightness value (LD) and a corresponding brightness pupil diameter signal (PD _he ii) signal is generated, (f) the first pupil diameter signal (PD _measured signal) and the brightness pupil diameter signal (PD _he ii- Signal) are fed to a signal processing device (22) to generate a second pupil diameter signal (PD _corr signal) corresponding to the brightness-corrected value of the pupil diameter (PDkorr) from the relationship PDkorr = PDmess "PDheii, where PD _he ii = a + b -exρ (-k-LD) with a, b, ke R, b ≠ 0 and k> 0, in which

(g) the parameters a, b and k are determined from estimated values or experimentally determined values.

2. Pupillometric method according to claim 1, characterized in that the values used to determine the parameters a, b and / or k are determined experimentally at a specific location and / or on a single person.

3. Pupillometric method according to claim 1 or 2, characterized in that the values used to determine the parameters a, b and / or k are determined experimentally on several people.

4. Pupillometric method according to one of claims 1-3 for determining the physiological activation potential of the person.

5. Application of the method according to one of claims l 4 to a person viewing a screen.

6. Application according to claim 5, characterized in that the screen is part of a television set or a monitor of a workplace.

7. Device for determining a brightness-corrected value of the pupil diameter (PDkorr) of a person, comprising:

(a) means for measuring the diameter of at least one of the pupils of the person and for generating a first pupil diameter signal (PD _m e _SS signal) reflecting the actual pupil diameter (PD _mess ) from this measurement, and (b) means for measuring the ambient brightness in the vicinity of the pupil to be measured and for generating a brightness signal (LD) reflecting this ambient brightness from this measurement, characterized by

(c) means for determining a brightness pupil diameter value (PD _he ii) caused by the brightness as a function of the determined brightness value (LD) and for generating a corresponding brightness pupil diameter signal (PD _he ii signal), and

(d) Signal processing means for generating a second pupil _diameter signal (PD _ko signal) corresponding to the value of the pupil diameter (PD _corr ) corrected for the brightness from the first pupil diameter signal (PD _mess signal) and the brightness pupil diameter signal (PD _he ii signal ) from the relationship PDkorr = PDmess - PD _h ell, where PDheii = a + b-exp (-k-LD) with a, b, ke R, b ≠ 0 and k> 0.

8. The device according to claim 7, characterized by means for determining the parameters a, b and / or k in equation PD _he ii = a + b-exp (-k-LD) according to any one of claims 1-3.

9. Use of the method according to one of claims 1-4, the application according to claim 5 or 6 or the device according to claim 7 or 8 for process control of a device in which a control signal generated as a function of the determined corrected pupil diameter value (PD _corr ) Device is switched on and process control of the device takes place as a function of this control signal.

10. Use according to claim 9, wherein the process control is used to control multimedia learning systems.