WO2004021157A1

WO2004021157A1 - Method and apparatus for generating control signals using electro-oculographic potentials

Info

Publication number: WO2004021157A1
Application number: PCT/GB2003/003806
Authority: WO
Inventors: David Nugent; Karla Ziri-Castro; Jesus Lopez; Cedric Babu
Original assignee: University Of Ulster
Priority date: 2002-09-02
Filing date: 2003-09-02
Publication date: 2004-03-11
Also published as: AU2003269107A1; AU2003269107A8; GB0220265D0

Abstract

Control signals are generated by means of electrodes (1) contacting the skin adjacent a users eyes to detect electro-oculographic (EOG) signals produced by movement of the eyes. The EOG signals are amplified, band-pass filtered and integrated before being applied to level detectors (8) to derive digital signals which are stored in memory (14) and then decoded (18) to provide a number of trigger signals (17). In a preferred form, left, right and up motions provide 3-bit words which are decoded to give eight possible trigger signals, which may be used to simulate a computer mouse.

Description

"Method and Apparatus for Generating Control Signals Using Electro-Oculographic Potentials"

This invention relates to the generation of control signals using electro-oculographic potentials.

The invention is particularly, but not exclusively, useful in controlling the use of a personal computer (PC) by means of eye movements only. The ready availability of sizeable computing power gives the possibility for severely disabled persons to communicate and to control powered devices despite their physical limitations.

A number of means for enabling severely disabled persons to control computers exist. Most of these require some small degree of mobility, for example by use of a mouth-held pointer or by detecting head movement. However, there are people whose physical disability is so severe that they have controlled movement only of the eyes. There are known systems which detect eye movement and use this to control computers, but the known systems are complex and expensive .

It is possible to detect the movement of the eye with infrared technology [3] . Such systems have the disadvantage of using light reflected from the eye and thus being prone to interference from other sources .

It is known to detect eye movement by use of advanced image processing techniques to detect retinal movement [4] . This can be considered to be more of a diagnostic tool which is reflected in its high cost and is not directly reported to be an access technology, i.e. an interface or device which enables a user to control external devices via eye movements.

Scientific studies have been reported which utilise brain wave analysis to detect eye orientation or eye movement [5] , but these are not commercially viable owing to their technical intricacies.

Other prior art systems require particular visual stimuli to be presented to the user and corresponding reaction codes to be monitored.

The present invention has as one object the provision of a simple, cheap and reliable system which enables a computer or other device to be controlled solely by eye movements. The present invention in one aspect provides a method of generating a control signal comprising detecting at least one electro-oculogram (EOG) signal at the skin adjacent the eye of a user, and processing said EOG signal to form an input suitable for controlling a predetermined device.

From another aspect, the invention provides apparatus for use in generating control signals, the apparatus comprising electrode means for placement on the skin of a user adjacent the eye, and signal processing means connected in use to receive EOG signals from said electrode means, the signal processing means being adapted to process said EOG signal to form an input suitable for controlling a predetermined device.

The invention further provides a control signal generating device for use in the foregoing method, the device comprising a frame of the nature of spectacles or goggles dimensioned to fit on the face of a user, and a plurality of electrodes mounted on the interior of the frame so as to be, in use, held against the user's skin adjacent the eyes.

Preferred features and advantages of the invention will be apparent from the claims and from the following description.

An embodiment of the invention will now be described, by way of example only, with reference to the drawings, in which: Figure 1 illustrates part of one system embodying the present invention, showing acquisition of EOG signals; Figure 2 shows signal processing steps used in this embodiment; Figure 3 illustrates the nature of the EOG signals after filtering and ampli ication; Figure 4 is an integrator used in this embodiment; Figure 5 illustrates a phase detection operation; Figure 6 shows a level detection and memory stage ; Figure 7 is an overall schematic of this embodiment; Figure 8 is a truth table for a 3 -bit decoder of Figure 7; and Figure 9 is a perspective view of one embodiment of control signal generating device.

The invention is based upon making use of the electro-oculogram (EOG) signal. The EOG itself is generated from the stationary potential which exists between the cornea and the retina of the eye. This stationary dipole can be used to sense or to measure the position of the eye.

Referring to Figure 1, electrodes 1 placed on the skin around the eyes make use of the potential between the cornea and the retina to record a zero voltage when the gaze is straight ahead, but a negative or positive voltage as the gaze moves out of centre. In this example, electrode la between the eyes acts as a common ground, with electrodes lb at the outer corners of the eyes detecting left and right horizontal movement. Electrodes lc above and below one eye detect vertical movement . the _y electrodes 1 are silver-silver chloride electrodes which are readily available for skin surface measurements, but any other suitable electrode may be used.

There is an almost linear relationship between the horizontal (or vertical) angle of the gaze and the voltage produced [1] , and this voltage is called the electro-oculogram (EOG) .

In Figure 1, the horizontal electrodes lb and vertical electrodes lc provide differential inputs to two recording instrumentation amplifiers 2 with high common mode rejection and a frequency response from 0 to 30 Hz. EOG signals are conventionally regarded as lying in the 0 - 30 Hz band and processed as such; however, a preferred feature of the present invention is to restrict the signal to be processed to a narrower band, preferably 3 - 10 Hz, which is effected by bandpass filters 3. This filtering has the benefit of permitting high integrity detection of long swing eye movements with eventual eye blinks, without DC component interference and high EMG artefact filtering. detected potentials may range from 0.05 to 3.5 mV. When sight attention remains fixed, the stationary eye dipole locates itself symmetrically between the two opposite electrodes and thus the EOG signal is zero. If for example the sight is directed towards the left, the cornea, which possesses a positive charge, ensures the left-most electrode lb is more positive. In the same way, this phenomenon exists in the upper and lower electrodes lc which register vertical movements [2] .

Figure 2 gives an overview of the signal processing strategy.

After amplifying and filtering the EOG signal associated with a discrete movement, an EOG signal in the form of a biphasic sweep 4 is obtained. As seen in Figure 3, the signal 4 has a phase of zero degrees (as at 5) for a left movement and a phase of 180 degrees (as at 6) for a right movement. Similarly, an up movement has a phase of zero degrees, and a down movement 180 degrees. A phase detection stage 7 detects the phase and provides an input to a level detection stage 13, which in turn drives a memory module 16.

The phase detection stage 7 of this example is shown in more detail in Figure 4 and is based on an integrator using an operational amplifier 8. The integrator uses an input resistor 10 and feedback capacitor 9 to give the relationship

If CR = Is (C = lμF, R = 1MΩ, as practical values) , then

The effect on the signal is shown in Figure 5. After the integrator 12 the signal energy concentrates more up or down depending on the associated discrete movement.

This processes the EOG signal up to the level where simple level detection by level detectors 8 (Figure 6) is enough to accomplish a discrete finite control signal. It is desirable, however, to sustain such signal in time. This is achieved using D-type flip- flops 14 in a divider configuration (output connected with input) to form the memory stage 16 (Figure 2) . The flip-flops 14 set a logic 0 or a logic 1 sequentially each time it receives a rising edge on its clock input. In this example, three flip-flops 14 (for left, right and up) enable a 3- bit word 15 to be generated for control purposes. The 3 -bit word 15 makes possible 8 (2³) different combinations and thus 8 control triggers.

Figure 7 shows the overall schematic of the present embodiment. The 3 -bit word 15 is applied to a 3 -bit decoder 18 to generate the control triggers 17. Figure 8 is an example of the truth table for the decoding .

The foregoing example uses the eye movements left, right and up to give three bits and thus eight possible trigger signals because eight triggers are suitable for simulating the various control functions of a computer mouse. However, the invention can be incorporated in systems giving fewer or more bits and thus fewer or more trigger signals. For example, using only left and right movements would give two bits which can be decoded to give four trigger signals; or left, right, up and down movements would make four bits available and thus 16 possible trigger signals.

The present invention is thought to be particularly useful in simulating the operation of a computer mouse, since the output can readily be compatible with Microsoft Accessibility Options. The invention may, however, be used in other control schemes, for example to control the movement of a powered wheelchair, or to provide inputs for controlling the operation of aircraft systems while the pilot's hands and feet are engaged with the primary controls.

The preferred forms of the invention give an output which is essentially digital, and thus interfacing with computers and other digital systems is facilitated. Figure 9 shows an example of a suitable device 20 for engaging the electrodes with the user's face. The device 20 comprises a frame 21 defining apertures 22, and legs 23 for locating the device on the user's ears. The apertures 22 may be left open, glazed with clear glass, or glazed with prescription lenses, as desired. The electrodes 1 are secured to the frame so as to contact the appropriate parts of the user's face. In order to provide the necessary contact, the frame 21 may be more akin to goggles or a ski mask than conventional spectacles, and may be provided with a resilient cushion to ensure contact with the face. For the same reason, the legs 23 may be replaced with an elastic strap or cord.

The device 20 can communicate with a computer or other controlled device via a cable (not shown) . Alternatively the device 20 can incorporate a wireless communication device such as a Bluetooth module. This has the advantage that the user device 20 can communicate with a number of external devices each equipped with a compatible Bluetooth module, and to reduce the complexity of the communications process the identification of the device to be controlled can be based on simple proximity.

Interoperability is promoted by the conversion of the eight triggers into common functionalities, for example up, down, on, off etc. These can be pre- configured during initialisation of the suite of external devices to be deployed by the user. In addition to being able to preconfigure and customise the communications protocol and the trigger commands, variability of EOG between persons and intra-eye variability can also be accommodated. By means of initialisation training software, each user can personalise the operation of the control signals and indicate the typical operational range of the EOG for each eye for up, down, left and right motions. This accommodates the varying needs of users who may present differences in the physical operation of their eyes.

In controlling a computer, the trigger signals can be directly mapped onto left, right, up and down cursor movements following the corresponding eye movements. In addition simple blinking of each eye can be used to represent click, double click, and drag and drop operations . No additional software is required on the computer to support this, as the digital signals produced by the device are compatible with Microsoft's accessibility options.

Testing of one example of the invention on a cohort of 15 patients resulted in reports of ease of use and acceptability. Each found it to be user friendly, easy to use and quick to learn. the mean effective learning time, while patients were fed with a feedback signal protocol, was 22 minutes.

The invention thus provides a reliable interface which permits the user to control an external device using only eye movements . The technology will not impeded the user and will not suffer from interference. The invention may be used across the whole spectrum of disability, and also by persons without disability. The user device can be made in an aesthetically acceptable form.

References

1. www.mcbrown . btinternet . co .uk/medict/entries/ELE CTROOCU OGRAPH .html

2. Kris, C, "Electro -oculography" , in O. Glasser (ed.), Medical Physics, Vol III, 692-700 (1960)

3. US Patent No 4,081,623 (Vogeley)

4. Arabnia, H R, A computer input device for medically impaired users of computers, Proc. John Hopkins Nat Search for Computing Applications to Assist Persons wi th Disabili ty, pp. 131-134 (1992)

5. US Patent No 4,651,145 (Sutter)

Claims

1. A method of generating a control signal comprising detecting at least one electro-oculogram (EOG) signal at the skin adjacent the eye of a user, and processing said EOG signal to form an input suitable for controlling a predetermined device.

2. A method according to claim 1, in which the EOG signals detected correspond to eye movements selected from one or more of left, right, up and down.

3. A method according to claim 1 or claim 2, in which the EOG signal, or each EOG signal, is filtered to pass a band of 3 - 10 Hz .

4. A method according to claim 3, in which the or each filtered signal is amplified before being further processed.

5. A method according to claim 3 or claim 4, in which the or each filtered EOG signal is processed by phase detection followed by level detection.

6. A method according to any preceding claim, in which three signals are used, and in which three-bit decoding is used to provide possible eight trigger signals.

7. A method according to claim 6, in which the trigger signals are used to simulate the operation of a computer mouse .

8. A method according to claim 6 or claim 7, in which the three signals are left, right and up.

9. Apparatus for use in generating control signals, the apparatus comprising electrode means for placement on the skin of a user adjacent the eye, and signal processing means connected in use to receive EOG signals from said electrode means, the signal processing means being adapted to process said EOG signal to form an input suitable for controlling a predetermined device.

10. Apparatus according to claim 9, in which the electrode means comprises electrodes disposed to either side of and above and below an eye of the user.

11. Apparatus according to claim 10, in which the electrode means further includes an electrode disposed adjacent the outer side of the other eye of the user.

12. Apparatus according to claim 10 or claim 11, in which said electrodes are disposed on the interior of goggles or spectacles wearable by a user such that, in use, the electrodes rest upon the user's skin.

13. Apparatus according to any of claims 10 to 13, in which the electrodes are silver-silver chloride electrodes.

14. Apparatus according to any of claims 9 to 13, including one or more bandpass filters arranged to filter the electrode signals to form one or more EOG signals for processing.

15. Apparatus according to claim 14, in which the or each bandpass filter passes a band of 3 - 10 Hz.

16. Apparatus according to claim 14 or claim 15, including amplifier means connected to amplify the signals applied to the bandpass filters .

17. Apparatus according to claim 16, in which the amplifier means comprises one or more instrumentation amplifiers.

18. Apparatus as claimed in any of claims 9 to 17, in which the signal processing means comprises, for each EOG signal used, a phase detector followed by a level detector.

19. Apparatus as claimed in claim 18, in which the signal processing means further includes, for each EOG signal used, a memory connected to the output of the level detector.

20. Apparatus as claimed in claim 19, in which each memory comprises a D-type flip-flop.

21. Apparatus as claimed in claim 19 or claim 20, in which the electrode means is arranged to produce EOG signals corresponding to left, right and up, and in which the memories are connected to a 3-bit decoder to provide eight possible control triggers.

22. Apparatus according to any of claims 9 to 21, including a wireless communication means for transmitting trigger signals to a device controlled by the apparatus.

23. A control signal generating device for use in the method of claim 1, the device comprising a frame of the nature of spectacles or goggles dimensioned to fit on the face of a user, and a plurality of electrodes mounted on the interior of the frame so as to be, in use, held against the user's skin adjacent the eyes.

24. The device of claim 23, in which there are five electrodes positioned to contact the skin at the nose, the outer end of each eye, and above and below one eye.

25. The device of claim 23 or claim 24, including a wireless communication means for transmitting signals to a nearby receiver.