GB2579618A

GB2579618A - An event-related optical signal device and a brain-computer interface system

Info

Publication number: GB2579618A
Application number: GB1819933.1A
Authority: GB
Inventors: Robert Dacombe James
Original assignee: Individual
Current assignee: Individual
Priority date: 2018-12-06
Filing date: 2018-12-06
Publication date: 2020-07-01
Anticipated expiration: 2038-12-06
Also published as: GB201819933D0; GB2579618B

Abstract

An event-related optical signal (EROS) device 10 for a brain-computer interface, the device comprising at least one light source 6 configured to flash at a rate of at least 1000 hertz at a wavelength such that the light is scattered by brain tissue. Additional embodiments also include means of detecting and outputting a response, and the interpreting of this output by a computer, which may apply machine learning. The wavelength used may be between 700nm and 900nm such that human skin and bone are mostly transparent to it, in the near infrared (NIR) area of the spectrum. The device may be a wearable headset and all of the components may be arranged on the same circuit board. The light source may be positioned 4 to 8 centimeters from the detection means.

Description

AN EVENT-RELATED OPTICAL SIGNAL DEVICE AND A BRAIN-COMPUTER INTERFACE SYSTEM

FIELD OF THE INVENTION

The invention relates to an event-related optical signal device for a brain-computer interface and a brain-computer interface system.

BACKGROUND OF THE INVENTION

The brain is the most complex organ which governs over the whole body. The human brain is extremely complex and contains approximately 100 billion neurons, each neuron is connected by synapses to thousands of other neurons. These neurons communicate with one another by means of long protoplasmic fibers called axons, which carry trains of signal pulses called action potentials to distant parts of the brain or body targeting specific recipient cells. Most commonly, the brain is involved in the control of the functions of the associated body. The brain may be considered the interface with the body. Work has been conducted on interpreting the signals from the brain with intentions of controlling the body and its thoughts, and research into brain-computer interfaces has been carried out to interpret and interface with the brain.

Typically, computers are controlled using a keyboard and mouse or by touch inputs, but this is inefficient and it takes a long time to control the computer compared to if one's thoughts could control the computer, This is becoming especially relevant in virtual and augmented reality where a user has to either use a keyboard and mouse, a controller, or their hand gestures. These existing technologies are difficult for a user to control and if these technologies were connected to the brain's output using brain imaging techniques then these could be vastly improved. Typing with fingers or thumbs is very slow and finding a technology which could increase the speed of interacting with computers would be vastly beneficial.

A brain-computer interface (3C1) is an interface to interpret the brain which is generally comprised of two parts, input, and output. The output receives information from the brain through the use of brain imaging/scanning and received signals can be transformed into computer commands, Input is stimulating the neurons to change how the neurons are firing and therefore changing the output. Both output and input can be executed invasively or non-invasively. Invasive techniques rely on some form of surgery or injections to deliver the hardware to the brain. An example of this is placing electrodes directly onto the brain:issue whilst open skull surgery has taken place. This provides quite a lot of issues including damaging brain tissue, the risk of the body rejecting the hardware, and powering this hardware. Non-invasive methods remove the need to perform surgery and the hardware is not implanted on the skull and can be worn on the head or attached to an external machine such as a computerised tomography (CT) scan.

There have been studies of brain-computer interfaces with a variety of results. Early studies focused on invasive methods but these were difficult to scale up because of the dangers caused by invasive hardware. However, more recently with cheaper and more powerful technologies becoming available such as Electroencephalograms (EEGs) and Functional Near-Infrared Spectroscopy (fNIRS), more studies have been carried out with non-invasive methods.

Non-invasive techniques for measuring brain activity have become more accurate over the past decade with smaller and more powerful components and a greater understanding of r0 cience.

The most popularly used non-invasive technique is the electroencephalogram (EEG). This is an electrophysiological monitoring method used to record electrical activity of the brain. Typically, the electrodes are placed along the scalp of the users head. The EEG measures voltage fluctuations resulting from ionic current within the neurons of the brain. Typically, these devices are cheap to build but suffer from poor resolution compared to other technologies, and they also suffer from a lot of background noise making it difficult to obtain useful information, especially when examining more complicated parts of the brain.

Another example existing non-invasive technique is functional Near-Infrared Spectroscopy (fNIRS). This technique involves detecting the changes in blood oxygenation and flow that occur in response to neural activity. Light emitting diodes (LEDs) are used to shine light into the brain tissue and measure hemoglobin levels which correlate to neural activity. The NIR spectrum takes advantage of the, optical window in which skin, tissue, and bone are mostly transparent to NIR light while hemoglobin (Hb) and deoxygenated-hemoglobin (deoxy-Hb) are stronger absorbers at'these wavelengths. Differences of absorption spectra of deoxy-Hb and oxy-Hb allow the measurement of relative changes in hemoglobin concentration though the use of light attenuation at multiple wavelengths. Two or more wavelengths are selected, with one wavelength above and one below the isosbestic point of 810nm at which deoxy-Hb and oxy-Hb have identical absorption coefficients. Using the modified Beer-Lambert law, relative concentration can be calculated as a tunction of total photon path length. Typically, the lights emitter and detector are placed ipsilaterally (each emitter-detector pair are arranged on the same side) on the subjects skull so recorded measurements are due to back-scattered (reflected) light following elliptical pathways.

fNIRS devices are powerful and have been relatively successfully used as brain-computer interfaces but they suffer from substantial problems. The main issue faced by fNIRS technology is the delay that it takes to obtain a reading from the imaging technology. It usually takes between 5 and 7 seconds to get the output as it needs to take readings of how the hemoglobin oxygenation levels are changing. This is an issue for a BCI as it would be very slow to use, especially when attempting to decipher more complicated aspects of the brain. Computer technology has a much faster response (of the order of milliseconds), so a 5 to 7 second delay would hinder the performance of the BCI and take away from its input bandwidth limit.

More recently, the event-related optical signal (EROS) technique has been suggested as a BCI. EROS is similar to fNIRS in that LEDs are used along with detectors such as photodiodes. EROS is a non-invasive brain imaging technique which uses a near-infrared light source which is positioned on the scalp of a patient and the photons that enter the brain tissue are either absorbed or scattered.

Studies have been done on neurons and show that when a neuron is active it swells and becomes more transparent allowing more light to pass through. As a result, the light goes deeper into the brain. A detector such as a photodiode monitors the photons as they are emitted from the brain tissue and through the scalp. This technique measures neurons close to the surface usually around 1 to 3 centimetres into the skull, due to the scattering and absorption of the light, which is all that is needed for certain areas of the brain such as the Broca region in the cerebral cortex, the area of the brain governing language, facial neuron control, and speech production. EROS also provides very fast results compared to other techniques such as fNIRS. Different thoughts or actions cause varying amounts of neurons to be active and this effects the time it takes for the infrared light to reach the detectors. Monitoring the differences in the time-of-flight of the photons can be fed into a computer program to provide information on neural activity. Generally, EROS provides accurate enough data to obtain useful results as it does not suffer from the issue of background noise as in the EEG method.

Typically, two types of light sources have been available for use in EROS. The first are continuous fight sources, which use slowly oscillating (<10 kilohertz) sources of light. With this approach, only the total amount of light emitted and reaching a detector can be measured. Continuous measurements cannot distinguishs between light coming from the light source and background light so the results suffer due to environmental light. The second type of light source is a frequency modulated light source. These types of light sources are used to measure the time-of-flight of photons, which has been most commonly and conveniently measured in the frequency domain as'the phase delay of a photo density wave moving between the source and the detector.

Typical EROS devices face issues of high cost and large size. Known EROS device can cost up to US$350,000 and can take up a medium-sized room due to the a large amount, sometimes hundreds, of cables connecting a headset to a large computer.

BRIEF SUMMARY OF THE INVENTION

The invention in its various aspects is defined in the independent claims below to which reference should now be made. Optional features are set forth in the dependent claims, The inventors of the present application have appreciated that fast-flashing LEDs can be used in event-related optical signal (EROS) technology for increased accuracy and resolution of results.

Arrangements are described in more detail below and take the form of an event-related optical signal device for a brain-computer interface, the device comprising at least one light source configured to flash at a rate of at least 1000 hertz at a wavelength such that the light is scattered by brain tissue.

EROS is a non-invasive brain imaging technique which uses a near-infrared light source which is positioned on the scalp of a patient and the photons that enter the brain tissue are either absorbed or scattered. when a neuron is active it swells and becomes more transparent allowing more light to pass through meaning that the light goes deeper into the brain. A detector such as a photodiode monitors the photons as they are emitted from the brain tissue and through the scalp. This technique measures neurons close to the surface usually around 1 to 3 centimetres into the skull, due to the scattering and absorption of the light, which is all that is needed for certain areas of the brain such as the Broca region in the cerebral cortex, the area of the brain governing language, facial neuron control, and

S

speech production. Monitoring the differences in the time-of-flight of the photons can be fed into a computer program to provide information on neural activity.

An example of the invention is an Event-Related Optical Signal wearable headset which uses near-infrared light emitting diodes (LEDs) which have a very fast flash time such as 1000 hertz or faster, which emit a low level of infrared light and this light is then detected by near-infrared photodiodes also on the headset. The LEDs focus on the E3roca Region (for deciphering Language processing, Facial neuron control, and Speech production) and the Visual Cortex (for processing visual information). This imaging technique is used to create a non-invasive brain-computer interface to control a computer and/or the brain-computer interface headset. The Event-Related Optical Signal headset is then connected to machine learning techniques such as an Long Short-Term Memory* (LSTM) Recurrent Neural Network (RNN) to decipher this data into meaningful information which ran be used to control the brain-computer interface'or another computer. This could be language processing or speech production from the Broca region. Being able to understand words tike 'yes' or 'no', and 'back' is very beneficial for controlling computers such as virtual and augmented reality headsets.

According to a first aspect of the present disclosure, there is provided an event-related optical signal device comprising at least one light source configured to flash light at a rate of at least 1000 hertz at a wavelength such that the light is scattered by brain tissue.

The device may comprise at least one detector configured to detect the light scattered by the brain tissue, and the at least one detector may be at least one photodiode, such as a near-infrared photodiode. The at least one detector may comprise an output, and in use, the at least one detector generates data in response to, and based on, the detected light and sends the generated data to a computer device through the output.

The computer device may comprise machine learning algorithms, such as a Long Short-Term Memory (LTSM) Recurrent Neural Network (RNN), and data acquisition software configured to analyze or classify received data or signals. The learning rate hyperparameter of the machine learning algorithm may be set to a short learning rate in order to generalize the algorithm. The computer device may be in hardware connection with the event-related optical signal device or the computer device may be one or more cloud computers. The computer device may comprise an input configured to receive the data based on the detected light, and the computer device may be configured to apply at least one machine learning algorithm to the data and determine the intent of a user. The computer device may be configured to correlate the change in received light intensity compared to the intensity of the light emitted by the at least one light source and provide feedback on such a change.

The at least one light source may flash at a wavelength such that human skin and bone are at least mostly transparent. The wavelength may be in the near-infrared range of wavelengths, such as 700 nanometres to 2500 nanometres, for example between 700 nanometres and 900 nanometres, such as 800 nanometres.

The at least one light source, which may be a light emitting diode (LED), flashes at a rate of 1000 hertz or faster, or one flash per millisecond or less. For example, the at least one light source may flash at a rate of 100 megahertz or faster. Each flash cycle may comprise an on-time and an off-time. The on-time may be 5 nanoseconds or less, with the light taking 2.5 nanoseconds or less to reach a maximum light intensity from an off-state and 2.5 nanoseconds or less to reach an off-state from the maximum light intensity. The off-state may then also be 5 nanoseconds or less. By providing such a fast flash time, the accuracy of the data obtained may be increased. The time-of-flight of individual photons emitted in each flash cycle may be measured and the differences in the travel time of the photons may be compared. Therefore, by providing a' fast flash rate, more photons can be examined. Additionally, by flashing or pulsating the light source at these speeds, the signal-to-noise ratio of the system is greatly improved.

Conveniently, the device may be a wearable headset. Even more conveniently, the device may cover only a portion of the surface of the skull or scalp. In doing this, the light emitted by the light sources may be targeted at specific areas of the brain, such as the Broca region of the frontal lobe for language and speech analysis.

The at least one light source may be arranged in a specific array on the device. For example, the at least one detector may be positioned such that light that is emitted by an at least one light source and then scattered by the brain tissue is then detected by the at least one detector. To achieve this, the at least one detector may be positioned between 4 and 8 centimetres away from the at least one light source as'this is approximately where scattered light would exit the skull.

The at least one light source may be configured to provide a small beam angle. For example, the beam angle may be 10 degrees or less, such as 5 degrees or less. This provides accurate readings as specific parts of the brain are easily targeted with a beam with low divergence. This may be achieved with a ballistic shield.

The device may comprise an acquisition computer which may comprise a processor or microprocessor, such as an Arduino (registered trade mark) microprocessor, in communication connection with the at least one detector for collecting and/or storing signals and data relating to the light detected by the detector that are received from the at least one, detector. The computer device may be in communication connection with the acquisition computer.

The device may also comprise a control computer which may comprise a processor or microprocessor. The control computer may be configured to control the at least one light source and the at least one detector. The control computer and the acquisition computer may be the same computer or may be separate computers.

The device may also comprise a headset portion in connection with the array of the at least one light source and the at least one detector. The device may comprise one or more arrays of light sources and detectors. For example, the device may comprise two arrays, one positioned on each side of the skull to target the frontal lobes.

The control computer, acquisition computer, and the array of the at least one light source and the at least one detector may all be located on a single portion of the device. For example, all of the components may be positioned on a single component board such as a printed circuit board.

According to another aspect of the present disclosure, there is provided a brain-computer interface system comprising: a device for event related optical signal, the device comprising: at least one light source configured to flash light at a wavelength such that the light is scattered by brain tissue and at a rate of at least 1000 hertz; and at least one detector configured to detect the light scattered by the brain tissue, the at least one detector comprising an output, and a computer device comprising an input, wherein, in use: the at least one detector generates data in response to, and based on, the detected light and seconds the generated data to the computer device through the output; and the computer device is configured to apply at least one machine learning algorithm to the data According to another aspect of the invention, there is provided a method for providing an event-related optical signal brain-computer interface, the method comprising: flashing light at a rate of at least 1000 hertz at a wavelength such that the light is scattered by brain tissue; detecting the light scattered by brain tissue; and based on the detected fight, outputting a response to be interpreted.

According to a further aspect of the invention, there is provided a rcomputer program configured to carry out the steps of: flashing light at a rate of at least 1000 hertz at a wavelength such that the light is scattered by brain tissue; detecting the light scattered by brain tissue; and based on the detected light, outputting a response to be interpreted.

According to a further aspect of the invention, there is provided anon-transitory computer-readable medium on which are encoded instructions for carrying out the method of flashing light at a rate of at least 1000 hertz at a wavelength such that the light is scattered by brain tissue; detecting the light scattered by brain tissue; and based on the detected light, outputting a response to be interpreted. The non transitory computer-readable medium may be, for example, solid state memory, a CD-ROM or DVE1-ROM, or a hard disk-drive.

BRIEF DESCRIPTION OF THE DRAWINGS

The invention will be described in more detail, by way of example, ith reference to the accompanying drawings in which: Figure 1 is a side view of a device embodying an aspect of the present invention; Figure 2 is a schematic diagram of the device of Figure 1 in use showing the opposite side of the device than shown in Figure 1; Figure 3 is a schematic diagram of a system including the device of Figure 1 embodying an aspect of the present invention; and Figure 4 is a graph showing the I sh rate of the device of Figure 1. Like features of the drawngs are denoted by like reference numerals. 25 DETAILED DESCRIPTION OF PREFERRED EMBODIMENT An event-related optical signal device will now be described with reference to Figures 1 to 4.

Figure 1 illustrates an event related optical signal device 10. The device comprises an array of at least one light source which in this example are a plurality of light emitting diodes (LEDs) 6 and at least one detector which in this example are a plurality of photodioides 4. The photodiodes 4 are arranged such that they detect the light scattered by the LEDs 6. Due to the refractive index of brain tissue, the majority of light will be scattered and exit the skull between 4 rand 8 centimetres away from the LED. In this example, the array ot the components provides that the LEDs 6 and the photodiodes 4 are between 4 and 8 centimetres apart from one another. The LEDs 6 and photodiodes 4 are arranged on a component portion 2 of the device.

The component portion 2 also comprises an acquisition computer 14 and a control computer 16. The control computer 16 is in communication connection with the LEDs 6 and the photodiodes 4 and the acquisition computer 14 is in communication connection with the photodiodes 4 via an output from the photodiodes 4. In this example, the acquisition computer 14 and the control computer 16 form one computer and share a microprocessor 18. The microprocessor 18 comprises an output to a computer device 20. In this example, the computer device 20 is a cloud computer system. The cloud system is a plurality of servers, and, in particular, servers remote from the device 10 located on a different site. The portion 2 is in connection with a headset portion 8. The portion 2 is positioned to enable examination of a particular region of the brain, which in this example is the Broca region. The headset portion 8 which is in connection with the portion 2 enables the portion 2 to be adjusted to the desired position on the skull 12.

The device illustrated in Figure 1 comprises two component portions 2, the second component portion comprising the same components as the component portion 2 illustrated in Figure 1 but is not visible in the side view of Figure 1.

Figure 2 illustrates the device 10 of Figure 1 in use on a human head 12, and Figure 3 illustrates the device of Figure 1 in connection with the computer device 20.

Figure 4 illustrates the flash cycle of the at least one light source, the LEDs 6. The y axis of the graph provides the intensity of the LED 6, and the x axis provides the time taken.

In use, the wearable device 10 is place on the human's head 12. The component portions 2 are positioned such as to target a specific part of the brain. In this example, the Broca region is targeted.

The control computer 16 controls the LEDs 6 to flash. That is to cycle between a state where right is emitted to a state where light is not emitted. In this example, the LEDs 6 are controlled to fiash at a rate of 1000 hertz or faster, specifically, at a rate of 100 megahertz or one flash cycle per 10 nanoseconds. This flash cycle 21 is illustrated at Figure 4. The flash cycle is the time is takes for the intensity of the LED 6 to go from being at a minimum (off-state) to a maximum (on-state), back to a minimum (off-state), and then for the LED to remain off for the same time as it took for the LED to turn on and off again. The LED remains off for this time to allow the photons sent in the flash to be detected by the photodiode and distinguish the photons sent in the flash from photons sent in another flash.

Generally, for this application, it is desired that the flash cycle be as fast as the LED allows.

In this example, the flash cycle comprises a 2.5 nanosecond period, from point 22 to point 24 of Figure 4, for the LED 6 to reach maximum intensity from an off-state, and a 2.5 nanosecond period, from point 24 to point 26 of Figure 4, for the LED 6 to reach an off-state from maximum intensity. The LED 6 is then controlled to be off for 5 nanoseconds, from point 26 to point 28 of Figure 4, until the cycle is repeated. The wavelength of light emitted is such that the light will be scattered by brain tissue. Therefore, the light is emitted at a wavelength at which human skin and bone is mostly transparent. In this example, the light is emitted at a wavelength of 800 nanometres.

The light emitted by the LEDs 6 is configured to have a beam angle of 5 degrees by using a ballistic shield. Once the light has been emitted from the LEDs 6, the light travels through the skull and reaches a targeted region of the brain. Some of the emitted light will be absorbed, and some will be scattered by the brain tissue, Light scattered by the brain tissue will travel out of the brain and towards the surface of the head 12. Scattered light that exits the head 12 is detected by the photodiodes 4. The photodiodes 4 therefore detect the intensity of light scattered from the initially emitted beam and generate data in response to, and based on, the detected light.

The acquisition computer 14, in communication with the photodiodes 4, acquires the received data sent via the output of the photodiodes 4 to the acquisition computer 14. This cycle is repeated corresponding to the flash cycle of one flash per 10 nanoseconds.

Acquired data can be stored by the acquisition unit, or can be sent to a computer device 20, for processing.

In this example, the acquired data is sent to the computer device 20 to be processed via the output of the microprocessor. The computer device 20 then applies at least one machine learning algorithm to the received data. In this example, the computer device 20 applies a Long Short-Term Memory (LSTM) Recurrent Neural Network (RNN) to decipher the data into meaningful information which can be used to control the brain-computer interface or another computer.

Embodiments of the present invention have been described. It will be appreciated that variations and modifications may be made to the described embodiments within the scope of the present invention.

Claims

CLAIMS1. An event-related optical signal device for a brain-computer interface, the device comprising at least one light source configured to flash light at a rate of at least 1000 hertz at a wavelength such that the light is scattered by brain tissue.
2. An event-related optical signal device according to claim 1, wherein the device further comprises: at least one detector configured to detect the light scattered by the brain tissue.
3. An event-related optical signal device according to claim 2, wherein: the at least one detector is configured to provide a response based on the detected light to a computer device configured to receive and interpret the response.
4. An event-related optical signal device according to claim 3, wherein the device is configured to interpret the data by applying at least one taught machine learning algorithm to the data,
5. An event-related optical signal device according to any preceding claim, wherein the wavelength is such that human skin arid bone are at least mostly transparent at the wavelength.
6. An event-related optical signal device according to any preceding claim, wherein the at least ore light source flashes at a frequency of 100 megahertz or faster.
7. An event-related optical signal device according to any preceding claim, wherein the wavelength of the light is between 700 nanometres and 2500 nanometres.3.
An event-related optical signal' device according to any preceding claim, wherein the wavelength of the light is between 700 nanometres and 900 nanometres.
9. An event-related optical signal device according to any preceding claim, wherein the at least one detector is at least one photodiode.
10. An event-related optical signal device according to any preceding claim, wherein the at least one light source is at least one light emitting diode.
An event-related optical signal device according to any preceding claim, wherein the at least one light source is in communication connection with a control computer and the at least one light source is controlled by the control computer.
12. An event-related optical signal device according to any of claim 11, wherein: the at least one detector is in communication connection with an acquisition computer; and the acquisition computer is configured to receive le response provided by the at least one detector.
13. An event-related optical signal device according to claim 12, wherein the control computer, the acquisition computer, the at least one light source, and the at least one detector, are arranged on the same circuit board.
14. An event-related optical signal device according to any of claims 2 to 13, wherein the at least one light source is arranged between,4 and 8 centimetres away from the at least one detector.
15. An event-related optical signal device according to any preceding claim, wherein the at least one light source has a maximum flash rate of 1x1012 hertz.
16. An event-related optical signal device according to any preceding claim, wherein the device is a wearable headset.
17. An event-related optical signal device according to any preceding claim, wherein the light is scattered by neurons in the brain tissue.
18. A brain-computer interface system comprising: a device for event related optical signal, the device comprising: at least one light source configured to flash light at a rate of at least 1000 hertz at a wavelength such that the light is scattered by brain tissue; and at least one detector configured to detect the light scattered by the brain tissue, and provide a response based on the detected light, and a computer device configured to receive and interpret the response,
19. A brain-computer interface system according to claim 18, wherein the computer device interprets the response by applying at least one taught machine learning algorithm to the response.
20. A method for providing an event-related optical signal brain-computer interface, the method comprising: flashing light at a rate of at least 1000 hertz at a wavelength such that the light is scattered by brain tissue; detecting the light scattered by brain tissue; and based on the detected light, outputting a response to be interpreted.
21. A computer program configured to carry out the steps of the method of claim 20.
22. A non-transitory computer-readable medium on which are encoded instructions for carrying out the method of claim 20.