WO2017205476A1

WO2017205476A1 - Non-invasive eye-tracking control of neuromuscular stimulation system

Info

Publication number: WO2017205476A1
Application number: PCT/US2017/034192
Authority: WO
Inventors: David A. FRIEDENBERG
Original assignee: Battelle Memorial Institute
Priority date: 2016-05-26
Filing date: 2017-05-24
Publication date: 2017-11-30

Abstract

A non-invasive control system for neuromuscular stimulation includes an eye-tracking device, an electrical stimulation device, and software that interprets the eye movements of the user to determine an intended movement and sends electrical signal(s) to the stimulation device to achieve the intended movement. For example, the stimulation device may be a sleeve with electrodes worn on a paralyzed limb, with the intended movement being the movement of the limb.

Description

NON-INVASIVE EYE-TRACKING CONTROL OF NEUROMUSCULAR STIMULATION

SYSTEM

CROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application claims priority to U.S. Provisional Patent Application Serial No. 62/342, 140, filed on May 26, 2016, which is fully incorporated by reference herein.

BACKGROUND

[0002] The present disclosure relates to non-invasive methods, systems, and devices for controlling targets, and in particular paralyzed limbs, using a control interface driven by the user's eye movements to control electrical stimulation provided by a neuromuscular electrical stimulation sleeve. In one non-limiting embodiment, the disclosure relates to reanimating a limb, such as a hand, of spinal cord injury patients via non-invasive eye-tracking control of a neuromuscular stimulation system. Various devices and associated components and methods and processes of using the same are further included in this disclosure. It is to be appreciated that these methods, systems, and devices may also be amenable to other like applications.

[0003] Many millions of people suffer from some motor impairment. For example, it is estimated that worldwide, 10 million people are left disabled following a stroke each year. People also suffer from brain injury, failed back surgery or spinal cord injury. These injuries can result in motor impairment by damaging the link between the brain and the muscles of the body which are used for movement. For example, neurons in the brain may die, or the nerves between the brain and the muscle are severed. These disrupt the paths by which electrical signals travel from the brain to neuromuscular groups to effectuate coordinated muscle contraction patterns.

[0004] Neuromuscular stimulation devices have been used to deliver stimulation to restore movement to parts of the body not under volitional control. For example, subcutaneous implantable neurostimulation cuffs are wrapped around a target nerve and generally include one or more electrodes arranged to stimulate the nerve. Transcutaneous neurostimulation cuffs behave similarly to implantable cuffs, however there are important differences. Because the electrodes are placed on the surface of the skin, rather than below it, stimulation often can better target skeletal muscle tissue or muscle groups, rather than peripheral nerves located deeper under the skin. Muscular stimulation may be preferable to stimulating major peripheral nerves, e.g. ulnar, median, radial nerves, as stimulating these nerves may cause a patient to feel a tingling sensation and it is more difficult to effect the desired movement. By increasing the number and layout of electrodes in a neuromuscular cuff, current generation neuromuscular stimulation cuffs have been able to selectively stimulate individual muscles or muscle groups and achieve finer movements such as individual finger flexing and extension. Flexible-like transcutaneous cuffs have also been developed which fit around a human appendage such as a forearm to control the wrist or fingers.

[0005] Neurostimulation devices do not, by themselves, resolve the problem of motor impairment due to neural damage, because such a device by itself does not respond to volitional control. A system has been previously designed that includes an electrode array that is surgically implanted into the brain. Along with control algorithms, this system has improved motor impairment in a user with a spinal cord injury, conferring on him the ability to grasp, manipulate and release objects and perform functional tasks relevant to daily living. However, the complex surgery to implant the invasive electrode array carries risks and may not be viable for many patients from both a medical and cost perspective. It would be desirable to provide a non-invasive control system that can control electrical stimulation to permit the user to regain volitional control and perform the same type of functional movements as are possible with the invasive system.

BRIEF DESCRIPTION

[0006] The present disclosure relates to methods and systems for non-invasive control and movement of a target, such as a limb or prosthetic, by tracking the eye movement of a user to determine the desired movement and delivering one or more electrical signals to obtain the desired movement. These systems include an eye tracking device, a software application that interprets the user's intended movement, and a stimulation system to cause the intended movement to occur.

[0007] Disclosed herein are various embodiments of methods of moving a target, comprising: tracking eye movement of a user; based on the eye movement, determining a desired movement of the target; and delivering an electrical signal to the target to obtain the desired movement of the target.

[0008] The eye movement can be tracked by a system comprising at least one camera directed towards the eye. Alternatively, the system could be a head-mounted wearable system.

[0009] The user may look at a particular icon on a graphical interface to indicate the desired movement. Alternatively, the user might look at the target to indicate the desired movement.

[0010] The target of the intended movement can be a body limb or a prosthetic limb, a wheelchair, a cursor on a computer, an exoskeleton, a remote control device, or an external robotic arm. In particular embodiments, the target is a body limb or a prosthetic limb, and the electrical signal is delivered using a neuromuscular electronic stimulation system.

[0011] Also disclosed are systems for moving a target, comprising: an eye tracking device; and software that receives eye movement from the eye tracking device as input, determines a desired movement of the target, and outputs an electrical signal to a stimulation system to obtain the desired movement. The stimulation system can be a neuromuscular electrical stimulation sleeve.

[0012] These and other non-limiting aspects of the disclosure are more particularly discussed below.

BRIEF DESCRIPTION OF THE DRAWINGS

[0013] The following is a brief description of the drawings, which are presented for the purposes of illustrating the exemplary embodiments disclosed herein and not for the purposes of limiting the same.

[0014] FIG. 1 is a schematic of the overall non-invasive control and movement system.

[0015] FIG. 2 is a plan view of one embodiment of a neural sleeve that can be used for practicing the methods of the present disclosure.

[0016] FIG. 3 is an exemplary photograph showing two neural sleeve devices according to the embodiment of FIG. 2 which are wrapped around a patient's arm region in preparation for neuromuscular stimulation. [0017] FIG. 4 is diagram of another exemplary embodiment of a neural sleeve. In this embodiment, conductive pathways extend from two different connectors. The fingers extend in the same direction, and taper towards a center axis.

DETAILED DESCRIPTION

[0018] A more complete understanding of the methods and apparatuses disclosed herein can be obtained by reference to the accompanying drawings. These figures are merely schematic representations based on convenience and the ease of demonstrating the existing art and/or the present development, and are, therefore, not intended to indicate relative size and dimensions of the assemblies or components thereof.

[0019] Although specific terms are used in the following description for the sake of clarity, these terms are intended to refer only to the particular structure of the embodiments selected for illustration in the drawings, and are not intended to define or limit the scope of the disclosure. In the drawings and the following description below, it is to be understood that like numeric designations refer to components of like function.

[0020] The present disclosure relates to methods for providing an artificial neuromuscular stimulation system with volitional control, e.g. by determine a desired movement of a target by the user, then transmitting the desired movement to the target to perform the desired movement. For example, these methods are useful for patients who may have suffered nerve injury to bypass that disruption and "bridge" that gap to send electrical signals to a body limb, for example an arm or leg. Desirably, it would be as if the disruption did not exist; the patient would simply think of the desired movement of the body limb, and that movement would occur.

[0021] In other embodiments, rather than being a body limb, electrical signals may be delivered to a target that is non-human (e.g. an electronic device). This allows for control of the non-human target by use of the electrical signals. Examples of non-human targets that may be controlled in this way include: a prosthetic limb, a wheelchair, a cursor on a computer, an exoskeleton, a remote control device, and an external robotic arm.

[0022] In particular, the system includes an eye tracking device that allows the user to control the target using only his eyes. A software application will interpret the user's intent, i.e. the intended movement, and send out electrical signal(s) to produce the desired movement in the target. This type of system is non-invasive, or in other words the user does not have to be surgically operated upon, and no device (like an electrode array) needs to be implanted in the body as in previous iterations of this system.

[0023] FIG. 1 is a schematic of the overall non-invasive control and movement system. The overall non-invasive control system 100 is a software/hardware solution that combines an eye tracking device 110, a neuromuscular electrical stimulation system 120, and a custom designed software application 130 to interpret the eye movements into intended movement of a limb and send electrical signals that produce the intended movement. Desirably, this allows a user with a spinal cord injury (SCI) to volitionally execute a variety of grips, as well as wrist and individual finger movements using their own hand.

[0024] The software application 130 will receive input from the eye-tracking device 110 that provides information on the intended movement that the user desires. The software application 130 will output control signals to the stimulation system 120, and can also receive feedback from the stimulation system 120 on the movement attained so far, which can be used to adjust subsequent control signals. No invasive surgery is needed, and so for example brain signals from an implanted electrode array are not used in the present disclosure.

[0025] The first component of the overall non-invasive control system is a stimulation system that can deliver electrical stimulation. In particular embodiments, the stimulation system is a neuromuscular electrical stimulation system in the form of a sleeve that can be wrapped / attached to the limb of a user, such as the arm or leg. Such systems have been demonstrated to restore functionally significant wrist and hand movements for SCI patients when paired with an intuitive control system. Very broadly, such "neural sleeves" contain a set of electrodes that are used to provide stimulation to the body part when the patient's focus level and intent to move are sufficient. The term "sleeve" is used to refer to a structure that surrounds a body part, for example, an arm, leg, or torso. The neural sleeve can take the form of a shirt, or pants, if desired. The neural sleeve also contains sensors for monitoring movement of the body part (position, orientation, acceleration, etc.), which can be used to track the movement of the body part. [0026] The second component of the overall non-invasive control system is a noninvasive eye-tracking input system. Several different eye-tracking devices are commercially available that may be a practical vehicle for an SCI patient to control a computer. The eye tracking device may include a small camera or multiple cameras attached to the headset positioned on the patient. The camera(s) may also be positioned in any other manner so as to as to be able to view the patients eye's (e.g. the camera(s) may be positioned on a desk, positioned on top of a laptop screen, incorporated into a pair of glasses or so forth). In addition to camera(s) viewing a patient's eyes, camera(s) may also be positioned so that they are pointing outwardly on a pair of glasses that a patient is wearing. This allows a determination to be made of what the patient is looking at, and this information can be used in conjunction with the observations of the patient's eye movements.

[0027] The third component of the overall non-invasive control system is a software application that links the user's intent to actual movement. The software application will translate the user's input (i.e. eye movement) into commands for the stimulation that leads to movement of the desired target, e.g. the user's hand. Desirably, the software application will have a clean and intuitive interface that will feel natural to the end user.

[0028] The electrical stimulation pattern that is to be sent to the stimulation system by the software application (third component) is determined by an encoding algorithm that generates the appropriate spatiotemporal patterns to evoke appropriate muscle contractions. The present disclosure permits high definition stimulation. In high definition stimulation, the simulation pattern to the electrodes is "continuously" updated. For example, the stimulation pattern to the electrodes is updated once every 0.1 seconds (e.g. 10 Hz). However, shorter and longer update times are also contemplated; in fact, speeds up to 50 Hz are contemplated. As discussed above, the simulation pattern is provided based on the desired movement. In this regard, different patients need a different stimulation pattern to obtain the same movement of the target. Advantageously, this allows for delivery of a more effective stimulation pattern based on the individual characteristics of a user. The stimulation pattern is then sent to the stimulation system, which can for example use electrodes to stimulate the muscles needed to achieve the desired movement. [0029] In high definition stimulation, multiple signal patterns may be interleaved (e.g. by multiplexing) if more than one motion is desired (e.g. a compound motion). For example, the stimulation pattern needed to lift the arm may be directed to different muscles than those for rotating the wrist. Interleaving permits multiple stimulation patterns to be combined into a single stimulation signal sent to the stimulation system (e.g. neuromuscular sleeve), so that multiple movements can occur at the same time. Again, this permits the body limb to move more naturally. In addition, advantageously, interleaving prevents electric field patterns created by one stimulation pattern from interfering with electric fields created by another stimulation pattern. This increases the number of complex motions that the system is capable of. However, there is a practical limit to the number of stimulation patterns that may be interleaved due to the fact that when the pulse rate for a single simulation pattern becomes too low (e.g. less than 10 pulses per second), muscle twitches will start to become noticeable and movement smoothness becomes undesirable. Still, interleaving is very effective, and allows for multiple movements to be performed simultaneously (e.g. in a compound movement). This could not be achieved with only a single simulation pattern.

[0030] In addition, motions may be sequenced. One example of this in a natural system is a central pattern generator, which produces rhythmic patterned outputs without sensory feedback. The software / systems of the present disclosure can mimic central pattern generators by producing a repeatable sequence of events, for example to return a targeted body part to an initial state. Another example of sequenced motions is a functional series of motions. Examples of functional series of motions include: teeth brushing, scratching, stirring a drink, flexing a thumb, cylindrical grasping, pinching, etc. These motions allow for manipulation of real-world objects of various sizes.

[0031] As a result of the stimulation, the target / body part moves as imagined by the user, which can serve as feedback to the user. The electrical stimulation can be provided in the form of current-controlled, monophasic pulses of adjustable pulse rate, pulse width, and pulse amplitude which can be independently adjusted for each channel. This cycle repeats continuously. The stimulation signal / pattern sent to the electrodes can be changed continuously through each cycle if needed, or can be maintained, in order to complete the imagined movement. The software can monitor either or both the eye movement and the motion of the target as detected by the sensors. The stimulation signal sent to the target may be actively changed over time due to changes in motion / location, for example due to the shift in the electrode position relative to their targeted muscle groups as a body limb moves. Desirably, the decoder is also robust to context changes (such as arm position and speed).

[0032] It is contemplated that the eye-tracking device and the software application can work together to determine the desired movement. For example, various icons could be displayed on a graphical user interface (GUI) that indicate certain motions, e.g. flex the wrist, extend the wrist, lift the arm perpendicularly, etc. The user could stare at the icon corresponding to the desired motion, with the time spent staring at the icon being used as an indicator of the degree of motion (e.g. how much to flex the wrist), or a scale indicating the degree of motion could be located next to the icon. Alternatively, the user could stare at the body part whose movement is desired, and the GUI could then present movement options.

[0033] FIG. 2 is an illustration of one potential neural sleeve that can be used as a stimulation system in the methods of the present disclosure. The sleeve 700 as illustrated has an insulating substrate 722 that is shaped into four flexible conductive pathways 710, each pathway being formed from a finger 724 and a header 728. The flexible conductive pathways 710 extend in the same direction from the connector 730, which acts as a connector for one end of the pathways. In other words, the ends of the pathways distal from the connector are all located in the same direction relative to the connector, or put another way the connector 730 is at one end of the device. It is noted that the pathways 710 are shown here as extending at a 90-degree angle relative to the connector 730. The pathways can be attached to each other, for example by five webbings 725 which run between adjacent fingers 728.

[0034] The electrodes 740 are located or housed on the fingers 724, and are formed as a layer upon the substrate 722. The electrodes 740 run along the four fingers 724 and are electrically connected to the connector 730. The electrodes 740 are approximately 12 mm in diameter and spaced 15 mm apart. A conductive medium, e.g. hydrogel discs, can be laid upon the electrodes to facilitate contact with the user's skin. [0035] The connector 730 is used for interfacing with the software application / computer 120 of FIG. 1. If desired, an optional fork 726 can be located at the end of the pathways opposite the connector 730. The fork connects all of the fingers, and can be provided for structural support for design and mounting. Headers 728 extend between the connector 730 and the fingers 724. These headers are thinner than the fingers, and connect the fingers 724 to the connector 730. The headers are also part of the overall flexible conductive pathway, though they are not always required. Though not illustrated, webbings can also be provided between adjacent headers as well if desired. Again, the fork 726 is optional, though the connector 730 is required.

[0036] FIG. 3 shows two neuromuscular cuff devices 1010 of FIG. 2 being wrapped circumferentially around a patient's arm region 1020 in preparation for neuromuscular stimulation. The two cuff devices 1010 together provide 160 separate electrodes for stimulating finger or wrist movements. The fingers 1024 permit the neuromuscular cuff to fit around the arm region 1020 at points of varying circumference. Hydrogel discs 1016 (not shown) keep both cuffs 1010 adhered to the arm.

[0037] In another exemplary embodiment, the flexible conductive pathways on a neural sleeve 2110 do not need to be straight for their entire length. Referring now to FIG. 4, flexible conductive pathways 2124 extend from first connector 2130, which has a rectangular shape in this illustration. The flexible conductive pathways 2124 in this embodiment "change" directions as they extend from connector 2130. For example, an upper flexible conductive pathway 2124a first extends upwards from the connector 2130, then changes direction so that its electrodes 2140 are to the right of the connector 2130. A center flexible conductive pathway 2124b extends from the right-hand side of the connector 2130 off to the right of the connector. A lower flexible conductive pathway 2124c first extends downwards from the connector 2130, then changes direction so that its electrodes 2140 are also to the right of the connector 2130. Notably, none of the electrodes 2140 are present to the left of the connector 2130.

[0038] This embodiment of a neural sleeve 2110 also contains more than one connector. As illustrated here, the neural sleeve 2110 has a first connector 2130 and a second connector 2131. Flexible conductive pathways extend in the same direction (here, to the right) of both connectors. Webbings 2135 connect flexible conductive pathways extending from each connector 2130, 2131. There may be any number of webbings 2135, and the webbings 2135 may connect the flexible conductive pathways at any portion of their length. Here, the webbings 2135 are present along a a non-electrode-containing portion 2150 of the flexible conductive pathways (i.e. the header portion). Though not depicted, it is specifically contemplated that the flexible conductive pathways of one connector 2130 may be of a different length from the flexible conductive pathways of the other connector 2131.

[0039] The electrodes 2140 may be evenly spaced apart along the length of the flexible conductive pathways 2124, or their spacing may vary, for example becoming shorter or longer, as the distance from the connector 2130 increases. For example, muscle segments get smaller closer to the wrist, so the electrodes need to be closer together as well. However, the electrodes do not need to be present along the entire length of the flexible conductive pathways. As seen here, the flexible conductive pathways 2124 may include a non-electrode-containing portion 2150 extending from the connector. The flexible conductive pathway may also include a non-scalloped electrode- containing portion 2160, and a scalloped electrode-containing portion 2170 at the distal end of the flexible conductive pathway (i.e. distal from the connector). It should be noted that none of the flexible conductive pathways overlap with each other.

[0040] The electrode-containing portions 2160, 2170 of the flexible conductive pathways have a different shape from each other. One reason for this difference in shape is because, as seen here, the distal ends of the flexible conductive pathways 2124 extend inwardly towards a center axis 2105 of the neural sleeve 2110. Put another way, the flexible conductive pathways 2124 taper inwards towards the center axis 2105. The scalloped portions 2170 of adjacent flexible conductive pathways permit them to fit into a smaller area while still providing a suitable number of electrodes (note the electrodes do not change in size). However, the flexible conductive pathways 2124 all still extend in the same direction away from the connector 2130, i.e. to the right in this figure. Put another way, the flexible conductive pathways comprise a first portion which is transverse to the center axis 2105, and a second portion which is parallel to the center axis. These portions are particularly seen in the flexible conductive pathway 2124a, which first extends upwards (i.e. transversely to the center axis), then extends parallel to the center axis. [0041] This particular embodiment is intended to be used on a patient's arm with the two connectors 2130, 2131 located near the shoulder, and the scalloped portions 2170 near the wrist and hand.

[0042] The overall non-invasive control system should be able to execute various grips that are important for tasks of daily living, for example picking up objects, pouring the contents of one container into another, swiping a credit card, and playing a video game. These would be significant improvements in motor-impaired individuals. Additional improvements could include controlling two separate sleeves, one on each arm, using the eye-tracking device and software.

[0043] It will further be appreciated that the disclosed techniques may be embodied as a non-transitory storage medium storing instructions readable and executable by a computer, (microprocessor or microcontroller of an) embedded system, or various combinations thereof. The non-transitory storage medium may, for example, comprise a hard disk drive, RAID or the like of a computer; an electronic, magnetic, optical, or other memory of an embedded system, or so forth.

[0044] The present disclosure has been described with reference to exemplary embodiments. Obviously, modifications and alterations will occur to others upon reading and understanding the preceding detailed description. It is intended that the present disclosure be construed as including all such modifications and alterations insofar as they come within the scope of the appended claims or the equivalents thereof.

Claims

CLAIMS:

1 . A method of moving a target, comprising:

tracking eye movement of a user;

based on the eye movement, determining a desired movement of the target; and

delivering an electrical signal to the target to obtain the desired movement of the target.

2. The method of claim 1 , wherein the eye movement is tracked by a system comprising at least one camera directed towards the eye.

3. The method of claim 1 , wherein the system is a head-mounted wearable system.

4. The method of claim 1 , wherein the user looks at a particular icon on a graphical interface to indicate the desired movement.

5. The method of claim 1 , wherein the user looks at the target to indicate the desired movement.

6. The method of claim 1 , wherein the target is a body limb or a prosthetic limb, a wheelchair, a cursor on a computer, an exoskeleton, a remote control device, or an external robotic arm.

7. The method of claim 6, wherein the target is a body limb or a prosthetic limb, and the electrical signal is delivered using a neuromuscular electronic stimulation system.

8. A system for moving a target, comprising:

an eye tracking device; and

software that receives eye movement from the eye tracking device as input, determines a desired movement of the target, and outputs an electrical signal to a stimulation system to obtain the desired movement.

9. The system of claim 8, wherein the stimulation system is a neuromuscular electrical stimulation sleeve.