WO2018218086A1 - Système de détection de mouvements faciaux sur le visage d'un utilisateur - Google Patents

Système de détection de mouvements faciaux sur le visage d'un utilisateur Download PDF

Info

Publication number
WO2018218086A1
WO2018218086A1 PCT/US2018/034490 US2018034490W WO2018218086A1 WO 2018218086 A1 WO2018218086 A1 WO 2018218086A1 US 2018034490 W US2018034490 W US 2018034490W WO 2018218086 A1 WO2018218086 A1 WO 2018218086A1
Authority
WO
WIPO (PCT)
Prior art keywords
user
headset
face
electrode
elastic member
Prior art date
Application number
PCT/US2018/034490
Other languages
English (en)
Inventor
Kevin Lee
Ivan Roberto REYES
John P. Pella
Quentin Dewolf
Original Assignee
Silicon Algebra, Inc.
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Silicon Algebra, Inc. filed Critical Silicon Algebra, Inc.
Publication of WO2018218086A1 publication Critical patent/WO2018218086A1/fr

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F1/00—Details not covered by groups G06F3/00 - G06F13/00 and G06F21/00
    • G06F1/16—Constructional details or arrangements
    • G06F1/1613—Constructional details or arrangements for portable computers
    • G06F1/163—Wearable computers, e.g. on a belt
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/103—Detecting, measuring or recording devices for testing the shape, pattern, colour, size or movement of the body or parts thereof, for diagnostic purposes
    • A61B5/11—Measuring movement of the entire body or parts thereof, e.g. head or hand tremor, mobility of a limb
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/24—Detecting, measuring or recording bioelectric or biomagnetic signals of the body or parts thereof
    • A61B5/25—Bioelectric electrodes therefor
    • A61B5/279—Bioelectric electrodes therefor specially adapted for particular uses
    • A61B5/296—Bioelectric electrodes therefor specially adapted for particular uses for electromyography [EMG]
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/24—Detecting, measuring or recording bioelectric or biomagnetic signals of the body or parts thereof
    • A61B5/316—Modalities, i.e. specific diagnostic methods
    • A61B5/389—Electromyography [EMG]
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/68—Arrangements of detecting, measuring or recording means, e.g. sensors, in relation to patient
    • A61B5/6801—Arrangements of detecting, measuring or recording means, e.g. sensors, in relation to patient specially adapted to be attached to or worn on the body surface
    • A61B5/6802—Sensor mounted on worn items
    • A61B5/6803—Head-worn items, e.g. helmets, masks, headphones or goggles
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/72—Signal processing specially adapted for physiological signals or for diagnostic purposes
    • A61B5/7203—Signal processing specially adapted for physiological signals or for diagnostic purposes for noise prevention, reduction or removal
    • A61B5/7207—Signal processing specially adapted for physiological signals or for diagnostic purposes for noise prevention, reduction or removal of noise induced by motion artifacts
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
    • G06F3/015—Input arrangements based on nervous system activity detection, e.g. brain waves [EEG] detection, electromyograms [EMG] detection, electrodermal response detection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2203/00—Indexing scheme relating to G06F3/00 - G06F3/048
    • G06F2203/01—Indexing scheme relating to G06F3/01
    • G06F2203/011—Emotion or mood input determined on the basis of sensed human body parameters such as pulse, heart rate or beat, temperature of skin, facial expressions, iris, voice pitch, brain activity patterns

Definitions

  • This invention relates generally to the field of biometric sensors and more specifically to a new and useful system for detecting facial movements over a face of a user in the field of biometric sensors.
  • FIGURE 1 is a schematic representation of a system
  • FIGURE 2 is a schematic representation of one variation of the system
  • FIGURES 3A and 3B are schematic representations of one variation of the system
  • FIGURE 4 is a flowchart representation of one variation of the system
  • FIGURE 5 is a flowchart representation of one variation of the system
  • FIGURE 6 is a schematic representation of one variation of the system
  • FIGURE 7 is a flowchart representation of one variation of a first method.
  • FIGURE 8 is a flowchart representation of one variation of a second method.
  • a system for detecting facial movements over a face of a user obscured by a headset includes: a sensor assembly that includes: an elastic member 110: configured to couple to a headset and contact a face of a user, proximal and offset from ocular regions of the face of the user, when the headset is worn by the user, and wherein the elastic member is composed of a compressible material configured to conform to the face of the user when the headset is worn by the user.
  • the sensor assembly also includes a substrate arranged across an exterior surface of the elastic member that defines a set of discrete electrical channels.
  • the sensor assembly further includes a set of electrode tabs distributed along the elastic member, wherein each electrode tab is configured to proximally contact a muscular region on the face of the user when the headset is worn by the user and wherein each electrode tab includes a sense electrode that defines an electrical contact surface facing outwardly from an interior surface of the elastic member opposite the substrate. Each electrode tab is further configured to contact skin on the face of the user when the headset is worn by the user; and further includes an enclosed trace extending from the sense electrode at the interior face of the elastic member to a discrete electrical channel, in the set of discrete electrical channels, at the exterior face of the elastic member.
  • the system also includes a controller configured to sample a set of sense signals from the set of electrode tabs via the set of discrete electrical channels and to interpret changes in the set of sense signals over time as expressions on the face of the user when the headset is worn by the user.
  • the elastic member is coupled to the headset, which is a pair of optical glasses; and the elastic member is configured to contact the face of the user, proximal to supraorbital regions of the face of the user and offset from ocular regions of the face of the user, when the headset is worn by the user.
  • the variation of the system further includes a second elastic member configured to contact a nasal bridge region of the face of the user and offset from ocular regions of the face of the user, when the headset is worn by the user.
  • the variation of the system also includes a second set of the electrode tabs distributed along the second elastic member.
  • the controller is configured to sample a second set of sense signals from the second set of electrode tabs and to interpret changes in the first set of sense signals and the second set of sense signals over time as expressions on the face of the user when the headset is worn by the user.
  • a first manufacturing method S100 includes applying a layer of conductive ink to an exterior surface of a flexible substrate to form: a set of sense electrodes, a common junction, and a set of electrical traces, wherein each electrical trace connects each sense electrode in the set of sense electrodes to a corresponding electrical channel of the common junction in Block S110.
  • the first manufacturing method also includes applying a non-oxidative conductive layer over the set of sense electrodes and applying a nonconductive cover layer over the set of electrical traces 128 in Block S120.
  • the first manufacturing method further includes selectively cutting the flexible substrate around the applied layer of conductive ink to define: a substrate ring comprising the set of electrical traces 128 and the common junction, and a set of electrode tabs extending radially from the substrate ring, each electrode tab in the set of electrode tabs comprising a sense electrode in the set of sense electrodes in Block S130.
  • the first manufacturing method also includes assembling a sensor assembly by adhering: an interior surface of the substrate ring to an exterior surface of an elastic member configured to conform to a face of a user offset from the ocular regions of the face of the user, and an interior surface of each in the set of electrode tabs to an interior surface of the elastic member in Block S140.
  • the first manufacturing method also includes applying an adherent to an exterior surface of the sensor assembly, wherein the adherent is configured to transiently couple the sensor assembly to a view window of a virtual reality headset in Block S150.
  • a second manufacturing method S200 includes applying a layer of conductive ink to an exterior surface of a flexible substrate to form: a common junction, a set of electrical pads, and a set of electrical channels, wherein each electrical channel connects each electrical pad in the set of electrical pads to a corresponding electrical channel of the common junction in Block S210.
  • the second manufacturing method also includes selectively cutting the flexible substrate around the applied layer of conductive ink to define a substrate ring comprising the set of electrical pads, the set of electrical channels, and the common junction in Block S220.
  • the second manufacturing method further includes applying a layer of conductive ink to a flexible polymer sheet to form a set of electrode tabs, each electrode tab including: a junction pad, a sense electrode, and, an electrical trace connecting the junction pad and the sense electrode in Block S230.
  • the second manufacturing method also includes, for each electrode tab in the set of electrode tabs, separating the electrode tab from the flexible polymer sheet in Block S240.
  • the second manufacturing method also includes assembling an electrode-substrate assembly by adhering, for each electrode tab in the set of electrode tabs, the junction pad of the electrode tab to an electrical pad in the set of electrical pads on the substrate ring S250.
  • the second manufacturing method also includes assembling a sensor assembly by adhering: an interior surface of the substrate ring of the electrode-substrate assembly to an exterior surface of an elastic member configured to conform to a face of a user offset from the ocular regions of the face of the user; and an interior surface of each electrode tab in the set of electrode tabs to an interior surface of the elastic member in Block S260.
  • the second manufacturing method also includes applying an adherent to an exterior surface of the sensor assembly, the adherent configured to transiently couple the sensor assembly to a view window of a virtual reality headset in Block S270.
  • the system 100 includes: an elastic member 110 (or “gasket”), a substrate 130, and a set of sense electrodes and/or reference electrodes 124 that together form a passive sensor assembly 170; and a controller 140 (and signal processing circuitry) arranged remotely from the sensor assembly 170 that can be configured to interpret signals read from the sensor assembly 170 as facial expressions of a face in contact with the sensor assembly 170.
  • the elastic member or members 110 or define a soft, flexible (e.g., foam) gasket or series of pads configured to transiently or permanently install around a view window of a headset (e.g., around a display of a virtual reality headset), to be pressed against the user's face by the (rigid) view window of the headset, and to conform to the user's face when the (rigid) headset is worn over the user's eyes and face, thereby improving the user's comfort when wearing the headset and sealing the junction between the headset and the user's face from light ingress and egress.
  • the gasket can be implemented as a discontinuous set of pads attached to a pair of ocular glasses.
  • the weight of the optical glasses can hold the gaskets against the skin of a user at the bridge of the user's nose and at the user's temple above the user's eyes.
  • the set of passive electrodes 122 are distributed across the interior surface of the elastic member or members 110 and face opposite the headset such that these electrodes 122 contact the user's skin at proximal target (muscular) locations on the user's face when the headset is worn by the user; these electrodes 122 are each electrically coupled to discrete electrical channels 132 (or "traces") on the substrate 130 and arranged across the exterior surface of the elastic member no, each of which is electrically coupled to the controller 140, such as via a wire or flexible PCB. If the gaskets are instead attached to a pair of optical glasses, the discrete electrical traces 128 can be incorporated into a substrate 130 within the frame of the glasses.
  • the controller 140 is configured to read electromyography (or "EMG") signals - indicative of muscle movements and therefore expressions or expression changes on the user's face - from sense electrodes 123 in contact with regions of the user's face while the headset is worn by the user. Additionally, the controller 140 can read electrical signals generated by sense electrodes 123 moving across the surface of a user's skin as the user changes expression. The controller 140 can then process these signals (e.g., changes in these signals over time) to compute expressions or transitions between expressions on a user's face, such as by passing voltage potentials read from these sensors over a sequence of sampling periods into a neural network (e.g., a long short-term memory recurrent neural network).
  • a neural network e.g., a long short-term memory recurrent neural network
  • the controller 140 can then return values for these expressions to a gaming console or other computing device that can update a virtual avatar - representing the user within a virtual environment - to embody these expressions substantially in real-time.
  • the values representing a user's expression can be applied to other media, such as a text or audio message to "tag" the media with an emotion.
  • a media tag could indicate that a user was smiling while dictating a message into a pair of smart optical glasses or goggles integrated with the system 100.
  • the controller 140 can return values to a headset indicating whether, and to what degree, the user is squinting.
  • the controller 140, or another processing unit in the headset can then adjust the brightness of the screen, or adjust the opacity of the lenses, in the case of a pair of optical glasses or goggles, to reduce the amount of light incident to the user's eyes.
  • the system 100 can be integrated into or installed in a headset (e.g., a virtual reality head-mounted display headset, a pair of ski goggles, a pair of optical glasses, etc.) to detect facial expressions of a user wearing the headset by directly reading voltage potentials on the user's face through a small number of passive electrodes 122, correlating these voltage potentials with muscle movements, and interpreting these muscle movements as facial expressions (e.g., rather than remotely sensing the user's face, which may be visually obstructed by the headset).
  • a headset e.g., a virtual reality head-mounted display headset, a pair of ski goggles, a pair of optical glasses, etc.
  • the sensor assembly 170 (including the electrodes 122, substrate 130, and electrical traces 128) are separable and remote from the controller 140 and replaceable with a second sensor assembly 170. Because the gasket is designed to contact the user's skin, oxidation and wear may occur across these electrodes 122 during use on the sensor assembly 170; the sensor assembly 170 can therefore be fabricated with materials described herein and according to the manufacturing method S200 described below in order to limit total cost of the sensor assembly 170 such that the sensor assembly 170 may be conveniently replaced once oxidization has sufficiently inhibited function of the electrodes 122.
  • the sensor assembly 170 is manufactured by printing conductive ink onto a flexible substrate 130 to define a series of electrodes 122, electrical channels 132, and a common junction 136 on the surface of the flexible substrate 130.
  • a single component defines all the electrical components of the sensor assembly 170. Then the unitary structure defining the electrical components is adhered to the elastic member 110 in a single step, thereby reducing manufacturing costs.
  • the common junction 136, electrical channels 132, and electrical pads 134 are printed onto a substrate 130 which is then selectively cut from the substrate 130 to define a substrate ring.
  • Electrode tabs 120 are printed and cut from a separate flexible polymer sheet and adhered to radially extend from the substrate ring before being adhered to the elastic member 110.
  • the substrate ring can be manufactured from a lower cost substrate 130 than the electrode tabs 120, thereby reducing cost of the sensor assembly 170.
  • the system 100 includes electrode tabs 120, which define structural elements configured to position electrodes in contact with a user's face when the headset is worn by the user.
  • one subset of the electrode tabs 120 define sense electrode tabs
  • a second subset of the electrode tabs define reference electrode tabs. While the sense electrode tabs and the reference electrode tabs maybe both structurally consistent with electrode tabs 120 in general, they may differ in their positioning along the elastic member 110 and thus function to read either sense or reference signals from the user's skin. Therefore, structural descriptions of electrode tabs 120 below can apply to both sense electrode tabs and reference electrode tabs.
  • electrodes 122 can be divided into two subsets, including sense electrodes 123 and reference electrodes 124.
  • sense electrodes 123 and reference electrodes 124 can be structurally similar but can differ in their location on the elastic member 110 and how the controller processes signals received from each type of electrode. Therefore, structural descriptions of electrodes 122 below can apply to both sense electrodes 123 and reference electrodes 124.
  • the system 100 is described with reference to particular applications, including: the system 100's integration with a VR headset; and the system 100's integration with a pair of optical glasses or goggles.
  • the system 100 can be integrated into a wearable or other computing device of any other type or form, and the controller can implement any other methods or techniques to interpret a facial expression of a user wearing the device based on sense and/or reference signals read by sense and/or reference electrode tabs integrated into the system 100.
  • the system 100 can be configured to physically interface with an augmented or virtual reality (or "VR") headset, and to detect and communicate facial expressions, facial movements, eye movements and/or mouth movements to a computing device - such as an external game console or a processor integrated into the headset - that provides extended expressive functionality to the headset and associated computing device based on the detected facial expression of the user.
  • a computing device - such as an external game console or a processor integrated into the headset - that provides extended expressive functionality to the headset and associated computing device based on the detected facial expression of the user.
  • the sensor assembly 170 when the VR headset 150 with installed sensor assembly 170 is worn by a user, the sensor assembly 170 is configured to: sit between the view window of the VR headset 150 - around a display integrated into the VR headset 150 - and the user's face; to cushion the view window around the user's face; and conform to the user's facial structure while depressing the sense electrodes 123 and reference electrodes 124 against the user's skin.
  • the sensor assembly 170 can define a perimeter configured to extend across and to contact: the left and right supraorbital regions; the zygomatic regions; the infraorbital regions; and the lower frontal region and/or the upper nasal region of a user's face when the headset is worn by the user.
  • the sensor assembly 170 can thus include electrodes 122 supported directly against a user's skin - rather than against the user's facial hair - at convex regions of the user's face, which may enable sufficient electrode-to-skin contact and reduce noise in signals read from these electrodes 122 due to poor skin-electrode contact.
  • the system 100 defines an aftermarket headset augmentation system in which: the passive sensor assembly 170 is configured to replace a gasket arranged around the perimeter of the view window of the VR headset 150, such as with a hook-and-loop attachment system.
  • the controller 140 and signal processing circuitry can be arranged in a housing configured to install remotely from the sensor assembly 170, such as: to mount to the body of the headset with a hook-and-loop attachment system; or to connect to a head strap on the headset with a clip or loop extending from the housing and configured to receive or clip onto the head strap.
  • the signal processing circuitry can condition signals read from electrodes 122 in the sensor assembly 170, and the controller 140 can locally transform these signals into facial expressions, facial movements, and/or mouth movements of a user; and the system 100 can further include a wireless communication module configured to broadcast facial state updates or commands related to these detected facial expressions, facial movements, and/or mouth movements to the headset or to a computing device nearby (e.g., a gaming console) for embodiment in a virtual avatar within a virtual environment.
  • a wireless communication module configured to broadcast facial state updates or commands related to these detected facial expressions, facial movements, and/or mouth movements to the headset or to a computing device nearby (e.g., a gaming console) for embodiment in a virtual avatar within a virtual environment.
  • the system 100 defines an integrated headset product in which the headset is supplied with the sensor assembly 170 arranged around the view window and in which the controller 140 and signal processing circuitry are integrated into the headset.
  • conditioned signals - read from the electrodes 122 in the sensor assembly 170 and processed by the signal processing circuitry - can be transmitted (e.g., wirelessly or via a wired connection) to an external computing device nearby for transformation into facial expressions and/or facial movements of the user.
  • the sensor assembly 170 can be permanently installed around the view window of the headset, such as with adhesive.
  • the sensor assembly 170 can be disposable and configured to transiently install around the view window of the headset.
  • one component of a hook- and-loop attachment system can be adhered across all or a portion of the exterior surface of the sensor assembly 170 - opposite the electrodes 122; and a complementary component of a hook-and-loop attachment system can be attached to the headset at corresponding locations around the view window.
  • one unit of the sensor assembly 170 can be installed on the headset by mating the components of the hook-and-loop attachment system; after some period of use, once surfaces of the electrodes 122 exhibit sufficient wear, once signal quality from the first unit of the sensor assembly 170 has dropped below a threshold, or once a channel of the first unit of the sensor assembly 170 has ceased to function, the first unit of the sensor assembly 170 can be replaced with a second unit of the sensor assembly 170.
  • the sensor assembly 170 can be transiently (e.g., removably) connected to the headset with magnets, with mechanical fasteners, with snaps, or by installing the sensor assembly 170 into an undercut channel around the perimeter of the view window of the VR headset 150, etc.
  • the elastic member 110 can transiently couple to the headset via a hook-and-loop attachment surface arranged over the substrate 130, the hook-and- loop attachment surface configured to align with a corresponding hook-and-loop attachment surface on the headset.
  • a unit of the sensor assembly 170 can be integrated into or installed on a VR headset 150 or headset of any other type with a similar geometry (e.g. ski goggles, mountain bike goggles, face shields for manufacturing applications, etc.) and in any other suitable way.
  • the system 100 can be configured to integrate with a headset that can be a pair of optical glasses or goggles outfitted with a processor for controlling the opacity of the lenses, for displaying augmented reality images to the user, or for displaying a heads-up-display (HUD) to the user.
  • a headset that can be a pair of optical glasses or goggles outfitted with a processor for controlling the opacity of the lenses, for displaying augmented reality images to the user, or for displaying a heads-up-display (HUD) to the user.
  • the goggles e.g. ski goggles, mountain biking goggles, welding goggles, etc.
  • the sensor assembly 170 can be configured in a similar configuration to the VR headset variation, and can detect and communicate the facial expressions, facial movements, eye movements and/or mouth movements to a computing device or processor integrated with the goggles.
  • the integrated processor can then increase the opacity of the lenses of the goggles in response to detecting the user squi
  • the system can be integrated with a pair of optical glasses, and the electrodes 122 can be located between the nose pads of the optical glasses and the user's nose bridge region and between at least two points on the frame of the glasses in contact with the left and right supraorbital regions or left and right zygomatic regions of the user's face.
  • the electrodes 122 can be located in different positions than in the VR headset 150 variation.
  • the sensor assembly 170 can define discrete sense electrodes 123 in four or more separate locations on the pair of optical glasses: an upper left sense electrode coupled to the frame as it sweeps back toward the temple and configured to proximally contact a left upper orbicularis oculi muscular region of the face of the user; an upper right sense electrode coupled to the frame as it sweeps back toward the temple and configured to proximally contact a right upper orbicularis oculi muscular region of the face of the user; a left nose pad sense electrode coupled to the internal surface of the nose pads of the headset and configured to proximally contact a left levator labii superioris alaeque nasi muscular region (i.e.
  • the electrodes 122 and corresponding elastic members 110 can be located elsewhere on the frame, where it comes into close proximity to the user's skin (e.g. the left or right infraorbital or zygomatic regions of the face; or the left or right temples of the user).
  • the substrate 130, electrical traces 128, electrical pads 134, electrical channels 132, common junction 136, and controller 140 are located internally within the frame of the optical glasses. Electrical pads 134 on the internal substrate can be exposed by openings in the frame to allow for electrical contact between the electrodes 122 and the electrical pads 134 on the substrate 130.
  • the discrete elastic members 110 and corresponding electrodes 122 are individually replaceable using a snap or other mechanical fastening method - wherein a user can remove and replace each electrode 122 and/or discrete elastic member 110 from its location on the frame or nose pad of the optical glasses - while the substrate 130, electrical traces 128, electrical pads 134, electrical channels 132, and common junction 136 are permanently integrated within the frame of the glasses.
  • a gasket like elastic member 110 can be positioned across the top of the optical glasses frame in the supraorbital region of the user's face when the user is wearing the optical glasses, and can include multiple electrodes 122.
  • a second elastic member 110 or members can be positioned on the nose pads in the nose bridge region of the user's face when the user is wearing the optical glasses.
  • the electrodes 122 are manufactured from a durable conductive material, such as solid copper, silver chloride coated solid silver, or any other conductive metallic or non-metallic substance.
  • the durable conductive electrodes 122 can be integrated directly into the frame and can be permanent fixtures on the glasses.
  • the system 100 includes a controller 140 integrated into the optical glasses or goggles, which can read and process electrical signals from the sensor assembly 170.
  • the controller 140 or another processor integrated into the optical glasses or goggles can increase the opacity of electrochromic lens or lenses of the optical glasses or goggles via an applied voltage on the lenses.
  • the sensor assembly 170 can be integrated with a pair of optical glasses or goggles in any other way.
  • the elastic member 110 is configured to couple to a headset or pair of optical glasses or goggles; is configured to extend along supraorbital regions, zygomatic regions, and infraorbital regions of a face of a user when the headset is worn by the user; defines an opening configured to align with palpebral regions of the face of the user when the headset is worn by the user; and includes a compressible material configured to conform to the face of the user when the headset is worn by the user.
  • the elastic member 110 can be continuous and can extend along supraorbital regions and zygomatic regions of the face of the user and is offset from ocular regions of the face of the user when the headset is worn by the user, thereby allowing the user to see.
  • the elastic member 110 functions as a soft, compressible gasket that buffers the geometry of the headset, optical glasses, or goggles, around a view window or set of lenses to a user's unique facial structure and that depresses flexible electrodes 122 against skin on the user's face, thereby deforming the electrodes 122 into conformation with the user's facial structure to achieve sufficient contact between the electrodes 122 and the user's skin to reduce noise in EMG signals read from these electrodes 122 due to poor electrode contact.
  • the elastic member 110 can define a closed-loop configured to: extend from the lower frontal region of the face above the nasal bridge, across the left supraorbital region, around the left zygomatic region to the left infraorbital region, over the nose below the nasal bridge, across the right infraorbital region to the right zygomatic region, up the supraorbital region, and back to the lower frontal region of the face above the nasal bridge to close the loop.
  • the elastic member 110 can define an open loop, such as open over the nose of a user's face below the nasal bridge.
  • the elastic member no can include two separate, discrete components, including: a left component configured to contact the left side of a user's face around the user's left eye; and a right component configured to contact the right side of a user's face around the user's right eye.
  • the elastic member no can be constructed of foam, such as polyurethane or neoprene foam.
  • the elastic member no can be cast in a mold or cut from planar foam sheet, such as with a punch and die.
  • the elastic member no can be of closed-cell foam (e.g., neoprene) in order to reduce breathability of the elastic member no and thus increase local sweating on the user's face when the headset is worn by the user, which may in turn improve electrical conductivity between electrodes 122 and the user's skin and thus improve signal-to- noise ratio (or "SNR") of the signals detected by these electrodes 122.
  • closed-cell foam e.g., neoprene
  • the elastic member 110 can be discontinuous and can be positioned in different locations than in other variations.
  • the elastic member 110 can be located on the nasal pads of the optical glasses at the nasal bridge region of a user's face when wearing the optical glasses and/or on the frame of the optical glasses proximal to the left and right zygomatic regions of a user's face when wearing the optical glasses.
  • the elastic member 110 can be constructed out of a different elastic material, for example, a rubber or silicon compound.
  • the electrodes 122 can be directly affixed to the nose pads and/or frame of the optical glasses with no interposed elastic member 110.
  • one elastic member 110 extends across from the left to the right supraorbital regions of the face of the user when the user is wearing the optical glasses, while another elastic member 110 is located in the nasal bridge region of the face of the user.
  • the elastic member 110 can be of any other form and constructed in any other soft, compressible, flexible, and/or elastic material.
  • the sense electrodes 123 and reference electrodes 124 define passive, monopolar conductors configured to contact a user's skin directly and to communicate voltage potentials on a user's skin back to the controller 140 (e.g., via the substrate 130 and a connector, as described below).
  • the sense electrodes 123 can be arranged over regions of the elastic member 110 that: commonly oppose distinct muscle groups in the human face responsible for facial and mouth motions; and/or that commonly oppose convex regions of the human face (e.g., over the zygomatic or "cheek" bones, supraorbital bones of the human skull).
  • the sense electrodes 123 can be arranged in particular locations on the elastic member 110 in order to achieve both: detection of a limited number of EMG signals representative of a variety of facial expressions or facial expression changes; and effective contact with a user's skin.
  • the signal processing circuitry reads a difference between a signal received from a sense electrode and a signal received from the reference electrode, the reference electrode 124 can be arranged on a region of the elastic member 110 that commonly faces smaller or less active facial musculature (i.e. low-muscle regions), such as over the nasal bridge, in order to limit rejections of significant signal components from EMG signals received from the sense electrodes.
  • the system 100 includes a reference electrode 124 that is coupled to the elastic member 110 and configured to contact a low-muscle region on the face of the user when the headset is worn by the user; including a reference electrode 124 defining an electrical contact surface facing outwardly from an interior face of the elastic member 110 opposite the substrate 130 and configured to contact skin on the face of the user when the headset is worn by the user; and electrically coupled to an electrical channel 132, in the set of electrical channels 132, defined on the substrate 130.
  • the system 100 includes four sense electrodes 123, including: a lower left sense electrode 123A arranged at a left zygomaticus / left levator labii superioris muscle region of the elastic member 110 (i.e., under the left eye); an upper left sense electrode 123B arranged at a left upper orbicularis oculi muscle region of the elastic member 110 (i.e., over the left eye); a lower right sense electrode 123C arranged at a right zygomaticus / right levator labii superioris muscle region of the elastic member 110 (i.e., under the right eye); and an upper right sense electrode 123D arranged at a right upper orbicularis oculi muscle region of the elastic member 110 (i.e., over the right eye).
  • the system 100 can also include a single reference electrode 124 arranged over a procerus muscle at the nasal bridge region of the elastic member 110.
  • the system 100 further includes: an outer left sense electrode 123E arranged at a left outermost orbicularis oculi muscle region of the elastic member 110 (i.e., to the left of the left eye); and an outer right sense electrode 123F arranged at a right outermost orbicularis oculi muscle region of the elastic member no (i.e., to the right of the right eye).
  • the system loo can also include: an upper-inner left sense electrode arranged over a left upper orbicularis oculi / procerus muscle junction region of the elastic member no (i.e., over the left eye between the upper left sense electrode and the vertical centerline of the elastic member no); and an upper-inner right sense electrode arranged over a right upper orbicularis oculi / procerus muscle junction region of the elastic member no (i.e., over the right eye between the upper right sense electrode and the vertical centerline of the elastic member no).
  • the system 100 includes four sense electrodes including: an upper left sense electrode 123G coupled to the frame and configured to proximally contact a left upper orbicularis oculi muscular region of the face of the user when the headset is worn by the user; an upper right sense electrode 123H coupled to the frame and configured to proximally contact a right upper orbicularis oculi muscular region of the face of the user when the headset is worn by the user; a left nose pad sense electrode 123I coupled to the internal surface of the nose pads of the headset and configured to proximally contact a left levator labii superioris alaeque nasi muscular region of the face of the user when the headset is worn by the user; and a right nose pad sense electrode 123J coupled to the internal surface of the nose pads of the headset and configured to proximally contact a right levator labii superioris alaeque nas
  • the system 100 can include any other number of sense electrodes 123 and / or reference electrodes 124 arranged in any other configuration over the elastic member 110. Furthermore, the system 100 can exclude a physical reference electrode 124, and the controller 140 can instead calculate a virtual reference signal as a function of a linear combination of sense signals read from sense electrodes 123 on the sensor assembly 170.
  • the substrate 130 is arranged across an exterior surface of the elastic member 110 (i.e., opposite a user's face) and defines a set of discrete electrical channels 132.
  • the set of electrode tabs 120 are distributed along the elastic member 110 and define a set of electrodes 122, wherein each electrode 122 is arranged on the interior surface of the elastic member no, faces opposite the substrate 130 (i.e., toward a user's skin), and is electrically coupled to one electrical channel 132 on the substrate 130.
  • the substrate 130 and electrode tabs 120 form a unitary structure.
  • conductive metallic ink e.g., silver ink with a silver chloride coating over exposed regions
  • a sheet of flexible substrate material such as polyimide, polyether ether ketone (or "PEEK”), transparent conductive polyester film, or thermoplastic polyurethane (or "TPU") to form a set of discrete (round) electrodes 122, one discrete electrical channel 132 per electrode and terminating on one end at a common junction 136, and one discrete trace extending from a corresponding electrode 122 to a corresponding electrical channel 132, as shown in Block S110.
  • the common junction 136 can be defined near a planned edge of the substrate 130, the discrete electrical channels 132 can extend along a planned substrate 130 area matching the geometry of the elastic member 110, and the traces can extend over planned electrode tab areas to electrodes offset inside or outside of the planned substrate 130 area on the sheet of flexible substrate material, as shown in FIGURE 4.
  • the exterior surface of the sheet of flexible substrate material and the conductive material - exclusive of the electrodes 122 and the common junction 136 - can then be covered or coated as shown in Block S120.
  • a second layer of nonconductive material such as TPU, can be perforated at regions corresponding to the electrodes 122 and the common junction 136 on the exterior surface of the sheet of substrate material; and the second layer of nonconductive material can be aligned with and bonded to the exterior surface of the sheet of substrate material.
  • the exterior surface of the sheet of substrate material can be mechanically masked across the electrodes 122 and common junction 136; and a nonconductive coating (e.g., a polyurethane, an epoxy, a thermoplastic, or latex, etc.) can be sprayed across the exposed area of the sheet of substrate material to enclose the electrical channels 132, thereby preventing shorting across channels and protecting these channels against environmental contaminants.
  • a nonconductive coating e.g., a polyurethane, an epoxy, a thermoplastic, or latex, etc.
  • the nonconductive coating can be printed directly onto the flexible substrate material such that it encloses the electrical traces 128 and electrical channels 132 without the use of mechanical masking.
  • non-oxidative coatings such as a silver chloride coating can be applied to the electrodes 122 before or after the nonconductive coating has been applied.
  • the step of applying various inks to the flexible substrate 130 can include: applying a layer of silver ink to thermoplastic polyurethane; applying a layer of silver chloride over the set of sense electrodes 123; and applying a layer of thermoplastic polyurethane over the set of electrical traces 128, electrical channels 132, and the common junction 136.
  • the substrate 130 and the electrode tabs 120 form a unitary structure defining: a substrate ring defining the set of discrete electrical channels 132 and a common junction 136 and conforming to the geometry of the elastic member 110; for each electrode tab 120 in the set of electrode tabs 120: an electrode tab area defining the electrode tab 120 and extending radially from a perimeter of the substrate ring; and a cover layer covering the set of discrete electrical channels 132, the enclosed traces in the set of electrode tabs 120, and the common junction 136.
  • the sheet of substrate material and the layer of nonconductive material - enclosing the electrical channels 132, electrical traces 128, and electrodes 122 - can then be trimmed or selectively cut, such as with a computer-controlled plotting tool or laser cutter, to form a substrate-electrode assembly 160 including a substrate ring and a set of electrode tabs 120 extending from the substrate ring, wherein each electrode tab 120 terminates at or around an exposed electrode 122 (i.e., an electrode 122 not covered by the layer of nonconductive material), as shown in Block S130 .
  • each electrode tab 120 can extend outwardly from the perimeter of the substrate ring and can be folded around the exterior of the elastic member 110 to limit a user's visibility of electrode tabs 120 when the user wears the headset.
  • the electrode tabs 120 can extend inwardly toward the center of the substrate ring, such as to reduce material waste.
  • the process can be implemented continuously along a sheet roll of substrate material and an adjacent sheet roll of the nonconductive material, such as by: sequentially stamping or printing conductive material onto the exterior surface of a sheet of substrate material to form one discrete group of electrodes 122, traces, and electrical channels 132 per unit of the substrate-electrode assembly 160; curing the conductive material; perforating a sheet of the nonconductive material at the locations of electrodes 122 on the adjacent sheet of substrate material, such as with a first die cutter; aligning and merging the sheet of substrate material and the sheet of nonconductive material; and bonding the sheet of substrate material and the sheet of nonconductive material together, such as between a pair of heated rollers.
  • this stack can be passed continuously through a second die cutter, which can separate units of the substrate-electrode assembly 160 from the stack, each in the form of a substrate ring and a set of electrode tabs 120 extending radially from the ring and wherein each electrode tab 120 terminates at or around an exposed electrode 122 (i.e., an electrode 122 not covered by the nonconductive material).
  • the component can be adhered to the elastic member 110, as shown in Block S140, to assemble the sensor assembly 170.
  • the substrate ring - including the electrical traces 128, electrical channels 132, and common junction 136 - is configured to align with the shape of the elastic member 110.
  • an adhesive can be applied to the interior surface of the substrate ring such that it adheres with the exterior surface of the elastic member 110, wherein the interior surface of the substrate ring is the surface opposite the electrical traces 128 and common junction 136.
  • Adhesive can then be applied to the interior surface of the extending sense electrode tabs 120, which can then be folded or wrapped around the elastic member 110 such that the electrodes 122 at the end of the electrode tabs 120 face internally (i.e. toward the user's face) opposite their original orientation.
  • the electrodes 122 are adhered to the interior surface of the elastic member 110 and contact the face of the user when a headset including the elastic member 110 is worn by the user.
  • the substrate-electrode assembly 160 can be adhered, bonded, or otherwise coupled to the elastic member 110 in any other way.
  • an adherent can be applied to an exterior surface of the sensor assembly 170, wherein the adherent is configured to transiently couple the sensor assembly 170 to a view window of a VR headset 150, or around the lenses of a pair of optical goggles, as shown in Block S150.
  • the adherent can be a hook-and-loop attachment surface corresponding to a hook-and-loop attachment surface around the view window of the VR headset 150.
  • the adherent can be a temporary adhesive that can be effectively removed from the VR headset 150 by peeling the sensor assembly 170 from the headset.
  • the common junction 136 can be electrically coupled to the VR headset 150.
  • electrical channels 132 - each terminating on one end at a common junction 136 and on an opposite end at an electrical pad 134 - are fabricated over the interior surface of flexible PCB substrate 130 (e.g., a polyimide, PEEK, or polyester sheet or film), such as by etching a copper foil on the flexible PCB to form these electrical channels 132 and other features as shown in Block S210.
  • conductive ink can be printed onto the surface of the flexible substrate 130.
  • the flexible PCB substrate 130 is then masked (e.g., mechanically) over the common junction 136 and over the electrical pads 134; and exposed areas of substrate 130 are sealed, such as with a spray or dip coating or by adhering a sheet of nonconductive material (e.g., TPU) over these areas of the substrate 130.
  • the substrate 130 is then trimmed to form a ring of geometry substantially similar to that of the exterior surface of the elastic member 110, as shown in Block S220.
  • electrode tabs 120 are fabricated separately.
  • an electrode 122, a junction pad 126, and an electrical trace 128 - extending between the electrode 122 and the junction pad 126 - can be screen-printed onto the exterior surface of a first flexible, nonconductive layer or base layer (e.g., a sheet of TPU), as shown in Block S230.
  • a second flexible, nonconductive layer is bonded over the first flexible layer across the trace, thereby insulating the trace and completing the electronic tab, as shown in Block S232.
  • Electrodes tabs 120 in bulk on sheet rolls of flexible, nonconductive material; screen-print or stamp conductive material onto the exterior surface of the first sheet of flexible, nonconductive material to form discrete electrode 122, junction pad 126, and trace groups; perforate or selectively cut the second sheet of the flexible, nonconductive material at locations of electrodes 122 and junction pads 126 on the first sheet of flexible, nonconductive material, as shown in Block S240; bond the first and second sheets of flexible, nonconductive material together, such as by passing these two sheets - in alignment - through a pair of heated rollers; and then separate each electrode tab 120, such as with a punch and die.
  • an electrode 122 can include: a base layer including a nonconductive polymer; conductive ink deposited across a first surface of the base layer; a junction pad 126 comprising conductive ink deposited across the first surface of the base layer; and an enclosed trace.
  • the enclosed trace including an electrical trace 128 including conductive ink deposited between the junction pad 126 and the electrode 122 across the first surface of the base layer; and a nonconductive cover layer bonded to the base layer and enclosing the electrical trace 128 between the nonconductive cover layer and the base layer.
  • an electrode tab 120 can include: a base layer fabricated from thermoplastic polyurethane; conductive ink including silver ink with a silver chloride coating that prevents oxidation of the silver ink.
  • Electrode tabs 120 can then be connected to the substrate 130 to complete the substrate-electrode assembly 160, as shown in Block S250.
  • a junction pad 126 of an electrode tab 120 - exposed across the exterior surface of the electrode tab 120 - can be arranged over an exposed electrical pad 134 on the interior surface of the substrate 130 and can be fixed to the substrate 130, such as with the electrode tab 120 extending outwardly from the perimeter of the substrate 130 or facing toward the center of the substrate ring, as described above.
  • the junction pad 126 of the electrode tab 120 can be bonded to its corresponding electrical pad 134 on the substrate 130 with a conductive adhesive.
  • the electrode tab 120 and the substrate 130 can be mechanically fastened with a (conductive) rivet passing through the junction pad 126 and the electrical pad 134.
  • the first and/or second layers of the electrode tab 120 can extend around the perimeter of the junction pad 126 and can be of TPU; the electrode tab 120 can thus be bonded to the substrate 130 by compressing and heating the electrode tab 120 around the junction pad 126 against the substrate 130, such as after conductive paste is applied between the junction pad 126 and the electrical pad 134.
  • the substrate- electrode assembly 160 can be adhered to the elastic member 110, as shown in Block S260, to create the sensor assembly 170.
  • the substrate ring including the electrical channels 132 and common junction 136 is configured to align with the shape of the elastic member 110.
  • an adhesive can be applied to the interior surface of the substrate ring such that it adheres with the exterior surface of the elastic member 110, wherein the interior surface of the substrate ring is the surface including the electrical traces 128 and common junction 136.
  • Adhesive can then be applied to the interior surface of the extending sense electrode tabs 120, which can then be folded or wrapped around the elastic member 110 such that the electrodes 122 at the end of the electrode tabs 120 face internally, opposite their original orientation (i.e. toward the user's face).
  • the electrodes 122 are adhered to the interior surface of the elastic member 110 and contact the face of the user when a headset including the sensor assembly is worn by the user.
  • the substrate-electrode assembly 160 can be adhered, bonded, or otherwise coupled to the elastic member 110 in any other way.
  • the base layer of the electrode tab 120 can be bonded to an interior surface of the substrate 130, wherein the base layer is interposed between the substrate 130 and the elastic member 110, and the junction pad 126 contacting a discrete electrical channel 132 in the set of electrical channels 132 on the substrate 130; the base layer further defines a second surface, opposite the first surface of the base layer, wherein the second surface contacts an outer perimeter surface of the elastic member 110 and is coupled to the interior surface of the elastic member 110; and the electrode 122 of each electrode tab 120 faces outwardly from the first surface of the base layer and is configured to contact skin on the face of the user when the headset is worn by the user.
  • an adherent can be applied to an exterior surface of the sensor assembly 170, wherein the adherent is configured to transiently couple the sensor assembly 170 to a view window of a VR headset 150, or pair of optical goggles, in Block S270.
  • the adherent can be a hook-and- loop attachment surface corresponding to a hook-and-loop attachment surface around the view window of the VR headset 150, or pair of optical goggles.
  • the adherent can be a temporary adhesive that can be effectively removed from the VR headset 150, or pair of optical goggles, by peeling the sensor assembly 170 from the headset.
  • the common junction 136 can be electrically coupled to the VR headset 150, or pair of optical goggles.
  • a planar substrate-electrode assembly 160 - including a substrate ring with electrode tabs 120 extending outwardly or inwardly from the substrate ring - can be fabricated in any other way and in any other material.
  • the substrate 130 can be internally integrated directly into the frame of the optical glasses 180.
  • the internally integrated substrate 130 is a thin flexible PCB interposed between material layers of the frame of the optical glasses 180 and connecting directly with the controller 140 integrated with the frame of the optical glasses 180.
  • electrical channels 132 or wiring can be imbedded directly into the frame material itself, connecting the electrodes 122 to the controller 140 integrated with the optical glasses.
  • the electrical channels 132 or wiring can extend over or around the hinges of the optical glasses to a controller 140 integrated with the temples of the optical glasses.
  • electrode tabs 120 can be adhered to the internal substrate and extend out of the frame of the optical glasses 180 through slits in the frame of the optical glasses 180. In this implementation, depending on the orientation of the electrode tabs 120 exiting the frame of the optical glasses 180, the electrode tabs 120 can be adhered to elastic members 110 on the interior surface of the optical glasses 180.
  • the electrodes 122 can be affixed directly to the frame of the optical glasses 180, thereby coming into direct contact with electrical channels 132 integrated in the frame of the optical glasses.
  • the internal substrate can include spring contacts arranged at the attachment points for the electrodes 122 such that, when the electrodes 122 are attached to the frame, the electrodes 122 are electrically coupled to the electrical channels 132.
  • the electrical contact between electrodes 122 affixed to a pair of optical glasses and a controller 140 integrated with the pair of optical glasses can be established in any other way.
  • a common junction 138 can then be connected to the common junction 136 on the substrate 130.
  • a wire can be bonded or soldered directly over its corresponding electrical channel 132 at the common junction 136 on the substrate 130; opposing ends of these wires can terminate in a plug configured to transiently engage a receptacle coupled to the signal processing circuitry, and these wires can be bundled and wrapped to form a wiring harness 138.
  • a wiring harness 138 receptacle can be bonded, soldered, or mechanically fastened to the substrate 130 over the common junction 136; and a wiring harness 138 can then be installed in the wiring harness receptacle.
  • the substrate 130 includes a tongue extending outwardly from the substrate ring and terminating at its far end in the common junction
  • the discrete electrical channels 132 run along the tongue and terminate at the common junction 136 and are covered or coated with a nonconductive material, as described above.
  • the end of the tongue can function as a plug and can be configured to directly engage a receptacle coupled to the signal processing circuitry.
  • the system 100 can include a wiring harness 138 of any other form attached to the common junction 136 on the substrate 130 in any other way or any other feature or component configured to communicate voltage potentials at the sense electrodes 123 and reference electrodes 124 to the signal processing circuitry.
  • the elastic member 110 and the substrate-electrode assembly 160 can be bonded together or otherwise assembled, as shown in FIGURES 7 and 8.
  • the exterior surface of the elastic member 110 is bonded to the interior surface of the substrate 130; and electrode tabs 120 extending from the substrate ring defined by the substrate-electrode assembly 160 are wrapped around the elastic member 110 and bonded to the interior surface of the elastic member 110 with the exposed surfaces of the electrodes 122 facing opposite the elastic member 110 to form the passive sensor assembly 170.
  • the exposed surfaces of the electrodes 122 can contact select regions of the user's face; the elastic member 110 can depress the electrodes 122 against the user's skin around the view window or lens of the headset; and the electrodes 122 can locally deform with the elastic member 110 against the user's face to achieve both persistent contact between the electrodes 122 and the user's skin and sufficient comfort for the user.
  • an adhesive e.g., contact cement
  • adhesive can also be applied to select regions of the interior surface of the elastic member 110 adjacent each electrode tab 120; and each electrode tab 120 can then be wrapped around the elastic member 110 and bonded to the interior surface 112 of the elastic member 110.
  • a component of a hook-and-loop attachment system can then be adhered across all or a portion of the exterior surface of the substrate 130 to complete the sensor assembly 170; this hook-and-loop component on the sensor assembly 170 can then transiently engage a complementary hook-and-loop component arranged on a headset to transiently retain the sensor assembly 170 around the view window of the headset, as described above.
  • a button snap or other attachment component can be bonded or otherwise coupled to the sensor assembly 170 and configured to transiently retain the sensor assembly 170 around the view window of a headset.
  • the elastic member 110 and the substrate-electrode assembly 160 can be assembled in any other way.
  • an electrode tab 120 is perforated through its electrode 122.
  • perforations in the electrode 122 can enable the electrode tab 120 to deform and to conform against a user's skin, thereby improving electrical contact therebetween.
  • These perforations can also retain moisture (e.g., sweat) around the electrode 122 and adjacent the user's skin, thereby improving electrical conductivity between the electrode 122 and the user's skin.
  • these perforations may grip a user's skin, thereby reducing motion of the electrode 122 across the user's skin, which may induce noise in the channel.
  • the electrode tab 120 can be slit around or through the electrode 122 to improve conformation of the electrode 122 against the user's skin.
  • an electrode tab 120 can define any other geometry through or adjacent its electrode 122.
  • a thin flexible planar electrode 122 can be fabricated on a thin flexible planar sheet or film (e.g., TPU); this electrode tab 120 can then be wrapped around the elastic member 110 such that an exposed area of the electrode 122 faces a user's skin when a headset - outfitted with the sensor assembly 170 - is worn by the user.
  • a thin flexible planar electrode 122 can be fabricated on a thin flexible planar sheet or film (e.g., TPU); this electrode tab 120 can then be wrapped around the elastic member 110 such that an exposed area of the electrode 122 faces a user's skin when a headset - outfitted with the sensor assembly 170 - is worn by the user.
  • an electrode tab 120 can be formed into a three- dimensional structure. For example, once fabricated, an electrode tab 120 can be heat- formed over a domed (i.e., three-dimensional) mandrel in order to dome the electrode tab 120 across the electrode 122. The electrode tab 120 can then be installed over the elastic member 110 with the convex electrode 122 facing out from the interior surface of the elastic member 110.
  • a domed (i.e., three-dimensional) mandrel in order to dome the electrode tab 120 across the electrode 122.
  • the electrode tab 120 can then be installed over the elastic member 110 with the convex electrode 122 facing out from the interior surface of the elastic member 110.
  • an electrode tab 120 and its electrode 122 can be of any other two- or three-dimensional form. Electrode Fabrication on Elastic member
  • electrodes 122 are fabricated directly onto the elastic member 110, such as by screen-printing or spraying conductive material directly onto discrete regions of the elastic member 110 to form electrodes 122.
  • the elastic member 110 is cast in foam (e.g., open- celled foam to enable conductive ink to penetrate into and to be absorbed by the foam) and is perforated at the location of each electrode 122 to form a set of vias.
  • Conductive material e.g., silver ink
  • Conductive material can then be screen-printed onto the interior surface of the elastic member 110 around and into each via to form the set of discrete electrodes 122; in particular, conductive material can extend continuously from a discrete region on the interior surface of the elastic member 110, through the via, to the exterior surface of the elastic member 110.
  • the substrate 130 can include: a flexible PCB defining a geometry similar to the cross-section of the elastic member 110; one electrical pad 134 arranged on the interior surface of the flexible PCB at the location of each via in the elastic member 110; and one discrete electrical trace 128 arranged on the interior surface of the flexible PCB and extending from each electrical pad 134 to a common junction 136 near an edge of the flexible PCB.
  • the substrate 130 can then be aligned with a foam ring, and the interior surface of the substrate 130 can be bonded to the exterior surface of the foam ring; each electrical pad 134 on the interior surface of the substrate 130 can thus contact conductive material that coats an adjacent via in the foam ring, thereby electrically connecting the corresponding electrode 122 on the interior surface of the elastic member 110 to its corresponding electrical channel 132 on the substrate 130.
  • adhesive can be applied to the interior surface of substrate 130 outside of the electrical pads 134 and the common junction 136; conductive paste can be applied to (e.g., screen-printed onto) the electrical pads 134; and the substrate 130 can be aligned with and adhered to the elastic member 110 to complete the sensor assembly 170.
  • adhesive can be applied across the exterior surface of the elastic member 110; conductive paste can be applied to the exterior surface of the elastic member 110 around the vias; and the interior surface of the substrate 130 can be bonded to the exterior surface of the elastic member 110 to enclose open electrical channels 132 extending across the interior surface of the substrate 130.
  • the substrate 130 can include TPU, the electrical channels 132 and electrical pads 134 can be screen-printed onto the substrate 130, and the substrate 130 can be compressed against the exterior surface of the elastic member no and heated to bond the substrate 130 to the elastic member 110 with the substrate 130 and the elastic member 110 enclosing the electrical channels 132 and electrical pads
  • electrodes 122 can be fabricated directly onto the interior surface of the elastic member 110 in any other suitable way. Electrical channels 132 and the common junction 136 can be similarly fabricated across the exterior surface of the elastic member 110.
  • electrodes 122, electrical traces 128, junction pads 126, electrical channels 132, electrical pads 134, etc. can include silver ink printed directly onto the electrode tab 120 and directly onto the substrate 130 (or directly onto the elastic member 110). Exposed areas of silver ink not covered in a second layer of nonconductive material can be further coated with silver chloride ink, such as by screen-printing.
  • silver ink may exhibit relatively low volume electrical resistivity but may oxidize relatively quickly and may exhibit relatively high surface resistivity when oxidized; silver chloride ink may exhibit volume electrical resistivity greater than that of silver ink but may oxidize slowly relative to silver ink and may therefore exhibit low surface resistivity more consistently over time than exposed silver ink.
  • exposed silver ink can be coated with silver chloride ink in order to maintain relatively low bulk volume electrical resistivity and relatively low (and consistent) surface resistivity at each electrode 122 over time.
  • non-metallic conductive inks such as conductive carbon nanotube ink or graphene ink can be printed onto the electrode tab 120 or the substrate 130. Carbon nanotube or graphene ink may exhibit relatively low bulk volume electrical resistivity and surface resistivity while also being immune to oxidation. Because conductive non-metallic inks do not oxidize, a non- oxidative conductive coating such as silver chloride is not applied, thereby removing a step from the manufacturing process of the electrode tabs 120.
  • the controller 140 is configured: to read a set of sense signals read from the sense electrodes 123 defined by the set of sense electrode tabs 120; to interpret changes in the set of sense signals over time as expressions change on the face of the user when the headset is worn by the user; and to output the expressions to: a game console for representation on a virtual avatar in a virtual environment rendered on the headset; to a computational device to represent the user's emotion in connection with a concurrent message or other media; to an on-board processor on the headset to control the brightness or location of a screen or augmented reality projection of the headset; or to an on-board processor to control the opacity of a lens in a pair of optical glasses or goggles.
  • the controller 140 - in conjunction with signal processing circuitry - functions to read voltage potentials from sense electrodes 123 on the sensor assembly 170 (e.g., a relative to a reference potential read from the reference electrode 124) and to interpret these voltage potentials as facial expressions, transitions between facial expressions, facial movements, and/or mouth movements, etc.
  • the voltage potentials can be EMG signals from muscle activation in the user's face or signals created by the friction of a sense electrode 123 sliding along the skin of the user's face as she changes expression.
  • the signal processing circuitry filters sense and reference signals received from the sense electrodes 123 and reference electrodes 124; and outputs discrete digital values representative of a difference in voltage potential between a sense signal and the reference signals for each sense electrode 123 per sampling period.
  • the signal processing circuitry can repeat this process for each sampling period, such as at a rate of 20Hz.
  • the controller 140 can then pass these current digital values, digital values recorded during preceding sampling periods, and/or temporal changes in these digital values occurring over multiple sampling periods into a neural network (e.g., a long short-term memory recurrent neural network calibrated to a user wearing the headset) that outputs a list (e.g., a matrix) of movements and magnitudes of these movements in various muscle groups on the user's face.
  • a neural network e.g., a long short-term memory recurrent neural network calibrated to a user wearing the headset
  • a list e.g., a matrix
  • the controller 140 can then pass this list of movements and magnitudes of these movements into a characterization engine to associate these movements with a facial expression, facial movement, or mouth movement, such as one of a: large smile; soft smile; impassive; frown; confusion; surprise; shame; focus; exhaustion; anger; fear; sadness; happiness; disgust; contempt; frustration; boredom; mouth open or opening; eyes closed, closing, or squinting; mouth closed or closing; and/or various mouth shapes.
  • the controller 140 can then return the facial expression, facial movement, or mouth movement identified during the current sampling period to the game console or any other processor or computational device, such as via a wired or wireless connection.
  • the computation can then: project this facial expression, facial movement, or mouth movement onto a virtual avatar, such as by implementing a real-to-virtual expression mapping engine; update the virtual avatar within a virtual environment with this facial expression, facial movement, or mouth movement; generate a new frame representing the virtual environment and the virtual avatar; and serve this new frame to the headset for rendering on the display substantially in real-time.
  • the controller 140 can return the facial expression, such as a squint, to a processor in a pair of optical glasses, which may then adjust the opacity of the set of electrochromic lenses proportional to the degree to which the user is detected to be squinting. For example, if the squint detected by the controller 140 is subtle, the controller 140 may only slightly increase the opacity of the lenses. However, if the controller 140 detects that the user's eyes are almost entirely closed, the opacity of the glasses maybe increased to a higher degree.
  • the controller 140 can implement any other methods or techniques to transform sense and reference signals from the sensor assembly 170 into facial expressions, facial movements, and/or mouth movements, etc. on the user's face.
  • the systems and methods described herein can be embodied and/or implemented at least in part as a machine configured to receive a computer-readable medium storing computer-readable instructions.
  • the instructions can be executed by computer-executable components integrated with the application, applet, host, server, network, website, communication service, communication interface, hardware/firmware/ software elements of a user computer or mobile device, wristband, smartphone, or any suitable combination thereof.
  • Other systems and methods of the embodiment can be embodied and/or implemented at least in part as a machine configured to receive a computer-readable medium storing computer-readable instructions.
  • the instructions can be executed by computer-executable components integrated by computer-executable components integrated with apparatuses and networks of the type described above.
  • the computer-readable medium can be stored on any suitable computer readable media such as RAMs, ROMs, flash memory, EEPROMs, optical devices (CD or DVD), hard drives, floppy drives, or any suitable device.
  • the computer-executable component can be a processor but any suitable dedicated hardware device can (alternatively or additionally) execute the instructions.

Abstract

L'invention concerne un système de détection de mouvements faciaux d'un utilisateur comprenant : un ensemble capteur comportant un élément élastique (110) conçu pour s'accoupler à un casque et venir en contact avec le visage d'un utilisateur, à proximité et décalé par rapport à des régions oculaires du visage de l'utilisateur, et un substrat qui définit un ensemble de canaux électriques discrets et un ensemble de languettes d'électrodes réparties le long de l'élément élastique, chaque languette d'électrode étant conçue pour venir en contact proximal avec une région musculaire sur le visage de l'utilisateur; et un dispositif de commande conçu pour échantillonner un ensemble de signaux de détection provenant de l'ensemble de languettes d'électrodes via l'ensemble de canaux électriques discrets et pour interpréter des variations au fil du temps dans l'ensemble de signaux de détection sous forme d'expressions sur le visage de l'utilisateur lorsque le casque est porté par l'utilisateur.
PCT/US2018/034490 2017-05-24 2018-05-24 Système de détection de mouvements faciaux sur le visage d'un utilisateur WO2018218086A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US201762510651P 2017-05-24 2017-05-24
US62/510,651 2017-05-24

Publications (1)

Publication Number Publication Date
WO2018218086A1 true WO2018218086A1 (fr) 2018-11-29

Family

ID=64396034

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US2018/034490 WO2018218086A1 (fr) 2017-05-24 2018-05-24 Système de détection de mouvements faciaux sur le visage d'un utilisateur

Country Status (1)

Country Link
WO (1) WO2018218086A1 (fr)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20200008703A1 (en) * 2018-07-04 2020-01-09 Siemens Healthcare Gmbh Method for monitoring a patient during a medical imaging examination
US10924869B2 (en) 2018-02-09 2021-02-16 Starkey Laboratories, Inc. Use of periauricular muscle signals to estimate a direction of a user's auditory attention locus
US20220128813A1 (en) * 2019-06-07 2022-04-28 Hewlett-Packard Development Company, L.P. Eye movement controls in extended reality

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5983129A (en) * 1998-02-19 1999-11-09 Cowan; Jonathan D. Method for determining an individual's intensity of focused attention and integrating same into computer program
US20010031916A1 (en) * 1995-06-06 2001-10-18 Bennett Henry L. Electrode assembly and method for signaling a monitor
US20140318699A1 (en) * 2012-09-11 2014-10-30 Gianluigi LONGINOTTI-BUITONI Methods of making garments having stretchable and conductive ink
US20150250420A1 (en) * 2014-03-10 2015-09-10 Gianluigi LONGINOTTI-BUITONI Physiological monitoring garments
US20170060256A1 (en) * 2015-08-31 2017-03-02 Reach Bionics, Inc. System and method for controlling an electronic device with a facial gesture controller

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20010031916A1 (en) * 1995-06-06 2001-10-18 Bennett Henry L. Electrode assembly and method for signaling a monitor
US5983129A (en) * 1998-02-19 1999-11-09 Cowan; Jonathan D. Method for determining an individual's intensity of focused attention and integrating same into computer program
US20140318699A1 (en) * 2012-09-11 2014-10-30 Gianluigi LONGINOTTI-BUITONI Methods of making garments having stretchable and conductive ink
US20150250420A1 (en) * 2014-03-10 2015-09-10 Gianluigi LONGINOTTI-BUITONI Physiological monitoring garments
US20170060256A1 (en) * 2015-08-31 2017-03-02 Reach Bionics, Inc. System and method for controlling an electronic device with a facial gesture controller

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10924869B2 (en) 2018-02-09 2021-02-16 Starkey Laboratories, Inc. Use of periauricular muscle signals to estimate a direction of a user's auditory attention locus
US20200008703A1 (en) * 2018-07-04 2020-01-09 Siemens Healthcare Gmbh Method for monitoring a patient during a medical imaging examination
US20220128813A1 (en) * 2019-06-07 2022-04-28 Hewlett-Packard Development Company, L.P. Eye movement controls in extended reality

Similar Documents

Publication Publication Date Title
US20200133387A9 (en) System for detecting facial movements over a face of a user
US11822720B2 (en) Wearable computing device with electrophysiological sensors
WO2018218086A1 (fr) Système de détection de mouvements faciaux sur le visage d'un utilisateur
Lo et al. Skintillates: Designing and creating epidermal interactions
EP1970877B1 (fr) Affichage tactile electrique
CN205198685U (zh) 支撑使用者头部或头部和颈上可佩戴电刺激器的电极设备
CN101415567B (zh) 书写工具及其书写方法、计算机装置
CN100583007C (zh) 具有表面显示信息与互动功能的可动装置
US20160133151A1 (en) Modular wearable device for conveying affective state
US20220035317A1 (en) Wearable devices with overmolded electronic components and related methods
CN104820500A (zh) 一种具有脑电控制功能的虚拟现实头盔
CN105303793A (zh) 使用移动终端处理车辆的事故
WO2019036114A1 (fr) Capteur de potentiel électrique intra-auriculaire
CN101321495A (zh) 具有电极和运动传感器的医学传感器
WO2020205767A1 (fr) Procédés et appareil de détection et de classification de geste
US20230270363A1 (en) Smart Electrodes for Sensing Signals and Processing Signals Using Internally-Housed Signal-Processing Components at Wearable Devices and Wearable Devices Incorporating the Smart Electrodes
WO2016026028A1 (fr) Dispositif, système, procédé et/ou support lisible par un ordinateur destinés à être utilisés avec des données biologiques et non biologiques
US20110060206A1 (en) Device for determining body functions
WO2022098973A1 (fr) Système de commutation pour le fonctionnement d'un dispositif commandé
CN108670275A (zh) 信号处理方法及相关产品
KR20200101195A (ko) 전자 장치 및 그 제어 방법
US20230309829A1 (en) Smart Electrode Structures Made of a Conductive Deformable Material and Methods of Manufacturing the Smart Electrode Structures to Have Signal-Processing Components Embedded in the Conductive Deformable Material
Ab Wahid et al. Driving monitoring system application with stretchable conductive inks: A review
CN218728967U (zh) 一种指部动作的捕捉手套及虚拟现实系统
WO2023164211A1 (fr) Électrodes intelligentes pour détecter des signaux et traiter les signaux à l'aide de composants de traitement de signaux logés à l'intérieur au niveau de dispositifs pouvant être portés, et dispositifs pouvant être portés incorporant les électrodes intelligentes

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 18806184

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 18806184

Country of ref document: EP

Kind code of ref document: A1