WO2020264443A1 - Dispositif de détection multimodal portable - Google Patents

Dispositif de détection multimodal portable Download PDF

Info

Publication number
WO2020264443A1
WO2020264443A1 PCT/US2020/040009 US2020040009W WO2020264443A1 WO 2020264443 A1 WO2020264443 A1 WO 2020264443A1 US 2020040009 W US2020040009 W US 2020040009W WO 2020264443 A1 WO2020264443 A1 WO 2020264443A1
Authority
WO
WIPO (PCT)
Prior art keywords
sensing system
movement
pose
body part
sensors
Prior art date
Application number
PCT/US2020/040009
Other languages
English (en)
Inventor
Bruno Rodriques DE ARAUJO
Darren Leigh
David Holman
David Clark WILKINSON
Original Assignee
Tactual Labs Co.
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Tactual Labs Co. filed Critical Tactual Labs Co.
Priority claimed from US16/914,258 external-priority patent/US11759148B2/en
Publication of WO2020264443A1 publication Critical patent/WO2020264443A1/fr

Links

Classifications

    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/103—Detecting, measuring or recording devices for testing the shape, pattern, colour, size or movement of the body or parts thereof, for diagnostic purposes
    • A61B5/11—Measuring movement of the entire body or parts thereof, e.g. head or hand tremor, mobility of a limb
    • A61B5/1123—Discriminating type of movement, e.g. walking or running
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/0048—Detecting, measuring or recording by applying mechanical forces or stimuli
    • A61B5/0051—Detecting, measuring or recording by applying mechanical forces or stimuli by applying vibrations
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/103—Detecting, measuring or recording devices for testing the shape, pattern, colour, size or movement of the body or parts thereof, for diagnostic purposes
    • A61B5/11—Measuring movement of the entire body or parts thereof, e.g. head or hand tremor, mobility of a limb
    • A61B5/1107—Measuring contraction of parts of the body, e.g. organ, muscle
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/68—Arrangements of detecting, measuring or recording means, e.g. sensors, in relation to patient
    • A61B5/6801—Arrangements of detecting, measuring or recording means, e.g. sensors, in relation to patient specially adapted to be attached to or worn on the body surface
    • A61B5/6802—Sensor mounted on worn items
    • A61B5/681—Wristwatch-type devices
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/017—Gesture based interaction, e.g. based on a set of recognized hand gestures
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B2562/00—Details of sensors; Constructional details of sensor housings or probes; Accessories for sensors
    • A61B2562/02—Details of sensors specially adapted for in-vivo measurements
    • A61B2562/0204—Acoustic sensors
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B2562/00—Details of sensors; Constructional details of sensor housings or probes; Accessories for sensors
    • A61B2562/02—Details of sensors specially adapted for in-vivo measurements
    • A61B2562/0219—Inertial sensors, e.g. accelerometers, gyroscopes, tilt switches
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/02—Detecting, measuring or recording pulse, heart rate, blood pressure or blood flow; Combined pulse/heart-rate/blood pressure determination; Evaluating a cardiovascular condition not otherwise provided for, e.g. using combinations of techniques provided for in this group with electrocardiography or electroauscultation; Heart catheters for measuring blood pressure
    • A61B5/0205—Simultaneously evaluating both cardiovascular conditions and different types of body conditions, e.g. heart and respiratory condition
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/05—Detecting, measuring or recording for diagnosis by means of electric currents or magnetic fields; Measuring using microwaves or radio waves 
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/103—Detecting, measuring or recording devices for testing the shape, pattern, colour, size or movement of the body or parts thereof, for diagnostic purposes
    • A61B5/11—Measuring movement of the entire body or parts thereof, e.g. head or hand tremor, mobility of a limb
    • A61B5/1118—Determining activity level
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/68—Arrangements of detecting, measuring or recording means, e.g. sensors, in relation to patient
    • A61B5/6801—Arrangements of detecting, measuring or recording means, e.g. sensors, in relation to patient specially adapted to be attached to or worn on the body surface
    • A61B5/6813—Specially adapted to be attached to a specific body part
    • A61B5/6829—Foot or ankle

Definitions

  • the disclosed apparatus and method relate to the field of sensors, in particular the disclosed apparatus and method relate to gesture and human interaction sensors.
  • FIG. 1 shows a diagram of a sensing system.
  • FIG. 2 illustrates a sensing system incorporated into a wearable worn around the wrist of a user.
  • FIG. 3 illustrates a sensing system incorporated into a wearable worn around the wrist of a user.
  • FIG. 4 illustrates a sensing system incorporated into a wearable.
  • FIG. 5 illustrates a sensing system incorporated into a wearable.
  • FIG. 6 illustrates a cutaway view of a sensing system incorporated into a wearable 20.
  • FIG. 7 is a diagram illustrating the musculature of the arm.
  • FIG. 8 shows a pinch being determined by a sensing system.
  • FIG. 9 shows a touch of the fingertips being determined by the sensing system.
  • FIG. 10 shows a touch of a table being determined by the sensing system.
  • FIG. 1 1 shows a touch of a baseball being determined by the sensing system.
  • the present application contemplates an improved sensing system implementing a plurality of sensing modalities to characterize a movement or a pose of a body part.
  • the improved sensing system can be implemented on a wearable device. Through the use of multiple sensing architectures located strategically with respect to the anatomy of the body part an accurate representation of the body part can be achieved.
  • a user of a wearable device may interact with virtual objects by effecting a pinch or a tap that can then be discerned by the sensing system.
  • the term “pinch” refers to the act by a user of bringing two fingers in contact with each other.
  • the term“touch” refers to the act by a user of bringing a finger into contact with an object or another body part (including other fingers) without the subsequent pressure.
  • the term“tap” refers to the act by a user of a user bringing a digit into contact with an or another portion of the body and the time of contact is less than that of a touch.
  • the improved sensing systems disclosed herein can determine characteristics of the pose or movement (e.g., force, dwell, or within-contact motion - such as rubbing and sliding) thereby providing more sophisticated data to perfect the human-computer interaction.
  • the term“haptic” refers to any device capable of applying forces, vibrations, or motions to a user.
  • the improved sensing systems disclosed herein may be used in combination with other systems.
  • augmented reality headsets virtual reality headsets, mobile devices, wearables/smartwatches, external hand/finger tracking systems, cameras and infrared tracking systems, active or passive handheld props, or conductive or non-conductive surfaces, including touchscreens and computers; among many others.
  • the presently disclosed systems and methods present vast improvements over existing computer vision tracking systems, specifically in low-light or darkness conditions, when the body part is out of view of the tracking system (e.g., in the user’s pockets, behind the user’s back, or under a table), when interacting with held objects, and when wearing gloves or mittens.
  • first and second are not intended, in and of themselves, to imply sequence, time or uniqueness, but rather, are used to distinguish one claimed construct from another. In some uses where the context dictates, these terms may imply that the first and second are unique. For example, where an event occurs at a first time, and another event occurs at a second time, there is no intended implication that the first time occurs before the second time, after the second time or simultaneously with the second time. However, where the further limitation that the second time is after the first time is presented in the claim, the context would require reading the first time and the second time to be unique times.
  • first and a second frequency could be the same frequency, e.g., the first frequency being 10 Mhz and the second frequency being 10 Mhz; or could be different frequencies, e.g., the first frequency being 10 Mhz and the second frequency being 1 1 Mhz.
  • Context may dictate otherwise, for example, where a first and a second frequency are further limited to being frequency orthogonal to each other, in which case, they could not be the same frequency.
  • the presently disclosed systems and methods involve principles related to and for designing, manufacturing and using capacitive based sensors, signal infusion sensors (also known as signal injection), inertial measurement sensors (including accelerometers, gyroscopes, and magnetometers), pressure-based sensors, microphones, speakers, piezoelectric sensors, and millimeter wave signal-based sensors.
  • Some of the sensing modalities disclosed above may be implemented as microelectromechanical systems (MEMS).
  • MEMS microelectromechanical systems
  • the sensing modalities disclosed herein can employ a multiplexing scheme based on orthogonal signaling such as but not limited to frequency-division multiplexing (FDM), code-division multiplexing (CDM), or a hybrid modulation technique that combines both FDM and CDM methods.
  • FDM frequency-division multiplexing
  • CDM code-division multiplexing
  • hybrid modulation technique that combines both FDM and CDM methods.
  • References to frequency herein could also refer to other orthogonal signal bases.
  • this application incorporates by reference Applicants’ prior U.S. Patent No. 9,019,224, entitled“Low- Latency Touch Sensitive Device” and U.S. Patent No. 9,158,41 1 entitled “Fast Multi-Touch Post Processing.”
  • FDM frequency-division multiplexing
  • CDM code-division multiplexing
  • hybrid modulation technique that combines both FDM and CDM methods.
  • References to frequency herein could also refer to other orthogon
  • Orthogonal signals may be transmitted into a plurality of transmitting antennas (or conductors) and information may be received by receivers attached to a plurality of receiving antennas (or conductors).
  • receivers “sample” the signal present on the receiving antennas (or conductors) during a sampling period (t).
  • signal e.g., the sampled signal
  • touch events including, e.g., actual touch, near touch, hover and farther away events that cause a change in coupling between a transmitting antenna (or conductor) and receiving antennas (or conductor)).
  • one or more transmitting antennas can move with respect to one or more receiving antennas (or conductors), and such movement causes a change of coupling between at least one of the transmitting antennas (or conductors) and at least one of the receiving antennas (or conductors).
  • one or more transmitting antennas (or conductors) are relatively fixed with respect to one or more receiving antennas (or conductors), and the interaction of the signal and/or signals transmitted with environmental factors causes a change of coupling between at least one of the transmitting antennas (or conductors) and at least one of the receiving antennas (or conductors).
  • the transmitting antennas (or conductors) and receiving antennas (or conductors) may be organized in a variety of configurations, including, e.g., a matrix where the crossing points form nodes, and interactions are detected by processing of received signals.
  • the orthogonal signals are frequency orthogonal
  • spacing between the orthogonal frequencies, Af is at least the reciprocal of the measurement period t, the measurement period t being equal to the period during which the column conductors are sampled.
  • the signal processor of a mixed signal circuit is adapted to determine at least one value representing each frequency orthogonal signal transmitted to (or present on) a row conductor (or antenna). In an embodiment, the signal processor of the mixed signal circuit (or a downstream component or software) performs a Fourier transform on the signals present on a receive antenna (or conductor). In an embodiment, the mixed signal circuit is adapted to digitize received signals. In an embodiment, the mixed signal circuit (or a downstream component or software) is adapted to digitize the signals present on the receive conductor or antenna and perform a discrete Fourier transform (DFT) on the digitized information.
  • DFT discrete Fourier transform
  • the mixed signal circuit (or a downstream component or software) is adapted to digitize the signals present on the received conductor or antenna and perform a Fast Fourier transform (FFT) on the digitized information - an FFT being one type of discrete Fourier transform.
  • FFT Fast Fourier transform
  • a DFT treats the sequence of digital samples (e.g., window) taken during a sampling period (e.g., integration period) as though it repeats.
  • a sampling period e.g., integration period
  • signals that are not center frequencies i.e. , not integer multiples of the reciprocal of the integration period (which reciprocal defines the minimum frequency spacing)
  • the term orthogonal as used herein is not“violated” by such small contributions.
  • frequency orthogonal is used herein, two signals are considered frequency orthogonal if substantially all of the contribution of one signal to the DFT bins is made to different DFT bins than substantially all of the contribution of the other signal.
  • received signals are sampled at at least 1 MHz. In an embodiment, received signals are sampled at at least 2 MHz. In an embodiment, received signals are sampled at at least 4 Mhz. In an embodiment, received signals are sampled at 4.096 Mhz. In an embodiment, received signals are sampled at more than 4 MHz. To achieve kHz sampling, for example, 4096 samples may be taken at 4.096 MHz. In such an embodiment, the integration period is 1 millisecond, which per the constraint that the frequency spacing should be greater than or equal to the reciprocal of the integration period provides a minimum frequency spacing of 1 KHz.
  • the frequency spacing is equal to the reciprocal of the integration period.
  • the maximum frequency of a frequency-orthogonal signal range should be less than 2 MHz.
  • the practical maximum frequency of a frequency- orthogonal signal range should be less than about 40% of the sampling rate, or about 1 .6 MHz.
  • a DFT (which could be an FFT) is used to transform the digitized received signals into bins of information, each reflecting the frequency of a frequency-orthogonal signal transmitted which may have been transmitted by the transmitting antenna.
  • 2048 bins correspond to frequencies from 1 KHz to about 2 MHz. It will be apparent to a person of skill in the art in view of this disclosure that these examples are simply that, exemplary. Depending on the needs of a system, and subject to the constraints described above, the sample rate may be increased or decreased, the integration period may be adjusted, the frequency range may be adjusted, etc.
  • a DFT (which can be an FFT) output comprises a bin for each frequency orthogonal signal that is transmitted.
  • each DFT (which can be an FFT) bin comprises an in-phase (I) and quadrature (Q) component.
  • the sum of the squares of the I and Q components is used as measures corresponding to signal strength for that bin.
  • the square root of the sum of the squares of the I and Q components is used as measure corresponding to signal strength for that bin. It will be apparent to a person of skill in the art in view of this disclosure that a measure corresponding to the signal strength for a bin could be used as a measure related to activity, touch events, etc. In other words, the measure corresponding to signal strength in a given bin would change as a result of some activity proximate to the sensors, such as a touch event.
  • the sensing apparatuses discussed herein use transmitting and receiving antennas (also referred to herein as conductors, row conductors, column conductors, transmitting conductors, or receiving conductors). However, it should be understood that whether the transmitting antennas or receiving antennas are functioning as a transmitter, a receiver, or both depends on context and the embodiment.
  • the transmitters and receivers for all or any combination of the arrangements are operatively connected to a single integrated circuit capable of transmitting and receiving the required signals.
  • the transmitters and receivers are each operatively connected to a different integrated circuit capable of transmitting and receiving the required signals, respectively.
  • the transmitters and receivers for all or any combination of the patterns may be operatively connected to a group of integrated circuits, each capable of transmitting and receiving the required signals, and together sharing information necessary to such multiple 1C configuration.
  • the capacity of the integrated circuit i.e. , the number of transmit and receive channels
  • the requirements of the patterns i.e., the number of transmit and receive channels
  • all of the transmitters and receivers for all of the multiple patterns used by a controller are operated by a common integrated circuit, or by a group of integrated circuits that have communications therebetween.
  • the information from each circuit is combined in a separate system.
  • the separate system comprises a GPU and software for signal processing.
  • the mixed signal circuit is adapted to generate one or more signals and send the signals to the transmitting antennas via the transmitter. In an embodiment, the mixed signal circuit is adapted to generate a plurality of frequency orthogonal signals and send the plurality of frequency orthogonal signals to the transmitting antennas. In an embodiment, the mixed signal circuit is adapted to generate a plurality of frequency orthogonal signals and one or more of the plurality of frequency orthogonal signals to each of a plurality of transmit antennas. In an embodiment, the frequency orthogonal signals are in the range from DC up to about 2.5 GHz. In an embodiment, the frequency orthogonal signals are in the range from DC up to about 1.6 MHz. In an embodiment, the frequency orthogonal signals are in the range from 50 KHz to 200 KHz. The frequency spacing between the frequency orthogonal signals should be greater than or equal to the reciprocal of the integration period (i.e., the sampling period).
  • the mixed signal circuit (or a downstream component or software) is adapted to determine at least one value representing each frequency orthogonal signal transmitted by a transmitting antenna. In an embodiment, the mixed signal circuit (or a downstream component or software) performs a Fourier transform to the received signals. In an embodiment, the mixed signal circuit is adapted to digitize received signals. In an embodiment, the mixed signal circuit (or a downstream component or software) is adapted to digitize received signals and perform a discrete Fourier transform (DFT) on the digitized information. In an embodiment, the mixed signal circuit (or a downstream component or software) is adapted to digitize received signals and perform a Fast Fourier transform (FFT) on the digitized information.
  • DFT discrete Fourier transform
  • FFT Fast Fourier transform
  • aspects of the present invention relate to a sensing system comprising a plurality of different sensing modalities used to determine a movement or pose of a body part.
  • the sensing system 10 comprises a mixed signal circuit 100 in communication with a communications module 150 and plurality of sensor sub systems - described in further detail herein. It will be noted that for the sake of clarity the sensor systems described herein have been described as stand alone subsystems. However, in practice the entirety or a portion of these sensor systems may be implemented in the same integrated circuit.
  • the mixed signal circuit 100 comprises at least one of a processor, a microcontroller, memory, discrete electronic components, power management circuits, and communication modules (e.g., Wi-Fi, Bluetooth, NFC).
  • the sensing system 10 comprises a power supply (e.g., a battery) and associated power management circuitry both integrated and discrete.
  • the sensing system 10 comprises an inertial measurement unit sensor 200.
  • the inertial measurement unit (IMU) sensor 200 comprises at least one of an accelerometer 210 , a gyroscope, and a magnetometer.
  • the inertial measurement unit sensor 200 comprises a plurality of at least one of an accelerometer, a gyroscope, a magnetometer, and a combination thereof.
  • the inertial measurement unit sensor 200 comprises at least one haptic motor.
  • the sensing system 10 comprises a mechanical wave sensor 300.
  • the mechanical wave sensor 300 comprises devices adapted to transmit (e.g., speakers, haptic motors) 310 and receive (e.g., microphones, inertial measurement unit sensors) 320 mechanical waves.
  • the mechanical wave sensor 300 comprises a MEMS microphone.
  • the mechanical wave sensor 300 works by receiving the mechanical waves generated or transmitted by the user's body upon excitation of a body part. As will be noted by those skilled in the art different body parts have different resonant frequencies that are generated when those body parts are excited. Those signals generated by the different body parts then received by the mechanical wave sensor.
  • mechanical waves are transmitted (e.g., by using speakers or haptic motors) to the body and the propagated or reflected signal is received (e.g., by microphones or IMU sensors).
  • the sensing system 10 comprises a signal infusion sensor 400.
  • the signal infusion sensor 400 comprises transmitters 410 adapted to transmit a plurality of signals into the user.
  • the signal infusion sensor 400 further comprises receivers 420 adapted to receive the plurality of signal and to extrapolate information regarding a movement or a pose of a body part from the differences between the transmitted and the received signals.
  • the sensing system 10 comprises an outward sensing sensor 500.
  • outward-sensing sensor 500 comprises transmitters 510 adapted to transmit a plurality of signals which are then reflected by the body part and received by the receivers 520.
  • the outward sensing sensor 500 is further adapted to extrapolate information regarding a movement or a pose of a body part from the differences between the transmitted and the received signals.
  • the sensing system 10 uses the plurality of sensor sub-systems to create a comprehensive model of the body part being measured by fusing the data from each sensor.
  • the sensor modalities disclosed herein and incorporated by reference are complimentary.
  • each of the sensing modalities disclosed herein and incorporated by reference provides unique information that allows the sensing system 10 to determine specific characteristics of a movement or a pose effected by a body part.
  • one sensor can determine the start of an event, while another can determine the duration, force, etc.
  • one sensing modality may detect the initial interaction (e.g., using IMU sensors) between a digit and another body part thereby creating a time boundary in which another sensing modality (e.g., outward-sensing) determines a characteristic (e.g., force or dwell) to draw a conclusion about the interaction (e.g., whether it is a touch, a tap, or a pinch).
  • another sensing modality e.g., outward-sensing
  • determines a characteristic e.g., force or dwell
  • the wearable device 20 comprises a housing 22, at least one secondary housing 24, and a strap 26.
  • the wearable device 20 comprises only one housing.
  • the housing is capable of deforming around a body part of a user.
  • embodiments disclosed herein are not limited to any specific body part.
  • the wearable device may be secured around any body part (e.g., ankles, feet, arms, chest, legs, neck, waist, and hands).
  • FIGs. 4 and 5 illustrate an embodiment of a sensing system 10 incorporated into wearable 20.
  • portions of the IMU sensor 200 (not shown), the mechanical wave sensor 300 (not shown), and the signal infusion sensor 400, are housed in the secondary housing 24.
  • accelerometers of the IMU, MEMS microphones, and speakers sensor are located on the secondary housings 24 proximate to the radius and ulna bones.
  • the sensing system 10 comprises transmitters 410 and 510 located in secondary housing 24 proximate to the anterior of the wrist.
  • the sensing system 10 comprises receivers 420 and 520 located in the housing 22 proximate to the posterior of the wrist.
  • the sensing system 10 comprises at least one speaker 310, at least two microphones 320, and at least one accelerometer 210 located in a housing 24 proximate to the radius on one side and the ulna on the other side.
  • FIG. 7 shows a diagram of the musculature in the forearm of a person.
  • the placement of sensors around the wrist is determined by how a particular movement of the hand and fingers manifests in the associated anatomy.
  • resonance from the excitation of the bones, muscles, and other anatomy propagates to other parts of the anatomy.
  • excitation of the fingers e.g., by tapping, pinching, or touching
  • FIG. 8 shows a pinch between the index finger and the thumb being detected by the sensing system 10.
  • FIG. 9 shows a touch between the index finger and the thumb being determined by the sensing system 10.
  • the placement of the sensors proximate to those muscles that govern the activity of pinch and touch has been determined to be effective for detecting the internal movements within the wrist area that can be correlated to pinching, tapping, and touching.
  • the movement and position of physical structure of bones, tendons, veins, arteries, etc. within the wrist area are leveraged by the sensing system 10 to determine the motion of the fingers and determine other hand related behaviors.
  • the placement of the sensing system 10 to correlate with musculature, bone, tendon and/or ligament activity that determine other activities of the hands also facilitates such determinations.
  • FIG. 10 shows a touch of a table being determined by the sensing system 10.
  • FIG. 1 1 shows a touch of a baseball being determined by the sensing system 10.
  • the touch event is being determined by contact of the finger with the surface of an object and resultant impact that the touch event has on the underlying physical structure within the wrist area.
  • the sensing system is implemented in a wearable placed on the ankle.
  • the placement of the sensing system’s transmitting antennas and receiving antennas to correlate with musculature, bone, tendon and/or ligament activity that determine activities of the foot provides enhanced measurements of the foot activity.
  • the sensing system is implemented in a wearable placed on the arm.
  • the placement of the sensing system’s transmitting antennas and receiving antennas to correlate with musculature, bone, tendon and/or ligament activity that determine activities associated with the arm provides enhanced measurements of arm activity.
  • the sensing system is implemented in a sensing device placed on the chest.
  • the sensing system transmitting antennas and receiving antennas to correlate with musculature, bone, tendon and/or ligament activity that determine activities associated with the chest (e.g., breathing, heart rate, etc.) provides enhanced measurements of the associated chest activity.
  • the pressure adaptive sensor system is implemented in a wearable placed on the leg.
  • the placement of the sensing system’s transmitting antennas and receiving antennas to correlate with musculature, bone, tendon and/or ligament activity that determine activities associated with the leg provides enhanced measurements of leg activity.
  • the sensing system is implemented in a wearable placed on the head.
  • the placement of the sensing system’s transmitting antennas and receiving antennas to correlate with musculature, bone, tendon and/or ligament activity that determine activities associated with the head provides enhanced measurements of facial activity and head motion.
  • the sensing system is implemented in a wearable placed on the neck.
  • the placement of the sensing system’s transmitting antennas and receiving antennas to correlate with musculature, bone, tendon and/or ligament activity that determine activities associated with the neck provides enhanced measurements of vocalization, breathing, and other associated activities.
  • the pressure adaptive sensing system is implemented in a wearable placed on the waist.
  • the placement of the sensing system’s transmitting antennas and receiving antennas to correlate with musculature, bone, tendon and/or ligament activity that determine activities associated with the waist provide enhanced determination of movement and other associated activities.
  • the sensing system is implemented in a wearable placed on the hand.
  • the placement of the sensing system’s transmitting antennas and receiving antennas to correlate with musculature, bone, tendon and/or ligament activity that determine activities associated with the hand provides enhanced determination of fine hand movement.
  • the sensing system is implemented in a wearable placed on the foot.
  • the placement of the sensing system’s transmitting antennas and receiving antennas to correlate with musculature, bone, tendon and/or ligament activity that determine activities associated with the foot provides enhanced determination of fine foot movement.
  • An aspect of the disclosure is a multimodal sensing system.
  • the multimodal sensing system comprising a plurality of sensors operably located within a housing, wherein the housing is adapted to be placed proximate to a body part, the plurality of sensors adapted to receive a plurality of signals related to at least one of a movement and a pose of the body part; and, a processor operably connected to the plurality of sensors, wherein the processor is adapted to receive information from each of the plurality of sensors, and to process the information to determine a type of the at least one of a movement and a pose and at least one characteristic of the type of the at least one of a movement and a pose.
  • Another aspect of the disclosure is a method for sensing movement of a body part using a multimodal sensing system.
  • the method comprising placing a housing proximate to a body part, wherein operably located within the housing is a plurality of sensors and a processor, wherein the processor is operably connected to the plurality of sensors; receiving a plurality of signals related to at least one of a movement and a pose of the body part using the plurality of sensors; processing using the processor at least one of the plurality of signals; and, determining a type of the at least one of a movement and a pose and at least one characteristic of the type of the at least one of a movement and a pose.

Landscapes

  • Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Heart & Thoracic Surgery (AREA)
  • Surgery (AREA)
  • Biophysics (AREA)
  • Pathology (AREA)
  • Veterinary Medicine (AREA)
  • Biomedical Technology (AREA)
  • Public Health (AREA)
  • Medical Informatics (AREA)
  • Molecular Biology (AREA)
  • General Health & Medical Sciences (AREA)
  • Animal Behavior & Ethology (AREA)
  • Oral & Maxillofacial Surgery (AREA)
  • Physiology (AREA)
  • Dentistry (AREA)
  • General Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Human Computer Interaction (AREA)
  • General Physics & Mathematics (AREA)
  • Measurement Of The Respiration, Hearing Ability, Form, And Blood Characteristics Of Living Organisms (AREA)

Abstract

Un système de détection multimodal comprend une pluralité de capteurs placés à proximité d'une partie du corps Le système de détection reçoit, à l'aide d'une pluralité de capteurs, une pluralité de signaux associés à un mouvement et/ou une pose de la partie du corps Le système de détection extrapole ensuite des informations concernant le type de mouvement ou de pose et au moins une caractéristique du type de mouvement ou de pose.
PCT/US2020/040009 2019-06-26 2020-06-26 Dispositif de détection multimodal portable WO2020264443A1 (fr)

Applications Claiming Priority (4)

Application Number Priority Date Filing Date Title
US201962867006P 2019-06-26 2019-06-26
US62/867,006 2019-06-26
US16/914,258 US11759148B2 (en) 2019-05-22 2020-06-26 Wearable multimodal-sensing device
US16/914,258 2020-06-26

Publications (1)

Publication Number Publication Date
WO2020264443A1 true WO2020264443A1 (fr) 2020-12-30

Family

ID=74061419

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US2020/040009 WO2020264443A1 (fr) 2019-06-26 2020-06-26 Dispositif de détection multimodal portable

Country Status (2)

Country Link
TW (1) TW202118443A (fr)
WO (1) WO2020264443A1 (fr)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2024092368A1 (fr) * 2022-11-04 2024-05-10 Texavie Technologies Inc. Dispositif portable de capteur textile étirable pour suivre une ou plusieurs métriques corporelles

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20150070245A1 (en) * 2012-03-16 2015-03-12 City University Of Hong Kong Coil-based artificial atom for metamaterials, metamaterial comprising the artificial atom, and device comprising the metamaterial
US20150124566A1 (en) * 2013-10-04 2015-05-07 Thalmic Labs Inc. Systems, articles and methods for wearable electronic devices employing contact sensors
US20170075426A1 (en) * 2015-09-16 2017-03-16 Intel Corporation Techniques for gesture recognition using photoplethysmographic (ppmg) sensor and low-power wearable gesture recognition device using the same
US20180253151A1 (en) * 2017-03-02 2018-09-06 Samsung Electronics Co., Ltd. Method for recognizing a gesture and an electronic device thereof
US20190050082A1 (en) * 2017-03-20 2019-02-14 Tactual Labs Co. Biometric sensor

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20150070245A1 (en) * 2012-03-16 2015-03-12 City University Of Hong Kong Coil-based artificial atom for metamaterials, metamaterial comprising the artificial atom, and device comprising the metamaterial
US20150124566A1 (en) * 2013-10-04 2015-05-07 Thalmic Labs Inc. Systems, articles and methods for wearable electronic devices employing contact sensors
US20170075426A1 (en) * 2015-09-16 2017-03-16 Intel Corporation Techniques for gesture recognition using photoplethysmographic (ppmg) sensor and low-power wearable gesture recognition device using the same
US20180253151A1 (en) * 2017-03-02 2018-09-06 Samsung Electronics Co., Ltd. Method for recognizing a gesture and an electronic device thereof
US20190050082A1 (en) * 2017-03-20 2019-02-14 Tactual Labs Co. Biometric sensor

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2024092368A1 (fr) * 2022-11-04 2024-05-10 Texavie Technologies Inc. Dispositif portable de capteur textile étirable pour suivre une ou plusieurs métriques corporelles

Also Published As

Publication number Publication date
TW202118443A (zh) 2021-05-16

Similar Documents

Publication Publication Date Title
US11045117B2 (en) Systems and methods for determining axial orientation and location of a user's wrist
EP2661663B1 (fr) Procédé et appareil destinés au suivi de l'orientation d'un utilisateur
CN109313493A (zh) 用于基于手的运动和位置控制计算机的装置
US20130135223A1 (en) Finger-worn input devices and methods of use
US11036293B2 (en) Method for using fingers to interact with a smart glove worn on a hand
CN106705989B (zh) 记步方法、设备及终端
JP2014522258A (ja) モーションキャプチャのための近接センサーメッシュ
CN106662898A (zh) 利用超声的模式身体触摸
WO2018039360A1 (fr) Objets tactiles
US11759148B2 (en) Wearable multimodal-sensing device
Truong et al. Capacitive sensing 3d-printed wristband for enriched hand gesture recognition
US20190129508A1 (en) Method and System for Interacting with a Wearable Electronic Device
Hsiao et al. Tactile teacher: Sensing finger tapping in piano playing
Chen et al. A low latency on-body typing system through single vibration sensor
WO2020264443A1 (fr) Dispositif de détection multimodal portable
Saggio et al. A novel actuating–sensing bone conduction-based system for active hand pose sensing and material densities evaluation through hand touch
US11686835B2 (en) Positioned enhanced sensor detection
US20180364814A1 (en) Six degrees of freedom tracking of objects using sensors
Funato et al. Grip force estimation by emitting vibration
Morelli et al. Twuist: A discrete tactile-proprioceptive display for eye and ear free output on mobile devices
Kim et al. VibAware: Context-Aware Tap and Swipe Gestures Using Bio-Acoustic Sensing
CN107111282A (zh) 基于超声的多身体上动作检测
US20210100482A1 (en) Capactive based mechanomyography
WO2024139642A1 (fr) Procédé et appareil de détection de fréquence cardiaque, dispositif électronique et support d'enregistrement
Salemi Parizi Precise Inside-Out Electromagnetic Input Tracking

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20831047

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20831047

Country of ref document: EP

Kind code of ref document: A1