KR20240024095A - 신경 활동으로부터의 실시간 단어 및 음성 디코딩을 위한 방법들 및 디바이스들 - Google Patents

신경 활동으로부터의 실시간 단어 및 음성 디코딩을 위한 방법들 및 디바이스들 Download PDF

Info

Publication number
KR20240024095A
KR20240024095A KR1020237043726A KR20237043726A KR20240024095A KR 20240024095 A KR20240024095 A KR 20240024095A KR 1020237043726 A KR1020237043726 A KR 1020237043726A KR 20237043726 A KR20237043726 A KR 20237043726A KR 20240024095 A KR20240024095 A KR 20240024095A
Authority
KR
South Korea
Prior art keywords
attempted
word
subject
speech
electrical signal
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
KR1020237043726A
Other languages
English (en)
Korean (ko)
Inventor
데이비드 모세
제시 리우
션 메츠거
에드워드 창
Original Assignee
더 리전츠 오브 더 유니버시티 오브 캘리포니아
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 더 리전츠 오브 더 유니버시티 오브 캘리포니아 filed Critical 더 리전츠 오브 더 유니버시티 오브 캘리포니아
Publication of KR20240024095A publication Critical patent/KR20240024095A/ko
Pending legal-status Critical Current

Links

Classifications

    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/72—Signal processing specially adapted for physiological signals or for diagnostic purposes
    • A61B5/7235—Details of waveform analysis
    • A61B5/7264—Classification of physiological signals or data, e.g. using neural networks, statistical classifiers, expert systems or fuzzy systems
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
    • G06F3/015—Input arrangements based on nervous system activity detection, e.g. brain waves [EEG] detection, electromyograms [EMG] detection, electrodermal response detection
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/24—Detecting, measuring or recording bioelectric or biomagnetic signals of the body or parts thereof
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/24—Detecting, measuring or recording bioelectric or biomagnetic signals of the body or parts thereof
    • A61B5/25—Bioelectric electrodes therefor
    • A61B5/279—Bioelectric electrodes therefor specially adapted for particular uses
    • A61B5/291—Bioelectric electrodes therefor specially adapted for particular uses for electroencephalography [EEG]
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/24—Detecting, measuring or recording bioelectric or biomagnetic signals of the body or parts thereof
    • A61B5/25—Bioelectric electrodes therefor
    • A61B5/279—Bioelectric electrodes therefor specially adapted for particular uses
    • A61B5/291—Bioelectric electrodes therefor specially adapted for particular uses for electroencephalography [EEG]
    • A61B5/293—Invasive
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/24—Detecting, measuring or recording bioelectric or biomagnetic signals of the body or parts thereof
    • A61B5/316—Modalities, i.e. specific diagnostic methods
    • A61B5/369—Electroencephalography [EEG]
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/24—Detecting, measuring or recording bioelectric or biomagnetic signals of the body or parts thereof
    • A61B5/316—Modalities, i.e. specific diagnostic methods
    • A61B5/369—Electroencephalography [EEG]
    • A61B5/37—Intracranial electroencephalography [IC-EEG], e.g. electrocorticography [ECoG]
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/24—Detecting, measuring or recording bioelectric or biomagnetic signals of the body or parts thereof
    • A61B5/316—Modalities, i.e. specific diagnostic methods
    • A61B5/369—Electroencephalography [EEG]
    • A61B5/372—Analysis of electroencephalograms
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/40—Detecting, measuring or recording for evaluating the nervous system
    • A61B5/4058—Detecting, measuring or recording for evaluating the nervous system for evaluating the central nervous system
    • A61B5/4064—Evaluating the brain
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/48—Other medical applications
    • A61B5/4803—Speech analysis specially adapted for diagnostic purposes
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/72—Signal processing specially adapted for physiological signals or for diagnostic purposes
    • A61B5/7235—Details of waveform analysis
    • A61B5/7264—Classification of physiological signals or data, e.g. using neural networks, statistical classifiers, expert systems or fuzzy systems
    • A61B5/7267—Classification of physiological signals or data, e.g. using neural networks, statistical classifiers, expert systems or fuzzy systems involving training the classification device
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/72—Signal processing specially adapted for physiological signals or for diagnostic purposes
    • A61B5/7271—Specific aspects of physiological measurement analysis
    • A61B5/7278—Artificial waveform generation or derivation, e.g. synthesizing signals from measured signals
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/74—Details of notification to user or communication with user or patient; User input means
    • A61B5/7405—Details of notification to user or communication with user or patient; User input means using sound
    • A61B5/741—Details of notification to user or communication with user or patient; User input means using sound using synthesised speech
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/74—Details of notification to user or communication with user or patient; User input means
    • A61B5/742—Details of notification to user or communication with user or patient; User input means using visual displays
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L13/00—Speech synthesis; Text to speech systems
    • G10L13/08—Text analysis or generation of parameters for speech synthesis out of text, e.g. grapheme to phoneme translation, prosody generation or stress or intonation determination
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L15/00—Speech recognition
    • G10L15/24—Speech recognition using non-acoustical features

Landscapes

  • Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Biomedical Technology (AREA)
  • General Health & Medical Sciences (AREA)
  • Public Health (AREA)
  • Surgery (AREA)
  • Molecular Biology (AREA)
  • Animal Behavior & Ethology (AREA)
  • Medical Informatics (AREA)
  • Heart & Thoracic Surgery (AREA)
  • Biophysics (AREA)
  • Pathology (AREA)
  • Veterinary Medicine (AREA)
  • Psychiatry (AREA)
  • Artificial Intelligence (AREA)
  • Psychology (AREA)
  • Physiology (AREA)
  • Neurosurgery (AREA)
  • Signal Processing (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Mathematical Physics (AREA)
  • Evolutionary Computation (AREA)
  • Fuzzy Systems (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • Neurology (AREA)
  • Human Computer Interaction (AREA)
  • Theoretical Computer Science (AREA)
  • Acoustics & Sound (AREA)
  • Multimedia (AREA)
  • Computational Linguistics (AREA)
  • General Engineering & Computer Science (AREA)
  • Dermatology (AREA)
  • General Physics & Mathematics (AREA)
  • Machine Translation (AREA)
  • Measurement And Recording Of Electrical Phenomena And Electrical Characteristics Of The Living Body (AREA)
  • Compression, Expansion, Code Conversion, And Decoders (AREA)
  • Electrically Operated Instructional Devices (AREA)
KR1020237043726A 2021-05-26 2022-05-26 신경 활동으로부터의 실시간 단어 및 음성 디코딩을 위한 방법들 및 디바이스들 Pending KR20240024095A (ko)

Applications Claiming Priority (3)

Application Number Priority Date Filing Date Title
US202163193351P 2021-05-26 2021-05-26
US63/193,351 2021-05-26
PCT/US2022/031101 WO2022251472A1 (en) 2021-05-26 2022-05-26 Methods and devices for real-time word and speech decoding from neural activity

Publications (1)

Publication Number Publication Date
KR20240024095A true KR20240024095A (ko) 2024-02-23

Family

ID=84229189

Family Applications (1)

Application Number Title Priority Date Filing Date
KR1020237043726A Pending KR20240024095A (ko) 2021-05-26 2022-05-26 신경 활동으로부터의 실시간 단어 및 음성 디코딩을 위한 방법들 및 디바이스들

Country Status (8)

Country Link
US (1) US20240366157A1 (https=)
EP (1) EP4329615A4 (https=)
JP (1) JP2024521768A (https=)
KR (1) KR20240024095A (https=)
CN (1) CN117693315A (https=)
AU (1) AU2022282378A1 (https=)
CA (1) CA3220064A1 (https=)
WO (1) WO2022251472A1 (https=)

Families Citing this family (27)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11790169B2 (en) * 2021-04-02 2023-10-17 Salesforce, Inc. Methods and systems of answering frequently asked questions (FAQs)
EP4569511A4 (en) * 2022-08-09 2026-04-15 Univ Leland Stanford Junior SYSTEMS AND METHODS FOR DECODING SPEECH FROM NEURAL ACTIVITY
US12530535B2 (en) * 2022-10-31 2026-01-20 Zoom Communications, Inc. Intelligent prediction of next step sentences from a communication session
CN116206150A (zh) * 2023-01-09 2023-06-02 阿里巴巴（中国）有限公司 任务处理方法、地物分类方法及任务模型训练方法
US20240264670A1 (en) * 2023-02-03 2024-08-08 Georgia Tech Research Corporation Systems and Methods for Determining the Coupling Response of a Non-Linear Variant System
CN116225222A (zh) * 2023-02-26 2023-06-06 北京航空航天大学 基于轻量级梯度提升决策树的脑机交互意图识别方法及系统
US20240398317A1 (en) * 2023-06-05 2024-12-05 Northwestern University Method and system to decode speech production from non-frontal, non-post-central brain cortices
EP4704703A1 (en) * 2023-06-06 2026-03-11 The Regents of University of California Methods and systems for translation of neural activity into embodied digital-avatar animation
US12469507B2 (en) 2023-06-14 2025-11-11 Microsoft Technology Licensing, Llc Predictive context-based decoder correction
US20240419895A1 (en) * 2023-06-14 2024-12-19 Microsoft Technology Licensing, Llc Context-based decoder correction
WO2025076530A1 (en) * 2023-10-06 2025-04-10 Precision Neuroscience Corporation Systems and methods for visualizing brain activity in real time at high spatial and temporal resolution
CN117058514B (zh) * 2023-10-12 2024-04-02 之江实验室 基于图神经网络的多模态脑影像数据融合解码方法和装置
WO2025080841A1 (en) 2023-10-12 2025-04-17 The Regents Of The University Of California Methods for inside-out deployment of microelectrode arrays and devices and systems for same
CN117130490B (zh) * 2023-10-26 2024-01-26 天津大学 一种脑机接口控制系统及其控制方法和实现方法
CN117131426B (zh) * 2023-10-26 2024-01-19 一网互通(北京)科技有限公司 基于预训练的品牌识别方法、装置及电子设备
CN117238277B (zh) * 2023-11-09 2024-01-19 北京水滴科技集团有限公司 意图识别方法、装置、存储介质及计算机设备
CN117851769B (zh) * 2023-11-30 2024-06-21 浙江大学 一种面向侵入式脑机接口的汉字书写解码方法
US20250218434A1 (en) * 2023-12-29 2025-07-03 Cx360, Inc. Automated prompt finder
CN117708546B (zh) * 2024-02-05 2024-05-10 北京智冉医疗科技有限公司 基于侵入式脑机接口的高通量神经信号的解码方法及装置
US12588859B2 (en) * 2024-03-21 2026-03-31 The Education University Of Hong Kong System and method for interacting with human brain activities using EEG-fNIRS neurofeedback
CN118095447B (zh) * 2024-04-12 2024-06-25 清华大学 大语言模型分布式推理方法及装置、介质
CN118095295B (zh) * 2024-04-28 2024-07-09 昆明理工大学 渐进式预训练和提示增强低资源语言的跨语言摘要方法
CN118766414B (zh) * 2024-04-30 2025-05-09 中国科学院心理研究所 一种基于跨模态脑功能连接图谱的书写能力脑指纹构建方法及系统
WO2025235580A1 (en) * 2024-05-08 2025-11-13 The Regents Of The University Of California Systems and methods for decoding biosignals of a person indicative of speech
CN119402342B (zh) * 2024-09-24 2026-04-17 中国南方电网有限责任公司 电力通信网的故障定位方法、系统及电子设备
CN119691418B (zh) * 2024-11-26 2025-11-14 神州数码(中国)有限公司 模型评估方法、装置、电子设备和计算机可读存储介质
CN121096350A (zh) * 2025-09-03 2025-12-09 哈尔滨工业大学 一种基于脑电信号的语音解码方法、系统、设备及介质

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2014102722A1 (en) * 2012-12-26 2014-07-03 Sia Technology Ltd. Device, system, and method of controlling electronic devices via thought
US10130809B2 (en) * 2014-06-13 2018-11-20 Nervana, LLC Transcutaneous electrostimulator and methods for electric stimulation
US12008987B2 (en) * 2018-04-30 2024-06-11 The Board Of Trustees Of The Leland Stanford Junior University Systems and methods for decoding intended speech from neuronal activity
WO2021021714A1 (en) * 2019-07-29 2021-02-04 The Regents Of The University Of California Method of contextual speech decoding from the brain

Also Published As

Publication number Publication date
WO2022251472A1 (en) 2022-12-01
CA3220064A1 (en) 2022-12-01
CN117693315A (zh) 2024-03-12
AU2022282378A1 (en) 2023-12-14
JP2024521768A (ja) 2024-06-04
US20240366157A1 (en) 2024-11-07
WO2022251472A9 (en) 2023-11-09
EP4329615A1 (en) 2024-03-06
EP4329615A4 (en) 2025-01-01

Similar Documents

Publication Publication Date Title
KR20240024095A (ko) 신경 활동으로부터의 실시간 단어 및 음성 디코딩을 위한 방법들 및 디바이스들
Metzger et al. Generalizable spelling using a speech neuroprosthesis in an individual with severe limb and vocal paralysis
Moses et al. Neuroprosthesis for decoding speech in a paralyzed person with anarthria
Dash et al. Decoding imagined and spoken phrases from non-invasive neural (MEG) signals
US20260060621A1 (en) Detection of disease conditions and comorbidities
Willett et al. A high-performance speech neuroprosthesis
Moses et al. Real-time decoding of question-and-answer speech dialogue using human cortical activity
Makin et al. Machine translation of cortical activity to text with an encoder–decoder framework
Kunz et al. Inner speech in motor cortex and implications for speech neuroprostheses
WO2024254360A1 (en) Methods and systems for translation of neural activity into embodied digital-avatar animation
Dash et al. NeuroVAD: Real-time voice activity detection from non-invasive neuromagnetic signals
Stavisky Restoring speech using brain–computer interfaces
Wu et al. Adaptive LDA classifier enhances real-time control of an EEG brain–computer interface for decoding imagined syllables
Feng et al. Acoustic inspired brain-to-sentence decoder for logosyllabic language
Tan et al. Effective phoneme decoding with hyperbolic neural networks for high-performance speech BCIs
Jude et al. Restoring rapid natural bimanual typing with a neuroprosthesis after paralysis
Hou et al. Error encoding in human speech motor cortex
Ham et al. Vowel speech recognition from rat electroencephalography using long short-term memory neural network
Salomons et al. Frame-Based Phone Classification Using EMG Signals
Kohlberg et al. Development of a low-cost, noninvasive, portable visual speech recognition program
Metzger AI-Driven Brain-Computer Interfaces for Speech
US12168140B1 (en) Apparatus for neural stimulation and method of use
Wilson Algorithms for High-Performance Brain-Computer Interfaces
Roussel Analysis of cortical activity for the development of brain-computer interfaces for speech
Araya Improving Brain-to-Text Models Utilizing Recurrent Neural Networks and Transfer Learning for Neuroprosthetic Speech

Legal Events

Date Code Title Description
PA0105 International application

St.27 status event code: A-0-1-A10-A15-nap-PA0105

PG1501 Laying open of application

St.27 status event code: A-1-1-Q10-Q12-nap-PG1501

A201 Request for examination
PE0902 Notice of grounds for rejection

St.27 status event code: A-1-2-D10-D21-exm-PE0902