JP2024521768A - 神経活動からのリアルタイム単語及び発話復号のための方法及びデバイス - Google Patents

神経活動からのリアルタイム単語及び発話復号のための方法及びデバイス Download PDF

Info

Publication number
JP2024521768A
JP2024521768A JP2023572722A JP2023572722A JP2024521768A JP 2024521768 A JP2024521768 A JP 2024521768A JP 2023572722 A JP2023572722 A JP 2023572722A JP 2023572722 A JP2023572722 A JP 2023572722A JP 2024521768 A JP2024521768 A JP 2024521768A
Authority
JP
Japan
Prior art keywords
word
subject
signal data
trial
speech
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
JP2023572722A
Other languages
English (en)
Japanese (ja)
Other versions
JP2024521768A5 (enExample
Inventor
モーゼス，デヴィッド
リュウ，ジェシー
メッツガー，ショーン
チャン，エドワード
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
University of California, San Diego
Original Assignee
University of California, San Diego
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by University of California, San Diego filed Critical University of California, San Diego
Publication of JP2024521768A publication Critical patent/JP2024521768A/ja
Publication of JP2024521768A5 publication Critical patent/JP2024521768A5/ja
Pending legal-status Critical Current

Links

Classifications

    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/72—Signal processing specially adapted for physiological signals or for diagnostic purposes
    • A61B5/7235—Details of waveform analysis
    • A61B5/7264—Classification of physiological signals or data, e.g. using neural networks, statistical classifiers, expert systems or fuzzy systems
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
    • G06F3/015—Input arrangements based on nervous system activity detection, e.g. brain waves [EEG] detection, electromyograms [EMG] detection, electrodermal response detection
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/24—Detecting, measuring or recording bioelectric or biomagnetic signals of the body or parts thereof
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/24—Detecting, measuring or recording bioelectric or biomagnetic signals of the body or parts thereof
    • A61B5/25—Bioelectric electrodes therefor
    • A61B5/279—Bioelectric electrodes therefor specially adapted for particular uses
    • A61B5/291—Bioelectric electrodes therefor specially adapted for particular uses for electroencephalography [EEG]
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/24—Detecting, measuring or recording bioelectric or biomagnetic signals of the body or parts thereof
    • A61B5/25—Bioelectric electrodes therefor
    • A61B5/279—Bioelectric electrodes therefor specially adapted for particular uses
    • A61B5/291—Bioelectric electrodes therefor specially adapted for particular uses for electroencephalography [EEG]
    • A61B5/293—Invasive
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/24—Detecting, measuring or recording bioelectric or biomagnetic signals of the body or parts thereof
    • A61B5/316—Modalities, i.e. specific diagnostic methods
    • A61B5/369—Electroencephalography [EEG]
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/24—Detecting, measuring or recording bioelectric or biomagnetic signals of the body or parts thereof
    • A61B5/316—Modalities, i.e. specific diagnostic methods
    • A61B5/369—Electroencephalography [EEG]
    • A61B5/37—Intracranial electroencephalography [IC-EEG], e.g. electrocorticography [ECoG]
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/24—Detecting, measuring or recording bioelectric or biomagnetic signals of the body or parts thereof
    • A61B5/316—Modalities, i.e. specific diagnostic methods
    • A61B5/369—Electroencephalography [EEG]
    • A61B5/372—Analysis of electroencephalograms
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/40—Detecting, measuring or recording for evaluating the nervous system
    • A61B5/4058—Detecting, measuring or recording for evaluating the nervous system for evaluating the central nervous system
    • A61B5/4064—Evaluating the brain
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/48—Other medical applications
    • A61B5/4803—Speech analysis specially adapted for diagnostic purposes
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/72—Signal processing specially adapted for physiological signals or for diagnostic purposes
    • A61B5/7235—Details of waveform analysis
    • A61B5/7264—Classification of physiological signals or data, e.g. using neural networks, statistical classifiers, expert systems or fuzzy systems
    • A61B5/7267—Classification of physiological signals or data, e.g. using neural networks, statistical classifiers, expert systems or fuzzy systems involving training the classification device
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/72—Signal processing specially adapted for physiological signals or for diagnostic purposes
    • A61B5/7271—Specific aspects of physiological measurement analysis
    • A61B5/7278—Artificial waveform generation or derivation, e.g. synthesizing signals from measured signals
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/74—Details of notification to user or communication with user or patient; User input means
    • A61B5/7405—Details of notification to user or communication with user or patient; User input means using sound
    • A61B5/741—Details of notification to user or communication with user or patient; User input means using sound using synthesised speech
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/74—Details of notification to user or communication with user or patient; User input means
    • A61B5/742—Details of notification to user or communication with user or patient; User input means using visual displays
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L13/00—Speech synthesis; Text to speech systems
    • G10L13/08—Text analysis or generation of parameters for speech synthesis out of text, e.g. grapheme to phoneme translation, prosody generation or stress or intonation determination
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L15/00—Speech recognition
    • G10L15/24—Speech recognition using non-acoustical features

Landscapes

  • Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Health & Medical Sciences (AREA)
  • Biomedical Technology (AREA)
  • Heart & Thoracic Surgery (AREA)
  • Pathology (AREA)
  • Biophysics (AREA)
  • Medical Informatics (AREA)
  • Molecular Biology (AREA)
  • Surgery (AREA)
  • Animal Behavior & Ethology (AREA)
  • Public Health (AREA)
  • Veterinary Medicine (AREA)
  • Psychiatry (AREA)
  • Artificial Intelligence (AREA)
  • Psychology (AREA)
  • Physiology (AREA)
  • Neurosurgery (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Signal Processing (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • Evolutionary Computation (AREA)
  • Fuzzy Systems (AREA)
  • Mathematical Physics (AREA)
  • Neurology (AREA)
  • Human Computer Interaction (AREA)
  • Multimedia (AREA)
  • Acoustics & Sound (AREA)
  • Computational Linguistics (AREA)
  • General Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Dermatology (AREA)
  • General Physics & Mathematics (AREA)
  • Machine Translation (AREA)
  • Measurement And Recording Of Electrical Phenomena And Electrical Characteristics Of The Living Body (AREA)
  • Electrically Operated Instructional Devices (AREA)
  • Compression, Expansion, Code Conversion, And Decoders (AREA)
JP2023572722A 2021-05-26 2022-05-26 神経活動からのリアルタイム単語及び発話復号のための方法及びデバイス Pending JP2024521768A (ja)

Applications Claiming Priority (3)

Application Number Priority Date Filing Date Title
US202163193351P 2021-05-26 2021-05-26
US63/193,351 2021-05-26
PCT/US2022/031101 WO2022251472A1 (en) 2021-05-26 2022-05-26 Methods and devices for real-time word and speech decoding from neural activity

Publications (2)

Publication Number Publication Date
JP2024521768A true JP2024521768A (ja) 2024-06-04
JP2024521768A5 JP2024521768A5 (enExample) 2025-05-30

Family

ID=84229189

Family Applications (1)

Application Number Title Priority Date Filing Date
JP2023572722A Pending JP2024521768A (ja) 2021-05-26 2022-05-26 神経活動からのリアルタイム単語及び発話復号のための方法及びデバイス

Country Status (8)

Country Link
US (1) US20240366157A1 (enExample)
EP (1) EP4329615A4 (enExample)
JP (1) JP2024521768A (enExample)
KR (1) KR20240024095A (enExample)
CN (1) CN117693315A (enExample)
AU (1) AU2022282378A1 (enExample)
CA (1) CA3220064A1 (enExample)
WO (1) WO2022251472A1 (enExample)

Families Citing this family (30)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11790169B2 (en) * 2021-04-02 2023-10-17 Salesforce, Inc. Methods and systems of answering frequently asked questions (FAQs)
WO2024036213A1 (en) * 2022-08-09 2024-02-15 The Board Of Trustees Of The Leland Stanford Junior University Systems and methods for decoding speech from neural activity
US20240231491A9 (en) * 2022-10-24 2024-07-11 Precision Neuroscience Corporation Data-efficient transfer learning for neural decoding applications
US12530535B2 (en) * 2022-10-31 2026-01-20 Zoom Communications, Inc. Intelligent prediction of next step sentences from a communication session
CN116206150A (zh) * 2023-01-09 2023-06-02 阿里巴巴（中国）有限公司 任务处理方法、地物分类方法及任务模型训练方法
US12663863B2 (en) * 2023-02-03 2026-06-23 Georgia Tech Research Corporation Systems and methods for determining the coupling response of a non-linear variant system
CN116225222A (zh) * 2023-02-26 2023-06-06 北京航空航天大学 基于轻量级梯度提升决策树的脑机交互意图识别方法及系统
US20240398317A1 (en) * 2023-06-05 2024-12-05 Northwestern University Method and system to decode speech production from non-frontal, non-post-central brain cortices
WO2024254360A1 (en) * 2023-06-06 2024-12-12 The Regents Of The University Of California Methods and systems for translation of neural activity into embodied digital-avatar animation
US12469507B2 (en) 2023-06-14 2025-11-11 Microsoft Technology Licensing, Llc Predictive context-based decoder correction
US12657384B2 (en) * 2023-06-14 2026-06-16 Microsoft Technology Licensing, Llc Context-based decoder correction
US20250021653A1 (en) * 2023-07-13 2025-01-16 Robi Sen Defenses for Large Language Models
EP4788212A1 (en) * 2023-10-06 2026-08-12 Precision Neuroscience Corporation Systems and methods for visualizing brain activity in real time at high spatial and temporal resolution
CN117058514B (zh) * 2023-10-12 2024-04-02 之江实验室 基于图神经网络的多模态脑影像数据融合解码方法和装置
WO2025080841A1 (en) 2023-10-12 2025-04-17 The Regents Of The University Of California Methods for inside-out deployment of microelectrode arrays and devices and systems for same
CN117131426B (zh) * 2023-10-26 2024-01-19 一网互通(北京)科技有限公司 基于预训练的品牌识别方法、装置及电子设备
CN117130490B (zh) 2023-10-26 2024-01-26 天津大学 一种脑机接口控制系统及其控制方法和实现方法
CN117238277B (zh) * 2023-11-09 2024-01-19 北京水滴科技集团有限公司 意图识别方法、装置、存储介质及计算机设备
CN117851769B (zh) * 2023-11-30 2024-06-21 浙江大学 一种面向侵入式脑机接口的汉字书写解码方法
US20250218434A1 (en) * 2023-12-29 2025-07-03 Cx360, Inc. Automated prompt finder
CN117708546B (zh) * 2024-02-05 2024-05-10 北京智冉医疗科技有限公司 基于侵入式脑机接口的高通量神经信号的解码方法及装置
US12588859B2 (en) * 2024-03-21 2026-03-31 The Education University Of Hong Kong System and method for interacting with human brain activities using EEG-fNIRS neurofeedback
CN118095447B (zh) * 2024-04-12 2024-06-25 清华大学 大语言模型分布式推理方法及装置、介质
CN118095295B (zh) * 2024-04-28 2024-07-09 昆明理工大学 渐进式预训练和提示增强低资源语言的跨语言摘要方法
CN118766414B (zh) * 2024-04-30 2025-05-09 中国科学院心理研究所 一种基于跨模态脑功能连接图谱的书写能力脑指纹构建方法及系统
WO2025235580A1 (en) * 2024-05-08 2025-11-13 The Regents Of The University Of California Systems and methods for decoding biosignals of a person indicative of speech
CN119402342B (zh) * 2024-09-24 2026-04-17 中国南方电网有限责任公司 电力通信网的故障定位方法、系统及电子设备
CN119691418B (zh) * 2024-11-26 2025-11-14 神州数码(中国)有限公司 模型评估方法、装置、电子设备和计算机可读存储介质
CN120768624B (zh) * 2025-07-16 2026-06-02 清华大学 面向多模态对抗扰动的智能流量分析鲁棒性认证方法
CN121096350A (zh) * 2025-09-03 2025-12-09 哈尔滨工业大学 一种基于脑电信号的语音解码方法、系统、设备及介质

Family Cites Families (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20120123289A1 (en) * 2010-11-11 2012-05-17 I2S Micro Implantable Systems, Llc System and method for wireless transmission of neural data
US20180228394A1 (en) * 2010-12-07 2018-08-16 Emtee Associates, L.P. Systems and Methods for Assisting a Subject to Learn to Independently Identify and Alter His or Her Brain State
US20150338917A1 (en) * 2012-12-26 2015-11-26 Sia Technology Ltd. Device, system, and method of controlling electronic devices via thought
WO2014130571A1 (en) * 2013-02-19 2014-08-28 The Regents Of The University Of California Methods of decoding speech from the brain and systems for practicing the same
US10130809B2 (en) * 2014-06-13 2018-11-20 Nervana, LLC Transcutaneous electrostimulator and methods for electric stimulation
US12008987B2 (en) 2018-04-30 2024-06-11 The Board Of Trustees Of The Leland Stanford Junior University Systems and methods for decoding intended speech from neuronal activity
EP4003156A4 (en) 2019-07-29 2023-04-12 The Regents of The University of California PROCEDURE FOR CONTEXTUAL LANGUAGE DECODING FROM THE BRAIN
CN111728609B (zh) * 2020-08-26 2020-11-17 腾讯科技（深圳）有限公司 脑电信号的分类方法、分类模型的训练方法、装置及介质

Also Published As

Publication number Publication date
EP4329615A4 (en) 2025-01-01
CN117693315A (zh) 2024-03-12
AU2022282378A1 (en) 2023-12-14
US20240366157A1 (en) 2024-11-07
WO2022251472A9 (en) 2023-11-09
CA3220064A1 (en) 2022-12-01
WO2022251472A1 (en) 2022-12-01
EP4329615A1 (en) 2024-03-06
KR20240024095A (ko) 2024-02-23

Similar Documents

Publication Publication Date Title
US20240366157A1 (en) Methods And Devices For Real-Time Word And Speech Decoding From Neural Activity
Dash et al. Decoding imagined and spoken phrases from non-invasive neural (MEG) signals
Metzger et al. Generalizable spelling using a speech neuroprosthesis in an individual with severe limb and vocal paralysis
Panachakel et al. Decoding covert speech from EEG-a comprehensive review
Kunz et al. Inner speech in motor cortex and implications for speech neuroprostheses
Moses et al. Neuroprosthesis for decoding speech in a paralyzed person with anarthria
Vorontsova et al. Silent EEG-speech recognition using convolutional and recurrent neural network with 85% accuracy of 9 words classification
Makin et al. Machine translation of cortical activity to text with an encoder–decoder framework
US20260060621A1 (en) Detection of disease conditions and comorbidities
WO2024254360A1 (en) Methods and systems for translation of neural activity into embodied digital-avatar animation
US20250252958A1 (en) Method of Contextual Speech Decoding From the Brain
Abdulghani et al. Imagined speech classification using EEG and deep learning
Wu et al. Adaptive LDA classifier enhances real-time control of an EEG brain–computer interface for decoding imagined syllables
Tan et al. Effective phoneme decoding with hyperbolic neural networks for high-performance speech BCIs
Alonso-Vázquez et al. EEG-based classification of spoken words using machine learning approaches
Ham et al. Vowel speech recognition from rat electroencephalography using long short-term memory neural network
Jiang et al. Decoding covert speech from EEG using a functional areas spatio-temporal transformer
Naebi et al. The Performance of a Lip-Sync Imagery Model, New Combinations of Signals, a Supplemental Bond Graph Classifier, and Deep Formula Detection as an Extraction and Root Classifier for Electroencephalograms and Brain–Computer Interfaces
Xiang et al. Cross-Modal Alignment and Rectified Flow-Based Latent Representation Synthesis for Enhanced Speech-Driven Alzheimer’s Disease Detection
Wang et al. Decoding linguistic representations of human brain
Zhang et al. Imagined Speech Brain–Computer Interface: A Task-Oriented Review of Neural Decoding
Berry Machine learning methods for articulatory data
Metzger AI-Driven Brain-Computer Interfaces for Speech
Tan et al. Decoding Chinese phonemes from intracortical brain signals with hyperbolic-space neural representations
Taleb Parkinson's desease detection by multimodal analysis combining handwriting and speech signals

Legal Events

Date Code Title Description
A521 Request for written amendment filed

Free format text: JAPANESE INTERMEDIATE CODE: A523

Effective date: 20240513

A521 Request for written amendment filed

Free format text: JAPANESE INTERMEDIATE CODE: A523

Effective date: 20250522

A621 Written request for application examination

Free format text: JAPANESE INTERMEDIATE CODE: A621

Effective date: 20250522

A131 Notification of reasons for refusal

Free format text: JAPANESE INTERMEDIATE CODE: A131

Effective date: 20260616