KR20240111760A - 자연어 프로세싱을 위한 경로 드롭아웃 - Google Patents

자연어 프로세싱을 위한 경로 드롭아웃 Download PDF

Info

Publication number
KR20240111760A
KR20240111760A KR1020247018769A KR20247018769A KR20240111760A KR 20240111760 A KR20240111760 A KR 20240111760A KR 1020247018769 A KR1020247018769 A KR 1020247018769A KR 20247018769 A KR20247018769 A KR 20247018769A KR 20240111760 A KR20240111760 A KR 20240111760A
Authority
KR
South Korea
Prior art keywords
learning model
machine
attention
utterance
dropout
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
KR1020247018769A
Other languages
English (en)
Korean (ko)
Inventor
탄 티엔 부
투옌 꽝 팜
마크 에드워드 존슨
탄 롱 동
Original Assignee
오라클 인터내셔날 코포레이션
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 오라클 인터내셔날 코포레이션 filed Critical 오라클 인터내셔날 코포레이션
Publication of KR20240111760A publication Critical patent/KR20240111760A/ko
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L15/00—Speech recognition
    • G10L15/06—Creation of reference templates; Training of speech recognition systems, e.g. adaptation to the characteristics of the speaker's voice
    • G10L15/063—Training
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/082—Learning methods modifying the architecture, e.g. adding, deleting or silencing nodes or connections
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F40/00—Handling natural language data
    • G06F40/20—Natural language analysis
    • G06F40/279—Recognition of textual entities
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F40/00—Handling natural language data
    • G06F40/30—Semantic analysis
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/044—Recurrent networks, e.g. Hopfield networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/084—Backpropagation, e.g. using gradient descent
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/0985—Hyperparameter optimisation; Meta-learning; Learning-to-learn
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N7/00—Computing arrangements based on specific mathematical models
    • G06N7/01—Probabilistic graphical models, e.g. probabilistic networks
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L15/00—Speech recognition
    • G10L15/08—Speech classification or search
    • G10L15/18—Speech classification or search using natural language modelling
    • G10L15/1815—Semantic context, e.g. disambiguation of the recognition hypotheses based on word meaning
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L15/00—Speech recognition
    • G10L15/22—Procedures used during a speech recognition process, e.g. man-machine dialogue

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Artificial Intelligence (AREA)
  • General Physics & Mathematics (AREA)
  • Computational Linguistics (AREA)
  • Health & Medical Sciences (AREA)
  • General Engineering & Computer Science (AREA)
  • General Health & Medical Sciences (AREA)
  • Software Systems (AREA)
  • Evolutionary Computation (AREA)
  • Computing Systems (AREA)
  • Mathematical Physics (AREA)
  • Data Mining & Analysis (AREA)
  • Biomedical Technology (AREA)
  • Molecular Biology (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Biophysics (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • Acoustics & Sound (AREA)
  • Human Computer Interaction (AREA)
  • Multimedia (AREA)
  • Probability & Statistics with Applications (AREA)
  • Algebra (AREA)
  • Computational Mathematics (AREA)
  • Mathematical Analysis (AREA)
  • Mathematical Optimization (AREA)
  • Pure & Applied Mathematics (AREA)
  • Machine Translation (AREA)
KR1020247018769A 2021-11-17 2022-11-16 자연어 프로세싱을 위한 경로 드롭아웃 Pending KR20240111760A (ko)

Applications Claiming Priority (3)

Application Number Priority Date Filing Date Title
US202163280580P 2021-11-17 2021-11-17
US63/280,580 2021-11-17
PCT/US2022/050076 WO2023091468A1 (en) 2021-11-17 2022-11-16 Path dropout for natural language processing

Publications (1)

Publication Number Publication Date
KR20240111760A true KR20240111760A (ko) 2024-07-17

Family

ID=86323954

Family Applications (1)

Application Number Title Priority Date Filing Date
KR1020247018769A Pending KR20240111760A (ko) 2021-11-17 2022-11-16 자연어 프로세싱을 위한 경로 드롭아웃

Country Status (6)

Country Link
US (1) US12412563B2 (https=)
JP (1) JP2024543062A (https=)
KR (1) KR20240111760A (https=)
CN (1) CN118235143A (https=)
GB (1) GB2625476A (https=)
WO (1) WO2023091468A1 (https=)

Families Citing this family (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US12572852B2 (en) 2021-12-23 2026-03-10 Oracle International Corporation Lexical dropout for natural language processing
US12566921B2 (en) * 2021-12-23 2026-03-03 Oracle International Corporation Gazetteer integration for neural named entity recognition
US12493918B2 (en) * 2022-01-21 2025-12-09 Walmart Apollo, Llc Systems and methods for dispute resolution
US12554934B2 (en) * 2022-10-13 2026-02-17 Oracle International Corporation Multi-task model with context masking
US12530545B2 (en) 2022-10-13 2026-01-20 Oracle International Corporation Data augmentation and batch balancing for training multi-lingual model
US12554518B2 (en) * 2023-08-04 2026-02-17 Dell Products L.P. Smart input devices with user behavior predictions
CN118968975A (zh) * 2024-07-18 2024-11-15 安徽海轩教育科技有限公司 人工智能助理系统

Family Cites Families (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11138392B2 (en) * 2018-07-26 2021-10-05 Google Llc Machine translation using neural network models
CN110349676B (zh) 2019-06-14 2021-10-29 华南师范大学 时序生理数据分类方法、装置、存储介质和处理器
CN113139585B (zh) * 2021-03-30 2022-03-29 太原科技大学 一种基于统一多尺度密集连接网络的红外与可见光图像融合方法
US11922290B2 (en) * 2021-05-24 2024-03-05 Visa International Service Association System, method, and computer program product for analyzing multivariate time series using a convolutional Fourier network
US12417372B2 (en) * 2021-06-15 2025-09-16 Riiid Inc. System for predicting user drop-out rate and tracking user knowledge based on artificial intelligence learning and method therefor
US20220405575A1 (en) * 2021-06-18 2022-12-22 Riiid Inc. System for predicting user drop-out rate based on artificial intelligence learning and method therefor

Also Published As

Publication number Publication date
US12412563B2 (en) 2025-09-09
GB202404323D0 (en) 2024-05-08
GB2625476A (en) 2024-06-19
CN118235143A (zh) 2024-06-21
US20230154455A1 (en) 2023-05-18
JP2024543062A (ja) 2024-11-19
WO2023091468A1 (en) 2023-05-25

Similar Documents

Publication Publication Date Title
US12361219B2 (en) Context tag integration with named entity recognition models
US12099816B2 (en) Multi-factor modelling for natural language processing
JP7682202B2 (ja) ドメイン外（ｏｏｄ）検出のための改良された技術
US12512091B2 (en) Fine-tuning multi-head network from a single transformer layer of pre-trained language model
US12288550B2 (en) Framework for focused training of language models and techniques for end-to-end hypertuning of the framework
US12217497B2 (en) Extracting key information from document using trained machine-learning models
EP4165540A1 (en) Entity level data augmentation in chatbots for robust named entity recognition
US12412563B2 (en) Path dropout for natural language processing
US12518129B2 (en) Method and system for over-prediction in neural networks
KR102821062B1 (ko) 사전-트레이닝된 언어 모델들에 대한 긴 텍스트를 핸들링하기 위한 시스템 및 기술들
KR20240091051A (ko) 문서들로부터의 임베딩된 데이터의 추출을 위한 딥 러닝 기술들
US12572852B2 (en) Lexical dropout for natural language processing
US12518098B2 (en) Fusion of word embeddings and word scores for text classification
US12566921B2 (en) Gazetteer integration for neural named entity recognition
KR20240096829A (ko) 해시 임베딩들을 사용하는 언어 검출을 위한 와이드 및 딥 네트워크
US20230136965A1 (en) Prohibiting inconsistent named entity recognition tag sequences
KR20240091214A (ko) 데이터로부터의 질문 및 답변 쌍들의 추출을 위한 규칙-기반 기술들
WO2023091436A1 (en) System and techniques for handling long text for pre-trained language models

Legal Events

Date Code Title Description
PA0105 International application

Patent event date: 20240604

Patent event code: PA01051R01D

Comment text: International Patent Application

PG1501 Laying open of application