JP2024543062A - 自然言語処理のパスのドロップアウト - Google Patents

自然言語処理のパスのドロップアウト Download PDF

Info

Publication number
JP2024543062A
JP2024543062A JP2024527585A JP2024527585A JP2024543062A JP 2024543062 A JP2024543062 A JP 2024543062A JP 2024527585 A JP2024527585 A JP 2024527585A JP 2024527585 A JP2024527585 A JP 2024527585A JP 2024543062 A JP2024543062 A JP 2024543062A
Authority
JP
Japan
Prior art keywords
machine learning
learning model
dropout
utterance
attention
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
JP2024527585A
Other languages
English (en)
Japanese (ja)
Other versions
JP2024543062A5 (https=
Inventor
ブー，タン・ティエン
ファム，トゥエン・クアン
ジョンソン，マーク・エドワード
ドゥオン，タン・ロン
Original Assignee
オラクル・インターナショナル・コーポレイション
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by オラクル・インターナショナル・コーポレイション filed Critical オラクル・インターナショナル・コーポレイション
Publication of JP2024543062A publication Critical patent/JP2024543062A/ja
Publication of JP2024543062A5 publication Critical patent/JP2024543062A5/ja
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L15/00—Speech recognition
    • G10L15/06—Creation of reference templates; Training of speech recognition systems, e.g. adaptation to the characteristics of the speaker's voice
    • G10L15/063—Training
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F40/00—Handling natural language data
    • G06F40/20—Natural language analysis
    • G06F40/279—Recognition of textual entities
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F40/00—Handling natural language data
    • G06F40/30—Semantic analysis
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/044—Recurrent networks, e.g. Hopfield networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/082—Learning methods modifying the architecture, e.g. adding, deleting or silencing nodes or connections
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/084—Backpropagation, e.g. using gradient descent
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/0985—Hyperparameter optimisation; Meta-learning; Learning-to-learn
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N7/00—Computing arrangements based on specific mathematical models
    • G06N7/01—Probabilistic graphical models, e.g. probabilistic networks
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L15/00—Speech recognition
    • G10L15/08—Speech classification or search
    • G10L15/18—Speech classification or search using natural language modelling
    • G10L15/1815—Semantic context, e.g. disambiguation of the recognition hypotheses based on word meaning
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L15/00—Speech recognition
    • G10L15/22—Procedures used during a speech recognition process, e.g. man-machine dialogue

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Artificial Intelligence (AREA)
  • General Physics & Mathematics (AREA)
  • Computational Linguistics (AREA)
  • Health & Medical Sciences (AREA)
  • General Engineering & Computer Science (AREA)
  • General Health & Medical Sciences (AREA)
  • Software Systems (AREA)
  • Evolutionary Computation (AREA)
  • Computing Systems (AREA)
  • Mathematical Physics (AREA)
  • Data Mining & Analysis (AREA)
  • Biomedical Technology (AREA)
  • Molecular Biology (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Biophysics (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • Acoustics & Sound (AREA)
  • Human Computer Interaction (AREA)
  • Multimedia (AREA)
  • Probability & Statistics with Applications (AREA)
  • Algebra (AREA)
  • Computational Mathematics (AREA)
  • Mathematical Analysis (AREA)
  • Mathematical Optimization (AREA)
  • Pure & Applied Mathematics (AREA)
  • Machine Translation (AREA)
JP2024527585A 2021-11-17 2022-11-16 自然言語処理のパスのドロップアウト Pending JP2024543062A (ja)

Applications Claiming Priority (3)

Application Number Priority Date Filing Date Title
US202163280580P 2021-11-17 2021-11-17
US63/280,580 2021-11-17
PCT/US2022/050076 WO2023091468A1 (en) 2021-11-17 2022-11-16 Path dropout for natural language processing

Publications (2)

Publication Number Publication Date
JP2024543062A true JP2024543062A (ja) 2024-11-19
JP2024543062A5 JP2024543062A5 (https=) 2025-06-16

Family

ID=86323954

Family Applications (1)

Application Number Title Priority Date Filing Date
JP2024527585A Pending JP2024543062A (ja) 2021-11-17 2022-11-16 自然言語処理のパスのドロップアウト

Country Status (6)

Country Link
US (1) US12412563B2 (https=)
JP (1) JP2024543062A (https=)
KR (1) KR20240111760A (https=)
CN (1) CN118235143A (https=)
GB (1) GB2625476A (https=)
WO (1) WO2023091468A1 (https=)

Families Citing this family (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US12572852B2 (en) 2021-12-23 2026-03-10 Oracle International Corporation Lexical dropout for natural language processing
US12566921B2 (en) * 2021-12-23 2026-03-03 Oracle International Corporation Gazetteer integration for neural named entity recognition
US12493918B2 (en) * 2022-01-21 2025-12-09 Walmart Apollo, Llc Systems and methods for dispute resolution
US12554934B2 (en) * 2022-10-13 2026-02-17 Oracle International Corporation Multi-task model with context masking
US12530545B2 (en) 2022-10-13 2026-01-20 Oracle International Corporation Data augmentation and batch balancing for training multi-lingual model
US12554518B2 (en) * 2023-08-04 2026-02-17 Dell Products L.P. Smart input devices with user behavior predictions
CN118968975A (zh) * 2024-07-18 2024-11-15 安徽海轩教育科技有限公司 人工智能助理系统

Family Cites Families (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11138392B2 (en) * 2018-07-26 2021-10-05 Google Llc Machine translation using neural network models
CN110349676B (zh) 2019-06-14 2021-10-29 华南师范大学 时序生理数据分类方法、装置、存储介质和处理器
CN113139585B (zh) * 2021-03-30 2022-03-29 太原科技大学 一种基于统一多尺度密集连接网络的红外与可见光图像融合方法
US11922290B2 (en) * 2021-05-24 2024-03-05 Visa International Service Association System, method, and computer program product for analyzing multivariate time series using a convolutional Fourier network
US12417372B2 (en) * 2021-06-15 2025-09-16 Riiid Inc. System for predicting user drop-out rate and tracking user knowledge based on artificial intelligence learning and method therefor
US20220405575A1 (en) * 2021-06-18 2022-12-22 Riiid Inc. System for predicting user drop-out rate based on artificial intelligence learning and method therefor

Also Published As

Publication number Publication date
US12412563B2 (en) 2025-09-09
GB202404323D0 (en) 2024-05-08
GB2625476A (en) 2024-06-19
CN118235143A (zh) 2024-06-21
KR20240111760A (ko) 2024-07-17
US20230154455A1 (en) 2023-05-18
WO2023091468A1 (en) 2023-05-25

Similar Documents

Publication Publication Date Title
JP7703667B2 (ja) 固有表現認識モデルを用いたコンテキストタグ統合
JP7682202B2 (ja) ドメイン外（ｏｏｄ）検出のための改良された技術
JP7561836B2 (ja) 自然言語処理のためのストップワードデータ拡張
US12099816B2 (en) Multi-factor modelling for natural language processing
JP7686678B2 (ja) 堅牢な固有表現認識のためのチャットボットにおけるエンティティレベルデータ拡張
JP7721559B2 (ja) 自然言語処理のためのノイズデータ拡張
US12512091B2 (en) Fine-tuning multi-head network from a single transformer layer of pre-trained language model
US12288550B2 (en) Framework for focused training of language models and techniques for end-to-end hypertuning of the framework
JP7771196B2 (ja) 自然言語プロセッサのための複数特徴均衡化
US12412563B2 (en) Path dropout for natural language processing
KR102821062B1 (ko) 사전-트레이닝된 언어 모델들에 대한 긴 텍스트를 핸들링하기 위한 시스템 및 기술들
JP2024540111A (ja) 文書からの埋め込まれるデータの抽出のための深層学習技術
US12572852B2 (en) Lexical dropout for natural language processing
JP2024540387A (ja) ハッシュ埋め込みを用いた言語検出のための広範な深層ネットワーク
JP2025528391A (ja) 名前付きエンティティ認識モデルの訓練を容易にするための適応的訓練データ拡大

Legal Events

Date Code Title Description
A521 Request for written amendment filed

Free format text: JAPANESE INTERMEDIATE CODE: A523

Effective date: 20250605

A621 Written request for application examination

Free format text: JAPANESE INTERMEDIATE CODE: A621

Effective date: 20250605

A131 Notification of reasons for refusal

Free format text: JAPANESE INTERMEDIATE CODE: A131

Effective date: 20260428