JP6789303B2 - ニューラルネットワークを用いたテキストセグメントの係り受け解析の生成 - Google Patents

ニューラルネットワークを用いたテキストセグメントの係り受け解析の生成 Download PDF

Info

Publication number
JP6789303B2
JP6789303B2 JP2018539314A JP2018539314A JP6789303B2 JP 6789303 B2 JP6789303 B2 JP 6789303B2 JP 2018539314 A JP2018539314 A JP 2018539314A JP 2018539314 A JP2018539314 A JP 2018539314A JP 6789303 B2 JP6789303 B2 JP 6789303B2
Authority
JP
Japan
Prior art keywords
neural network
training
dependency analysis
token
text sequence
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
JP2018539314A
Other languages
English (en)
Japanese (ja)
Other versions
JP2019511033A5 (enExample
JP2019511033A (ja
Inventor
ユアン・チャン
ディヴィッド・ジョセフ・ワイス
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Google LLC
Original Assignee
Google LLC
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Google LLC filed Critical Google LLC
Publication of JP2019511033A publication Critical patent/JP2019511033A/ja
Publication of JP2019511033A5 publication Critical patent/JP2019511033A5/ja
Application granted granted Critical
Publication of JP6789303B2 publication Critical patent/JP6789303B2/ja
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F40/00—Handling natural language data
    • G06F40/20—Natural language analysis
    • G06F40/205—Parsing
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/084—Backpropagation, e.g. using gradient descent
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F40/00—Handling natural language data
    • G06F40/20—Natural language analysis
    • G06F40/279—Recognition of textual entities
    • G06F40/284—Lexical analysis, e.g. tokenisation or collocates
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/0499—Feedforward networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/09—Supervised learning

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Health & Medical Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Artificial Intelligence (AREA)
  • General Physics & Mathematics (AREA)
  • Computational Linguistics (AREA)
  • General Engineering & Computer Science (AREA)
  • Biomedical Technology (AREA)
  • Evolutionary Computation (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • Data Mining & Analysis (AREA)
  • Biophysics (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • Machine Translation (AREA)
JP2018539314A 2016-03-18 2017-03-17 ニューラルネットワークを用いたテキストセグメントの係り受け解析の生成 Active JP6789303B2 (ja)

Applications Claiming Priority (3)

Application Number Priority Date Filing Date Title
US201662310513P 2016-03-18 2016-03-18
US62/310,513 2016-03-18
PCT/US2017/023047 WO2017161320A1 (en) 2016-03-18 2017-03-17 Generating dependency parses of text segments using neural networks

Publications (3)

Publication Number Publication Date
JP2019511033A JP2019511033A (ja) 2019-04-18
JP2019511033A5 JP2019511033A5 (enExample) 2019-12-19
JP6789303B2 true JP6789303B2 (ja) 2020-11-25

Family

ID=58547795

Family Applications (1)

Application Number Title Priority Date Filing Date
JP2018539314A Active JP6789303B2 (ja) 2016-03-18 2017-03-17 ニューラルネットワークを用いたテキストセグメントの係り受け解析の生成

Country Status (6)

Country Link
US (1) US10878188B2 (enExample)
EP (1) EP3394798A1 (enExample)
JP (1) JP6789303B2 (enExample)
KR (1) KR102201936B1 (enExample)
CN (1) CN109313719B (enExample)
WO (1) WO2017161320A1 (enExample)

Families Citing this family (22)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109558583A (zh) * 2017-09-27 2019-04-02 株式会社理光 一种自动生成文摘的方法、装置及设备
US10482162B2 (en) * 2017-11-30 2019-11-19 International Business Machines Corporation Automatic equation transformation from text
US10721190B2 (en) * 2018-07-31 2020-07-21 Microsoft Technology Licensing, Llc Sequence to sequence to classification model for generating recommended messages
CN110033091B (zh) 2018-12-13 2020-09-01 阿里巴巴集团控股有限公司 一种基于模型进行预测的方法和装置
RU2721190C1 (ru) * 2018-12-25 2020-05-18 Общество с ограниченной ответственностью "Аби Продакшн" Обучение нейронных сетей с использованием функций потерь, отражающих зависимости между соседними токенами
US10956474B2 (en) 2019-03-14 2021-03-23 Microsoft Technology Licensing, Llc Determination of best set of suggested responses
US11501137B2 (en) * 2019-06-28 2022-11-15 International Business Machines Corporation Feature engineering in neural networks optimization
US11436486B2 (en) * 2019-08-19 2022-09-06 Advanced Micro Devices, Inc. Neural network internal data fast access memory buffer
CN110765243A (zh) * 2019-09-17 2020-02-07 平安科技（深圳）有限公司 自然语言处理系统的构建方法、电子装置及计算机设备
CN112800737B (zh) * 2019-10-29 2024-06-18 京东科技控股股份有限公司 自然语言文本生成方法和装置以及对话系统
CN111178039B (zh) * 2019-12-18 2023-05-09 北京明略软件系统有限公司 一种模型训练方法、装置、实现文本处理的方法及装置
EP4091063A4 (en) * 2020-01-17 2024-02-21 nference, inc. SYSTEMS AND METHODS FOR MAPPING A TERM TO A VECTOR REPRESENTATION IN A SEMANTIC SPACE
US12271694B1 (en) * 2020-04-24 2025-04-08 Educational Testing Service Machine learning-based automated narrative text scoring including emotion arc characterization
CN111832316B (zh) * 2020-06-30 2024-05-24 北京小米松果电子有限公司 语义识别的方法、装置、电子设备和存储介质
CN111832315B (zh) * 2020-06-30 2024-04-30 北京小米松果电子有限公司 语义识别的方法、装置、电子设备和存储介质
CN112528671A (zh) * 2020-12-02 2021-03-19 北京小米松果电子有限公司 语义分析方法、装置以及存储介质
CN112699665B (zh) * 2021-03-25 2021-08-27 北京智源人工智能研究院 一种安全报告文本的三元组抽取方法、装置及电子设备
CN113762445B (zh) * 2021-04-26 2023-11-07 重庆大学 一种基于装配式建筑的标识解析系统
CN113468878B (zh) * 2021-07-13 2024-09-06 腾讯科技（深圳）有限公司 词性标注方法、装置、电子设备及存储介质
US12387725B2 (en) 2021-12-16 2025-08-12 Samsung Electronics Co., Ltd. Electronic device and method for recommending voice command thereof
CN119296528B (zh) * 2024-09-25 2025-09-16 哈尔滨工业大学(深圳)(哈尔滨工业大学深圳科技创新研究院) 一种基于语音端到端的依存结构分析方法及装置
CN119379426B (zh) * 2024-12-31 2025-04-08 杭银消费金融股份有限公司 一种风控拒绝用户数据处理系统

Family Cites Families (17)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH03263266A (ja) 1990-03-14 1991-11-22 Fujitsu Ltd 未知語の品詞選択装置
US6076051A (en) * 1997-03-07 2000-06-13 Microsoft Corporation Information retrieval utilizing semantic representation of text
CN1313972A (zh) * 1998-08-24 2001-09-19 Bcl计算机有限公司 自适应的自然语言接口
US7016828B1 (en) * 2000-10-23 2006-03-21 At&T Corp. Text-to-scene conversion
JP2007502459A (ja) 2003-08-12 2007-02-08 コーニンクレッカ　フィリップス　エレクトロニクス　エヌ　ヴィ 対話システム用音声入力インタフェース
GB2428508B (en) * 2005-07-15 2009-10-21 Toshiba Res Europ Ltd Parsing method
US8700403B2 (en) * 2005-11-03 2014-04-15 Robert Bosch Gmbh Unified treatment of data-sparseness and data-overfitting in maximum entropy modeling
US7603330B2 (en) * 2006-02-01 2009-10-13 Honda Motor Co., Ltd. Meta learning for question classification
US8874434B2 (en) * 2010-06-02 2014-10-28 Nec Laboratories America, Inc. Method and apparatus for full natural language parsing
US9454962B2 (en) 2011-05-12 2016-09-27 Microsoft Technology Licensing, Llc Sentence simplification for spoken language understanding
US8935151B1 (en) * 2011-12-07 2015-01-13 Google Inc. Multi-source transfer of delexicalized dependency parsers
CN102662931B (zh) * 2012-04-13 2015-03-25 厦门大学 一种基于协同神经网络的语义角色标注方法
CN102866989B (zh) * 2012-08-30 2016-09-07 北京航空航天大学 基于词语依存关系的观点抽取方法
US9507852B2 (en) * 2013-12-10 2016-11-29 Google Inc. Techniques for discriminative dependency parsing
US10366327B2 (en) * 2014-01-31 2019-07-30 Google Llc Generating vector representations of documents
CN104049755B (zh) * 2014-06-18 2017-01-18 中国科学院自动化研究所 信息处理方法及装置
CN104575501B (zh) * 2015-01-19 2017-11-03 北京云知声信息技术有限公司 一种收音机语音操控指令解析方法及系统

Also Published As

Publication number Publication date
CN109313719B (zh) 2022-03-22
US20190073351A1 (en) 2019-03-07
KR102201936B1 (ko) 2021-01-12
CN109313719A (zh) 2019-02-05
KR20180099822A (ko) 2018-09-05
EP3394798A1 (en) 2018-10-31
US10878188B2 (en) 2020-12-29
JP2019511033A (ja) 2019-04-18
WO2017161320A1 (en) 2017-09-21

Similar Documents

Publication Publication Date Title
JP6789303B2 (ja) ニューラルネットワークを用いたテキストセグメントの係り受け解析の生成
JP7253593B2 (ja) 意味解析モデルのトレーニング方法及び装置、電子機器並びに記憶媒体
US12204851B2 (en) Method for generating pre-trained language model, electronic device and storage medium
JP7358698B2 (ja) 語義表現モデルの訓練方法、装置、デバイス及び記憶媒体
JP7273108B2 (ja) モデルトレーニング方法、装置、電子デバイス、記憶媒体、プログラム
JP6776448B2 (ja) 機械学習タスクの暗黙的ブリッジング
JP7670905B2 (ja) スパースアテンションメカニズムを備えたアテンションニューラルネットワーク
JP6790286B2 (ja) 強化学習を用いたデバイス配置最適化
AU2017324937B2 (en) Generating audio using neural networks
JP7431833B2 (ja) 言語シーケンスラベリング方法、装置、プログラム及びコンピューティング機器
KR102195223B1 (ko) 전역적으로 노멀화된 신경 네트워크들
WO2021159201A1 (en) Initialization of parameters for machine-learned transformer neural network architectures
CN117933221A (zh) 构建和处理用于动态结构化机器学习模型的计算图
JP2018537788A (ja) 外部メモリを用いたニューラルネットワークの拡張
JP7495543B2 (ja) 事前トレーニングモデルのプロンプトベクトルの決定方法、装置及び電子機器
CN110663049A (zh) 神经网络优化器搜索
CN110402445A (zh) 使用递归神经网络处理序列数据
CN108563488A (zh) 模型训练方法及装置、构建容器镜像的方法及装置
CN111133458A (zh) 增强神经网络
JP7596559B2 (ja) フィードフォワード空間変換ユニットを備えたニューラルネットワーク
CN114792097B (zh) 预训练模型提示向量的确定方法、装置及电子设备
CN112329429A (zh) 文本相似度学习方法、装置、设备以及存储介质
JP7375096B2 (ja) 分散表現生成システム、分散表現生成方法及び分散表現生成プログラム
CN120994776A (zh) 应用于目标场景的问题推送方法、装置、设备和可读介质
Ramanujam Factorial Hidden Markov Models for full and weakly supervised supertagging

Legal Events

Date Code Title Description
A621 Written request for application examination

Free format text: JAPANESE INTERMEDIATE CODE: A621

Effective date: 20180813

A977 Report on retrieval

Free format text: JAPANESE INTERMEDIATE CODE: A971007

Effective date: 20190726

A131 Notification of reasons for refusal

Free format text: JAPANESE INTERMEDIATE CODE: A131

Effective date: 20190805

A524 Written submission of copy of amendment under article 19 pct

Free format text: JAPANESE INTERMEDIATE CODE: A524

Effective date: 20191105

A02 Decision of refusal

Free format text: JAPANESE INTERMEDIATE CODE: A02

Effective date: 20200420

A521 Request for written amendment filed

Free format text: JAPANESE INTERMEDIATE CODE: A523

Effective date: 20200807

C60 Trial request (containing other claim documents, opposition documents)

Free format text: JAPANESE INTERMEDIATE CODE: C60

Effective date: 20200807

A911 Transfer to examiner for re-examination before appeal (zenchi)

Free format text: JAPANESE INTERMEDIATE CODE: A911

Effective date: 20200814

C21 Notice of transfer of a case for reconsideration by examiners before appeal proceedings

Free format text: JAPANESE INTERMEDIATE CODE: C21

Effective date: 20200817

TRDD Decision of grant or rejection written
A01 Written decision to grant a patent or to grant a registration (utility model)

Free format text: JAPANESE INTERMEDIATE CODE: A01

Effective date: 20201005

A61 First payment of annual fees (during grant procedure)

Free format text: JAPANESE INTERMEDIATE CODE: A61

Effective date: 20201102

R150 Certificate of patent or registration of utility model

Ref document number: 6789303

Country of ref document: JP

Free format text: JAPANESE INTERMEDIATE CODE: R150

R250 Receipt of annual fees

Free format text: JAPANESE INTERMEDIATE CODE: R250

R250 Receipt of annual fees

Free format text: JAPANESE INTERMEDIATE CODE: R250

R250 Receipt of annual fees

Free format text: JAPANESE INTERMEDIATE CODE: R250