JP6651629B2 - 外部メモリを用いたニューラルネットワークの拡張 - Google Patents

外部メモリを用いたニューラルネットワークの拡張 Download PDF

Info

Publication number
JP6651629B2
JP6651629B2 JP2018530744A JP2018530744A JP6651629B2 JP 6651629 B2 JP6651629 B2 JP 6651629B2 JP 2018530744 A JP2018530744 A JP 2018530744A JP 2018530744 A JP2018530744 A JP 2018530744A JP 6651629 B2 JP6651629 B2 JP 6651629B2
Authority
JP
Japan
Prior art keywords
neural network
locations
weight
external memory
write
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
JP2018530744A
Other languages
English (en)
Japanese (ja)
Other versions
JP2018537788A5 (https=
JP2018537788A (ja
Inventor
アレクサンダー・ベンジャミン・グレイヴス
イヴォ・ダニエルカ
ティモシー・ジェームズ・アレクサンダー・ハーレー
マルコム・ケヴィン・キャンベル・レイノルズ
グレゴリー・ダンカン・ウェイン
Original Assignee
ディープマインド テクノロジーズ リミテッド
ディープマインド テクノロジーズ リミテッド
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by ディープマインド テクノロジーズ リミテッド, ディープマインド テクノロジーズ リミテッド filed Critical ディープマインド テクノロジーズ リミテッド
Publication of JP2018537788A publication Critical patent/JP2018537788A/ja
Publication of JP2018537788A5 publication Critical patent/JP2018537788A5/ja
Application granted granted Critical
Publication of JP6651629B2 publication Critical patent/JP6651629B2/ja
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/044—Recurrent networks, e.g. Hopfield networks
    • G06N3/0442—Recurrent networks, e.g. Hopfield networks characterised by memory or gating, e.g. long short-term memory [LSTM] or gated recurrent units [GRU]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/082—Learning methods modifying the architecture, e.g. adding, deleting or silencing nodes or connections
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/0499—Feedforward networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/09—Supervised learning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/092—Reinforcement learning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/06—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons
    • G06N3/063—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons using electronic means

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • General Health & Medical Sciences (AREA)
  • General Physics & Mathematics (AREA)
  • Evolutionary Computation (AREA)
  • Computational Linguistics (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Artificial Intelligence (AREA)
  • Neurology (AREA)
  • Image Analysis (AREA)
  • Image Processing (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
JP2018530744A 2015-12-10 2016-12-09 外部メモリを用いたニューラルネットワークの拡張 Active JP6651629B2 (ja)

Applications Claiming Priority (3)

Application Number Priority Date Filing Date Title
US201562265912P 2015-12-10 2015-12-10
US62/265,912 2015-12-10
PCT/US2016/066020 WO2017100711A1 (en) 2015-12-10 2016-12-09 Augmenting neural networks with external memory

Publications (3)

Publication Number Publication Date
JP2018537788A JP2018537788A (ja) 2018-12-20
JP2018537788A5 JP2018537788A5 (https=) 2020-01-23
JP6651629B2 true JP6651629B2 (ja) 2020-02-19

Family

ID=57708799

Family Applications (1)

Application Number Title Priority Date Filing Date
JP2018530744A Active JP6651629B2 (ja) 2015-12-10 2016-12-09 外部メモリを用いたニューラルネットワークの拡張

Country Status (6)

Country Link
US (3) US10832134B2 (https=)
EP (1) EP3371747B1 (https=)
JP (1) JP6651629B2 (https=)
KR (1) KR102158683B1 (https=)
CN (1) CN108431832B (https=)
WO (1) WO2017100711A1 (https=)

Families Citing this family (21)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10044751B2 (en) * 2015-12-28 2018-08-07 Arbor Networks, Inc. Using recurrent neural networks to defeat DNS denial of service attacks
US20180189266A1 (en) * 2017-01-03 2018-07-05 Wipro Limited Method and a system to summarize a conversation
EP3566182A1 (en) * 2017-02-06 2019-11-13 Deepmind Technologies Limited Memory augmented generative temporal models
US11037330B2 (en) * 2017-04-08 2021-06-15 Intel Corporation Low rank matrix compression
US11243944B2 (en) * 2017-06-29 2022-02-08 Futurewei Technologies, Inc. Dynamic semantic networks for language understanding and question answering
CN107508866B (zh) * 2017-08-08 2020-10-02 重庆大学 减小移动设备端神经网络模型更新的传输消耗的方法
US11188820B2 (en) 2017-09-08 2021-11-30 International Business Machines Corporation Deep neural network performance analysis on shared memory accelerator systems
US10853725B2 (en) * 2018-05-18 2020-12-01 Deepmind Technologies Limited Neural networks with relational memory
JP6906478B2 (ja) * 2018-05-23 2021-07-21 株式会社東芝 情報処理装置、情報処理方法、およびプログラム
KR102788532B1 (ko) 2018-05-30 2025-03-31 삼성전자주식회사 뉴럴 네트워크 시스템, 이를 포함하는 어플리케이션 프로세서 및 뉴럴 네트워크 시스템의 동작방법
US11775815B2 (en) * 2018-08-10 2023-10-03 Samsung Electronics Co., Ltd. System and method for deep memory network
US20200090035A1 (en) * 2018-09-19 2020-03-19 International Business Machines Corporation Encoder-decoder memory-augmented neural network architectures
CN112789626B (zh) * 2018-09-27 2024-11-08 渊慧科技有限公司 可扩展和压缩的神经网络数据储存系统
CN110377342B (zh) * 2019-06-10 2022-08-30 平安科技（深圳）有限公司 基于卷积神经网络的显存处理方法、装置及存储介质
CN112149049A (zh) 2019-06-26 2020-12-29 北京百度网讯科技有限公司 用于变换矩阵的装置和方法、数据处理系统
US11651209B1 (en) * 2019-10-02 2023-05-16 Google Llc Accelerated embedding layer computations
US11461645B2 (en) * 2019-12-02 2022-10-04 International Business Machines Corporation Initialization of memory networks
KR20210079785A (ko) 2019-12-20 2021-06-30 삼성전자주식회사 뉴럴 네트워크의 컨볼루션 연산 처리 방법 및 장치
US11604976B2 (en) 2020-04-29 2023-03-14 International Business Machines Corporation Crossbar arrays for computations in memory-augmented neural networks
US20220245217A1 (en) * 2021-01-29 2022-08-04 Oracle International Corporation Adaptive Selection of Source Matrix Version for Matrix Multiply Operations
US20260072786A1 (en) * 2024-09-10 2026-03-12 SK Hynix Inc. Read threshold prediction based on weight-sharing deep neural networks in non-volatile memory devices

Family Cites Families (13)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5167006A (en) * 1989-12-29 1992-11-24 Ricoh Company, Ltd. Neuron unit, neural network and signal processing method
US6092018A (en) * 1996-02-05 2000-07-18 Ford Global Technologies, Inc. Trained neural network engine idle speed control system
AU5804100A (en) * 1999-05-24 2000-12-12 Ip Century Ag Neuronal network for computer-assisted knowledge management
US20050041453A1 (en) * 2003-08-22 2005-02-24 Brazis Paul W. Method and apparatus for reading and writing to solid-state memory
GB0426982D0 (en) * 2004-12-09 2005-01-12 Secr Defence Early detection of sepsis
US7747070B2 (en) * 2005-08-31 2010-06-29 Microsoft Corporation Training convolutional neural networks on graphics processing units
US8239611B2 (en) * 2007-12-28 2012-08-07 Spansion Llc Relocating data in a memory device
US9514739B2 (en) * 2012-06-06 2016-12-06 Cypress Semiconductor Corporation Phoneme score accelerator
US9141906B2 (en) * 2013-03-13 2015-09-22 Google Inc. Scoring concept terms using a deep network
CN104657776B (zh) * 2013-11-22 2018-06-05 华为技术有限公司 神经网络系统、基于神经网络系统的图像解析方法和装置
CN103617235B (zh) * 2013-11-26 2017-01-25 中国科学院信息工程研究所 一种基于粒子群算法的网络水军账号识别方法及系统
CN105940395B (zh) * 2014-01-31 2019-07-16 谷歌有限责任公司 生成文档的矢量表示
CN103824291B (zh) * 2014-02-24 2017-01-11 哈尔滨工程大学 连续量子雁群算法演化脉冲耦合神经网络系统参数的自动图像分割方法

Also Published As

Publication number Publication date
EP3371747A1 (en) 2018-09-12
WO2017100711A1 (en) 2017-06-15
CN108431832B (zh) 2022-09-13
JP2018537788A (ja) 2018-12-20
US12299575B2 (en) 2025-05-13
KR102158683B1 (ko) 2020-09-22
CN108431832A (zh) 2018-08-21
US20210117801A1 (en) 2021-04-22
US20170169332A1 (en) 2017-06-15
EP3371747B1 (en) 2023-07-19
US10832134B2 (en) 2020-11-10
KR20180091850A (ko) 2018-08-16
US20250315676A1 (en) 2025-10-09

Similar Documents

Publication Publication Date Title
JP6651629B2 (ja) 外部メモリを用いたニューラルネットワークの拡張
US12099928B2 (en) Augmented recurrent neural network with external memory
US20240135955A1 (en) Generating audio using neural networks
JP6676165B2 (ja) 畳み込みゲート制御再帰型ニューラルネットワーク
US12260334B2 (en) Neural programming
US11210579B2 (en) Augmenting neural networks with external memory
US10706840B2 (en) Encoder-decoder models for sequence to sequence mapping
US10083169B1 (en) Topic-based sequence modeling neural networks
CN109661672B (zh) 使用强化学习利用外部存储器增强神经网络
US20170140753A1 (en) Generating target sequences from input sequences using partial conditioning
US11755879B2 (en) Low-pass recurrent neural network systems with memory

Legal Events

Date Code Title Description
A621 Written request for application examination

Free format text: JAPANESE INTERMEDIATE CODE: A621

Effective date: 20180710

A977 Report on retrieval

Free format text: JAPANESE INTERMEDIATE CODE: A971007

Effective date: 20190827

A131 Notification of reasons for refusal

Free format text: JAPANESE INTERMEDIATE CODE: A131

Effective date: 20190909

A524 Written submission of copy of amendment under article 19 pct

Free format text: JAPANESE INTERMEDIATE CODE: A524

Effective date: 20191206

TRDD Decision of grant or rejection written
A01 Written decision to grant a patent or to grant a registration (utility model)

Free format text: JAPANESE INTERMEDIATE CODE: A01

Effective date: 20191223

A61 First payment of annual fees (during grant procedure)

Free format text: JAPANESE INTERMEDIATE CODE: A61

Effective date: 20200122

R150 Certificate of patent or registration of utility model

Ref document number: 6651629

Country of ref document: JP

Free format text: JAPANESE INTERMEDIATE CODE: R150

R250 Receipt of annual fees

Free format text: JAPANESE INTERMEDIATE CODE: R250

R250 Receipt of annual fees

Free format text: JAPANESE INTERMEDIATE CODE: R250

R250 Receipt of annual fees

Free format text: JAPANESE INTERMEDIATE CODE: R250

R250 Receipt of annual fees

Free format text: JAPANESE INTERMEDIATE CODE: R250

S111 Request for change of ownership or part of ownership

Free format text: JAPANESE INTERMEDIATE CODE: R313113

R350 Written notification of registration of transfer

Free format text: JAPANESE INTERMEDIATE CODE: R350