JP7453563B2 - ニューラルネットワークシステム、ニューラルネットワークの学習方法及びニューラルネットワークの学習プログラム - Google Patents

ニューラルネットワークシステム、ニューラルネットワークの学習方法及びニューラルネットワークの学習プログラム Download PDF

Info

Publication number
JP7453563B2
JP7453563B2 JP2021569687A JP2021569687A JP7453563B2 JP 7453563 B2 JP7453563 B2 JP 7453563B2 JP 2021569687 A JP2021569687 A JP 2021569687A JP 2021569687 A JP2021569687 A JP 2021569687A JP 7453563 B2 JP7453563 B2 JP 7453563B2
Authority
JP
Japan
Prior art keywords
update
neural network
processors
gradient
learning
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
JP2021569687A
Other languages
English (en)
Japanese (ja)
Other versions
JPWO2021140643A1 (https=
Inventor
匠 檀上
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Fujitsu Ltd
Original Assignee
Fujitsu Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Fujitsu Ltd filed Critical Fujitsu Ltd
Publication of JPWO2021140643A1 publication Critical patent/JPWO2021140643A1/ja
Application granted granted Critical
Publication of JP7453563B2 publication Critical patent/JP7453563B2/ja
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/048—Activation functions
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/06—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons
    • G06N3/063—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons using electronic means
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/0499—Feedforward networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/09—Supervised learning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/098—Distributed learning, e.g. federated learning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/044—Recurrent networks, e.g. Hopfield networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • General Health & Medical Sciences (AREA)
  • General Physics & Mathematics (AREA)
  • Evolutionary Computation (AREA)
  • Computational Linguistics (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Artificial Intelligence (AREA)
  • Neurology (AREA)
  • Feedback Control In General (AREA)
  • Multi Processors (AREA)
  • Debugging And Monitoring (AREA)
JP2021569687A 2020-01-10 2020-01-10 ニューラルネットワークシステム、ニューラルネットワークの学習方法及びニューラルネットワークの学習プログラム Active JP7453563B2 (ja)

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/JP2020/000644 WO2021140643A1 (ja) 2020-01-10 2020-01-10 ニューラルネットワークシステム、ニューラルネットワークの学習方法及びニューラルネットワークの学習プログラム

Publications (2)

Publication Number Publication Date
JPWO2021140643A1 JPWO2021140643A1 (https=) 2021-07-15
JP7453563B2 true JP7453563B2 (ja) 2024-03-21

Family

ID=76787793

Family Applications (1)

Application Number Title Priority Date Filing Date
JP2021569687A Active JP7453563B2 (ja) 2020-01-10 2020-01-10 ニューラルネットワークシステム、ニューラルネットワークの学習方法及びニューラルネットワークの学習プログラム

Country Status (5)

Country Link
US (1) US20220300790A1 (https=)
EP (1) EP4089586A4 (https=)
JP (1) JP7453563B2 (https=)
CN (1) CN114930350A (https=)
WO (1) WO2021140643A1 (https=)

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP7413528B2 (ja) * 2021-12-03 2024-01-15 三菱電機株式会社 学習済モデル生成システム、学習済モデル生成方法、情報処理装置、プログラム、および推定装置
US20250106120A1 (en) * 2022-01-13 2025-03-27 Lg Electronics Inc. Method by which reception device performs end-to-end training in wireless communication system, reception device, processing device, storage medium, method by which transmission device performs end-to-end training, and transmission device
CN115169532A (zh) * 2022-07-06 2022-10-11 北京灵汐科技有限公司 基于众核系统的神经网络训练方法及装置、电子设备

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2019109875A (ja) 2017-12-18 2019-07-04 株式会社東芝 システム、プログラム及び方法
JP2019212111A (ja) 2018-06-06 2019-12-12 株式会社Ｐｒｅｆｅｒｒｅｄ Ｎｅｔｗｏｒｋｓ 分散学習方法及び分散学習装置

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2012079080A (ja) 2010-10-01 2012-04-19 Nippon Hoso Kyokai <Nhk> パラメタ学習装置およびそのプログラム
JP6880774B2 (ja) 2017-01-26 2021-06-02 日本電気株式会社 通信システム、分散計算システム、ノード、情報共有方法及びプログラム
KR102732517B1 (ko) * 2018-07-04 2024-11-20 삼성전자주식회사 뉴럴 네트워크에서 파라미터를 처리하는 방법 및 장치
CN110795228B (zh) * 2018-08-03 2023-08-25 伊姆西Ip控股有限责任公司 用于训练深度学习模型的方法和制品、以及计算系统
US10776164B2 (en) * 2018-11-30 2020-09-15 EMC IP Holding Company LLC Dynamic composition of data pipeline in accelerator-as-a-service computing environment

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2019109875A (ja) 2017-12-18 2019-07-04 株式会社東芝 システム、プログラム及び方法
JP2019212111A (ja) 2018-06-06 2019-12-12 株式会社Ｐｒｅｆｅｒｒｅｄ Ｎｅｔｗｏｒｋｓ 分散学習方法及び分散学習装置

Also Published As

Publication number Publication date
EP4089586A1 (en) 2022-11-16
US20220300790A1 (en) 2022-09-22
EP4089586A4 (en) 2023-02-01
WO2021140643A1 (ja) 2021-07-15
CN114930350A (zh) 2022-08-19
JPWO2021140643A1 (https=) 2021-07-15

Similar Documents

Publication Publication Date Title
CN115357554B (zh) 一种图神经网络压缩方法、装置、电子设备及存储介质
US11436065B2 (en) System for efficient large-scale data distribution in distributed and parallel processing environment
JP7268996B2 (ja) 計算のためのシステムと方法
JP6823495B2 (ja) 情報処理装置および画像認識装置
JP7453563B2 (ja) ニューラルネットワークシステム、ニューラルネットワークの学習方法及びニューラルネットワークの学習プログラム
CN111898733A (zh) 一种深度可分离卷积神经网络加速器架构
CN114626516B (zh) 一种基于对数块浮点量化的神经网络加速系统
CN108564168A (zh) 一种对支持多精度卷积神经网络处理器的设计方法
WO2021115052A1 (zh) 一种异构芯片的任务处理方法、任务处理装置及电子设备
US11481618B2 (en) Optimization apparatus and method for controlling neural network
US20240152765A1 (en) Training time and resource consumption prediction in deep learning
CN119026694B (zh) 混合专家模型分布式训练方法、装置、设备、介质及程序
CN115186821A (zh) 面向芯粒的神经网络推理开销估计方法及装置、电子设备
CN111831358A (zh) 权重精度配置方法、装置、设备及存储介质
CN112100450A (zh) 一种图计算数据分割方法、终端设备及存储介质
Muthappa et al. Hardware-based fast real-time image classification with stochastic computing
Lin et al. Multi-node bert-pretraining: Cost-efficient approach
TW201909040A (zh) 神經網路處理方法、裝置、設備及電腦可讀儲存介質
CN109032630A (zh) 一种参数服务器中全局参数的更新方法
CN118798275A (zh) 模型计算方法以及相关装置
US12399713B2 (en) Multiplication hardware block with adaptive fidelity control system
CN118095343A (zh) 一种基于增量聚合策略的分布式图神经网络训练方法
CN112346703B (zh) 一种用于卷积神经网络计算的全局平均池化电路
CN116739054A (zh) 一种基于fpga的a3c深度强化学习算法加速器
CN116644813B (zh) 一种利用量子电路确定最优组合方案的方法及装置

Legal Events

Date Code Title Description
A621 Written request for application examination

Free format text: JAPANESE INTERMEDIATE CODE: A621

Effective date: 20220711

A131 Notification of reasons for refusal

Free format text: JAPANESE INTERMEDIATE CODE: A131

Effective date: 20231003

A521 Request for written amendment filed

Free format text: JAPANESE INTERMEDIATE CODE: A523

Effective date: 20231109

TRDD Decision of grant or rejection written
A01 Written decision to grant a patent or to grant a registration (utility model)

Free format text: JAPANESE INTERMEDIATE CODE: A01

Effective date: 20240206

A61 First payment of annual fees (during grant procedure)

Free format text: JAPANESE INTERMEDIATE CODE: A61

Effective date: 20240219

R150 Certificate of patent or registration of utility model

Ref document number: 7453563

Country of ref document: JP

Free format text: JAPANESE INTERMEDIATE CODE: R150