CN114930350A - 神经网络系统、神经网络的学习方法以及神经网络的学习程序 - Google Patents

神经网络系统、神经网络的学习方法以及神经网络的学习程序 Download PDF

Info

Publication number
CN114930350A
CN114930350A CN202080092251.0A CN202080092251A CN114930350A CN 114930350 A CN114930350 A CN 114930350A CN 202080092251 A CN202080092251 A CN 202080092251A CN 114930350 A CN114930350 A CN 114930350A
Authority
CN
China
Prior art keywords
update
processors
neural network
gradients
learning
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202080092251.0A
Other languages
English (en)
Chinese (zh)
Inventor
檀上匠
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Fujitsu Ltd
Original Assignee
Fujitsu Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Fujitsu Ltd filed Critical Fujitsu Ltd
Publication of CN114930350A publication Critical patent/CN114930350A/zh
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/06—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons
    • G06N3/063—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons using electronic means
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/048—Activation functions
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/0499—Feedforward networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/09—Supervised learning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/098—Distributed learning, e.g. federated learning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/044—Recurrent networks, e.g. Hopfield networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • General Health & Medical Sciences (AREA)
  • General Physics & Mathematics (AREA)
  • Evolutionary Computation (AREA)
  • Computational Linguistics (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Artificial Intelligence (AREA)
  • Neurology (AREA)
  • Feedback Control In General (AREA)
  • Multi Processors (AREA)
  • Debugging And Monitoring (AREA)
CN202080092251.0A 2020-01-10 2020-01-10 神经网络系统、神经网络的学习方法以及神经网络的学习程序 Pending CN114930350A (zh)

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/JP2020/000644 WO2021140643A1 (ja) 2020-01-10 2020-01-10 ニューラルネットワークシステム、ニューラルネットワークの学習方法及びニューラルネットワークの学習プログラム

Publications (1)

Publication Number Publication Date
CN114930350A true CN114930350A (zh) 2022-08-19

Family

ID=76787793

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202080092251.0A Pending CN114930350A (zh) 2020-01-10 2020-01-10 神经网络系统、神经网络的学习方法以及神经网络的学习程序

Country Status (5)

Country Link
US (1) US20220300790A1 (https=)
EP (1) EP4089586A4 (https=)
JP (1) JP7453563B2 (https=)
CN (1) CN114930350A (https=)
WO (1) WO2021140643A1 (https=)

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP7413528B2 (ja) * 2021-12-03 2024-01-15 三菱電機株式会社 学習済モデル生成システム、学習済モデル生成方法、情報処理装置、プログラム、および推定装置
US20250106120A1 (en) * 2022-01-13 2025-03-27 Lg Electronics Inc. Method by which reception device performs end-to-end training in wireless communication system, reception device, processing device, storage medium, method by which transmission device performs end-to-end training, and transmission device
CN115169532A (zh) * 2022-07-06 2022-10-11 北京灵汐科技有限公司 基于众核系统的神经网络训练方法及装置、电子设备

Family Cites Families (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2012079080A (ja) 2010-10-01 2012-04-19 Nippon Hoso Kyokai <Nhk> パラメタ学習装置およびそのプログラム
JP6880774B2 (ja) 2017-01-26 2021-06-02 日本電気株式会社 通信システム、分散計算システム、ノード、情報共有方法及びプログラム
JP6877393B2 (ja) * 2017-12-18 2021-05-26 株式会社東芝 システム、プログラム及び方法
JP2019212111A (ja) * 2018-06-06 2019-12-12 株式会社Ｐｒｅｆｅｒｒｅｄ Ｎｅｔｗｏｒｋｓ 分散学習方法及び分散学習装置
KR102732517B1 (ko) * 2018-07-04 2024-11-20 삼성전자주식회사 뉴럴 네트워크에서 파라미터를 처리하는 방법 및 장치
CN110795228B (zh) * 2018-08-03 2023-08-25 伊姆西Ip控股有限责任公司 用于训练深度学习模型的方法和制品、以及计算系统
US10776164B2 (en) * 2018-11-30 2020-09-15 EMC IP Holding Company LLC Dynamic composition of data pipeline in accelerator-as-a-service computing environment

Also Published As

Publication number Publication date
JP7453563B2 (ja) 2024-03-21
EP4089586A1 (en) 2022-11-16
US20220300790A1 (en) 2022-09-22
EP4089586A4 (en) 2023-02-01
WO2021140643A1 (ja) 2021-07-15
JPWO2021140643A1 (https=) 2021-07-15

Similar Documents

Publication Publication Date Title
US11640522B2 (en) Computational efficiency improvements for artificial neural networks
CN109800883B (zh) 量子机器学习框架构建方法、装置及量子计算机
CA3135529A1 (en) Adaptive error correction in quantum computing
Tsianos et al. Communication/computation tradeoffs in consensus-based distributed optimization
US20220300790A1 (en) Neural network system, neural network learning method, and neural network learning program
Mahajan et al. A distributed block coordinate descent method for training l1 regularized linear classifiers
Zhang et al. Zeroth-order policy gradient for reinforcement learning from human feedback without reward inference
CN109284826A (zh) 神经网络处理方法、装置、设备及计算机可读存储介质
CN104834216A (zh) 一种基于bp神经网络调节pi控制器参数的电路及方法
Xiao An online algorithm for nonparametric correlations
CN121525890A (zh) 基于光量子的量子系统数据处理方法及光量子计算机
Kumar et al. Generalized simplex algorithm to solve fuzzy linear programming problems with ranking of generalized fuzzy numbers
CN111967590B (zh) 面向推荐系统矩阵分解方法的异构多xpu机器学习系统
Zheng et al. Improved adaptive war strategy optimization algorithm assisted-adaptive multi-head graph attention mechanism network for remaining useful life of complex equipment
Breschi et al. Cloud-aided collaborative estimation by admm-rls algorithms for connected diagnostics and prognostics
Ferrarotti et al. The benefits of sharing: a cloud-aided performance-driven framework to learn optimal feedback policies
US12105612B1 (en) Algorithmic architecture co-design and exploration
CN108197083A (zh) 一种数据中心线性回归与小波神经网络融合的短期工作负载预测方法
Xie et al. Trainbf: High-performance dnn training engine using bfloat16 on ai accelerators
CN112532464B (zh) 一种跨多数据中心的数据分布式处理加速方法及其系统
Xue et al. Distributed differentially private matrix factorization based on ADMM
Liu et al. Fully distributed event‐driven cooperation with unknown parameters under directed graphs
Li et al. FIONA: Fine-grained incoherent optical DNN accelerator search for superior efficiency and robustness
CN117313815B (zh) 一种优化MZI的ONNs相位配置的渐进式训练方法
CN116644813B (zh) 一种利用量子电路确定最优组合方案的方法及装置

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination