GB2576275A - Update management for RPU array - Google Patents

Update management for RPU array Download PDF

Info

Publication number
GB2576275A
GB2576275A GB1916146.2A GB201916146A GB2576275A GB 2576275 A GB2576275 A GB 2576275A GB 201916146 A GB201916146 A GB 201916146A GB 2576275 A GB2576275 A GB 2576275A
Authority
GB
United Kingdom
Prior art keywords
computer
implemented method
isotropic
scaling
update process
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Withdrawn
Application number
GB1916146.2A
Other languages
English (en)
Other versions
GB201916146D0 (en
Inventor
Gokmen Tayfun
Murat Onen Oguzhan
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
International Business Machines Corp
Original Assignee
International Business Machines Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by International Business Machines Corp filed Critical International Business Machines Corp
Publication of GB201916146D0 publication Critical patent/GB201916146D0/en
Publication of GB2576275A publication Critical patent/GB2576275A/en
Withdrawn legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F7/00—Methods or arrangements for processing data by operating upon the order or content of the data handled
    • G06F7/38—Methods or arrangements for performing computations using exclusively denominational number representation, e.g. using binary, ternary, decimal representation
    • G06F7/48—Methods or arrangements for performing computations using exclusively denominational number representation, e.g. using binary, ternary, decimal representation using non-contact-making devices, e.g. tube, solid state device; using unspecified devices
    • G06F7/544—Methods or arrangements for performing computations using exclusively denominational number representation, e.g. using binary, ternary, decimal representation using non-contact-making devices, e.g. tube, solid state device; using unspecified devices for evaluating functions by calculation
    • G06F7/5443—Sum of products
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
    • G06F17/10—Complex mathematical operations
    • G06F17/16—Matrix or vector computation, e.g. matrix-matrix or matrix-vector multiplication, matrix factorization
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F5/00—Methods or arrangements for data conversion without changing the order or content of the data handled
    • G06F5/01—Methods or arrangements for data conversion without changing the order or content of the data handled for shifting, e.g. justifying, scaling, normalising
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/0464—Convolutional networks [CNN, ConvNet]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/06—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons
    • G06N3/063—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons using electronic means
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/06—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons
    • G06N3/063—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons using electronic means
    • G06N3/065—Analogue means
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/084—Backpropagation, e.g. using gradient descent
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/09—Supervised learning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2207/00—Indexing scheme relating to methods or arrangements for processing data by operating upon the order or content of the data handled
    • G06F2207/38—Indexing scheme relating to groups G06F7/38 - G06F7/575
    • G06F2207/48—Indexing scheme relating to groups G06F7/48 - G06F7/575
    • G06F2207/4802—Special implementations
    • G06F2207/4814—Non-logic devices, e.g. operational amplifiers
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2207/00—Indexing scheme relating to methods or arrangements for processing data by operating upon the order or content of the data handled
    • G06F2207/38—Indexing scheme relating to groups G06F7/38 - G06F7/575
    • G06F2207/48—Indexing scheme relating to groups G06F7/48 - G06F7/575
    • G06F2207/4802—Special implementations
    • G06F2207/4818—Threshold devices
    • G06F2207/4824—Neural networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2207/00—Indexing scheme relating to methods or arrangements for processing data by operating upon the order or content of the data handled
    • G06F2207/38—Indexing scheme relating to groups G06F7/38 - G06F7/575
    • G06F2207/48—Indexing scheme relating to groups G06F7/48 - G06F7/575
    • G06F2207/4802—Special implementations
    • G06F2207/4828—Negative resistance devices, e.g. tunnel diodes, gunn effect devices

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • Mathematical Physics (AREA)
  • Computing Systems (AREA)
  • Biophysics (AREA)
  • Biomedical Technology (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Data Mining & Analysis (AREA)
  • Software Systems (AREA)
  • Computational Linguistics (AREA)
  • Molecular Biology (AREA)
  • General Health & Medical Sciences (AREA)
  • Evolutionary Computation (AREA)
  • Artificial Intelligence (AREA)
  • Mathematical Analysis (AREA)
  • Computational Mathematics (AREA)
  • Mathematical Optimization (AREA)
  • Pure & Applied Mathematics (AREA)
  • Neurology (AREA)
  • Algebra (AREA)
  • Databases & Information Systems (AREA)
  • Complex Calculations (AREA)
  • Management, Administration, Business Operations System, And Electronic Commerce (AREA)
  • Machine Translation (AREA)
GB1916146.2A 2017-04-14 2018-03-13 Update management for RPU array Withdrawn GB2576275A (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US15/487,701 US10783432B2 (en) 2017-04-14 2017-04-14 Update management for RPU array
PCT/IB2018/051644 WO2018189600A1 (en) 2017-04-14 2018-03-13 Update management for rpu array

Publications (2)

Publication Number Publication Date
GB201916146D0 GB201916146D0 (en) 2019-12-18
GB2576275A true GB2576275A (en) 2020-02-12

Family

ID=63790739

Family Applications (1)

Application Number Title Priority Date Filing Date
GB1916146.2A Withdrawn GB2576275A (en) 2017-04-14 2018-03-13 Update management for RPU array

Country Status (6)

Country Link
US (2) US10783432B2 (enExample)
JP (1) JP6986569B2 (enExample)
CN (1) CN110506282B (enExample)
DE (1) DE112018000723T5 (enExample)
GB (1) GB2576275A (enExample)
WO (1) WO2018189600A1 (enExample)

Families Citing this family (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10303998B2 (en) * 2017-09-28 2019-05-28 International Business Machines Corporation Floating gate for neural network inference
US11651231B2 (en) 2019-03-01 2023-05-16 Government Of The United States Of America, As Represented By The Secretary Of Commerce Quasi-systolic processor and quasi-systolic array
US11562249B2 (en) * 2019-05-01 2023-01-24 International Business Machines Corporation DNN training with asymmetric RPU devices
CN110750231B (zh) * 2019-09-27 2021-09-28 东南大学 一种面向卷积神经网络的双相系数可调模拟乘法计算电路
US11501148B2 (en) * 2020-03-04 2022-11-15 International Business Machines Corporation Area and power efficient implementations of modified backpropagation algorithm for asymmetric RPU devices
US11501023B2 (en) 2020-04-30 2022-11-15 International Business Machines Corporation Secure chip identification using resistive processing unit as a physically unclonable function
US11366876B2 (en) 2020-06-24 2022-06-21 International Business Machines Corporation Eigenvalue decomposition with stochastic optimization
US11568217B2 (en) * 2020-07-15 2023-01-31 International Business Machines Corporation Sparse modifiable bit length deterministic pulse generation for updating analog crossbar arrays
US11443171B2 (en) * 2020-07-15 2022-09-13 International Business Machines Corporation Pulse generation for updating crossbar arrays
US12488250B2 (en) * 2020-11-02 2025-12-02 International Business Machines Corporation Weight repetition on RPU crossbar arrays
US12165046B2 (en) * 2021-03-16 2024-12-10 International Business Machines Corporation Enabling hierarchical data loading in a resistive processing unit (RPU) array for reduced communication cost

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5258934A (en) * 1990-05-14 1993-11-02 California Institute Of Technology Charge domain bit serial vector-matrix multiplier and method thereof
US20040083193A1 (en) * 2002-10-29 2004-04-29 Bingxue Shi Expandable on-chip back propagation learning neural network with 4-neuron 16-synapse
US20170061281A1 (en) * 2015-08-27 2017-03-02 International Business Machines Corporation Deep neural network training with native devices
US20170091618A1 (en) * 2015-09-29 2017-03-30 International Business Machines International Scalable architecture for implementing maximization algorithms with resistive devices
US20170091616A1 (en) * 2015-09-29 2017-03-30 International Business Machines Corporation Scalable architecture for analog matrix operations with resistive devices

Family Cites Families (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH04153827A (ja) * 1990-10-18 1992-05-27 Fujitsu Ltd ディジタル乗算器
EP1508872A1 (en) * 2003-08-22 2005-02-23 Semeion An algorithm for recognising relationships between data of a database and a method for image pattern recognition based on the said algorithm
US9715655B2 (en) 2013-12-18 2017-07-25 The United States Of America As Represented By The Secretary Of The Air Force Method and apparatus for performing close-loop programming of resistive memory devices in crossbar array based hardware circuits and systems
US9466362B2 (en) 2014-08-12 2016-10-11 Arizona Board Of Regents On Behalf Of Arizona State University Resistive cross-point architecture for robust data representation with arbitrary precision
US20170061279A1 (en) * 2015-01-14 2017-03-02 Intel Corporation Updating an artificial neural network using flexible fixed point representation
CN105488565A (zh) * 2015-11-17 2016-04-13 中国科学院计算技术研究所 加速深度神经网络算法的加速芯片的运算装置及方法

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5258934A (en) * 1990-05-14 1993-11-02 California Institute Of Technology Charge domain bit serial vector-matrix multiplier and method thereof
US20040083193A1 (en) * 2002-10-29 2004-04-29 Bingxue Shi Expandable on-chip back propagation learning neural network with 4-neuron 16-synapse
US20170061281A1 (en) * 2015-08-27 2017-03-02 International Business Machines Corporation Deep neural network training with native devices
US20170091618A1 (en) * 2015-09-29 2017-03-30 International Business Machines International Scalable architecture for implementing maximization algorithms with resistive devices
US20170091616A1 (en) * 2015-09-29 2017-03-30 International Business Machines Corporation Scalable architecture for analog matrix operations with resistive devices

Also Published As

Publication number Publication date
US20180300622A1 (en) 2018-10-18
WO2018189600A1 (en) 2018-10-18
US11062208B2 (en) 2021-07-13
US10783432B2 (en) 2020-09-22
CN110506282A (zh) 2019-11-26
US20180300627A1 (en) 2018-10-18
CN110506282B (zh) 2023-04-28
JP6986569B2 (ja) 2021-12-22
DE112018000723T5 (de) 2019-10-24
GB201916146D0 (en) 2019-12-18
JP2020517002A (ja) 2020-06-11

Similar Documents

Publication Publication Date Title
GB2576275A (en) Update management for RPU array
SG10201707701PA (en) Performing Average Pooling In Hardware
Zhao et al. New conditions for global exponential stability of cellular neural networks with delays
Chen et al. A gradient based iterative solutions for Sylvester tensor equations
FI3539059T3 (fi) Kernel-askelluksen suorittaminen laitteistossa
JP2020517002A5 (enExample)
Yi et al. Comparison on neural solvers for the Lyapunov matrix equation with stationary & nonstationary coefficients
Wang et al. Does the fractional Brusselator with efficient dimension less than 1 have a limit cycle?
Babolian et al. An efficient method for nonlinear fractional differential equations: combination of the Adomian decomposition method and spectral method
Xin et al. Projective synchronization of chaotic fractional-order energy resources demand–supply systems via linear control
Malham et al. Efficient almost-exact Lévy area sampling
Ke et al. Analysis of an infinite multi-server queue with an optional service
JP2020027245A5 (ja) 情報処理方法、情報処理装置およびプログラム
Sathiyaraj et al. Controllability of Hilfer fractional stochastic system with multiple delays and Poisson jumps
Mozyrska Multiparameter fractional difference linear control systems
WO2024263741A3 (en) Context llm response system
Stamova On the Lyapunov theory for functional differential equations of fractional order
Liu et al. Consistency of the extended gradient identification algorithm for multi-input multi-output systems with moving average noises
Villa et al. An efficient algorithm for corona simulation with complex chemical models
Rebenda et al. A New Semi-analytical Approach for Numerical Solving of Cauchy Problem for Dierential Equations with Delay
CN114036457B (zh) 用于卷积运算的方法、计算设备、计算系统和存储介质
CN108595149B (zh) 可重构乘加运算装置
El-Kady et al. Monic Chebyshev approximations for solving optimal control problem with Volterra integro differential equations
Ghorbani Approximate solution of delay differential equations via variational iteration method
Najafi et al. Modified iteration method for solving fractional gas dynamics equation

Legal Events

Date Code Title Description
WAP Application withdrawn, taken to be withdrawn or refused ** after publication under section 16(1)