JP7139426B2 - 人工神経回路網用のシナプス重みを構築するための方法、システム、およびプログラム - Google Patents

人工神経回路網用のシナプス重みを構築するための方法、システム、およびプログラム Download PDF

Info

Publication number
JP7139426B2
JP7139426B2 JP2020526371A JP2020526371A JP7139426B2 JP 7139426 B2 JP7139426 B2 JP 7139426B2 JP 2020526371 A JP2020526371 A JP 2020526371A JP 2020526371 A JP2020526371 A JP 2020526371A JP 7139426 B2 JP7139426 B2 JP 7139426B2
Authority
JP
Japan
Prior art keywords
conductance
neurons
pairs
ann
output
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
JP2020526371A
Other languages
English (en)
Japanese (ja)
Other versions
JP2021503649A5 (cg-RX-API-DMAC7.html
JP2021503649A (ja
Inventor
バール、ジェフリー
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
International Business Machines Corp
Original Assignee
International Business Machines Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by International Business Machines Corp filed Critical International Business Machines Corp
Publication of JP2021503649A publication Critical patent/JP2021503649A/ja
Publication of JP2021503649A5 publication Critical patent/JP2021503649A5/ja
Application granted granted Critical
Publication of JP7139426B2 publication Critical patent/JP7139426B2/ja
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/084—Backpropagation, e.g. using gradient descent
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/06—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons
    • G06N3/063—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons using electronic means
    • G06N3/065—Analogue means
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N20/00—Machine learning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/0499—Feedforward networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/09—Supervised learning

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Software Systems (AREA)
  • Evolutionary Computation (AREA)
  • General Physics & Mathematics (AREA)
  • Artificial Intelligence (AREA)
  • Mathematical Physics (AREA)
  • Data Mining & Analysis (AREA)
  • Computing Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Molecular Biology (AREA)
  • General Health & Medical Sciences (AREA)
  • Computational Linguistics (AREA)
  • Neurology (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Medical Informatics (AREA)
  • Image Analysis (AREA)
  • Semiconductor Memories (AREA)
JP2020526371A 2017-11-20 2018-11-19 人工神経回路網用のシナプス重みを構築するための方法、システム、およびプログラム Active JP7139426B2 (ja)

Applications Claiming Priority (3)

Application Number Priority Date Filing Date Title
US15/818,521 US11074499B2 (en) 2017-11-20 2017-11-20 Synaptic weight transfer between conductance pairs with polarity inversion for reducing fixed device asymmetries
US15/818,521 2017-11-20
PCT/IB2018/059102 WO2019097494A1 (en) 2017-11-20 2018-11-19 Synaptic weight transfer between conductance pairs with polarity inversion for reducing fixed device asymmetries

Publications (3)

Publication Number Publication Date
JP2021503649A JP2021503649A (ja) 2021-02-12
JP2021503649A5 JP2021503649A5 (cg-RX-API-DMAC7.html) 2021-03-25
JP7139426B2 true JP7139426B2 (ja) 2022-09-20

Family

ID=66533047

Family Applications (1)

Application Number Title Priority Date Filing Date
JP2020526371A Active JP7139426B2 (ja) 2017-11-20 2018-11-19 人工神経回路網用のシナプス重みを構築するための方法、システム、およびプログラム

Country Status (6)

Country Link
US (2) US11074499B2 (cg-RX-API-DMAC7.html)
JP (1) JP7139426B2 (cg-RX-API-DMAC7.html)
CN (1) CN111373414B (cg-RX-API-DMAC7.html)
DE (1) DE112018004992B4 (cg-RX-API-DMAC7.html)
GB (1) GB2581310B (cg-RX-API-DMAC7.html)
WO (1) WO2019097494A1 (cg-RX-API-DMAC7.html)

Families Citing this family (13)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11074499B2 (en) 2017-11-20 2021-07-27 International Business Machines Corporation Synaptic weight transfer between conductance pairs with polarity inversion for reducing fixed device asymmetries
EP3701351A4 (en) 2018-01-30 2021-01-27 D5Ai Llc SELF-ORGANIZING PARTIALLY ORGANIZED NETWORKS
US11694065B2 (en) 2019-08-28 2023-07-04 Micron Technology, Inc. Spiking neural unit
US11183238B2 (en) 2019-08-28 2021-11-23 International Business Machines Corporation Suppressing outlier drift coefficients while programming phase change memory synapses
US12052260B2 (en) 2019-09-30 2024-07-30 International Business Machines Corporation Scalable and dynamic transfer learning mechanism
KR102782076B1 (ko) 2019-12-13 2025-03-18 삼성전자주식회사 스위칭 소자와 저항 소자를 포함하는 3차원 뉴로모픽 장치
US11475946B2 (en) 2020-01-16 2022-10-18 International Business Machines Corporation Synapse weight update compensation
KR102873065B1 (ko) * 2020-04-23 2025-10-16 삼성전자주식회사 뉴로모픽 컴퓨팅 장치 및 그것의 동작 방법
US12156486B2 (en) 2020-12-02 2024-11-26 International Business Machines Corporation Horizontal RRAM device and architecture for variability reduction
US11908458B2 (en) * 2020-12-29 2024-02-20 International Business Machines Corporation Customization of recurrent neural network transducers for speech recognition
US12626111B2 (en) * 2021-06-17 2026-05-12 International Business Machines Corporation Calibrating peripheral variability
KR102574156B1 (ko) 2021-07-06 2023-09-01 고려대학교 산학협력단 역전파와 음수 가중치 설정이 가능한 뉴로모픽 회로
KR102514652B1 (ko) * 2021-11-19 2023-03-29 서울대학교산학협력단 뉴로모픽 소자를 위한 가중치 전사 장치 및 이를 이용한 가중치 전사 방법

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2013108299A1 (ja) 2012-01-20 2013-07-25 パナソニック株式会社 ニューラルネットワーク回路の学習方法
JP2013534677A (ja) 2010-07-07 2013-09-05 クゥアルコム・インコーポレイテッド 生物規範型ネットワークのための伝達及びシナプス訓練方法及びハードウェア
JP2020528611A (ja) 2017-07-26 2020-09-24 インターナショナル・ビジネス・マシーンズ・コーポレーションInternational Business Machines Corporation 異なる重みの符号付きアナログ・コンダクタンス対によって人工ニューロン回路網に対するシナプス・ウェイトを構成するためのシステムおよび方法

Family Cites Families (26)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5101361A (en) * 1989-09-29 1992-03-31 The United States Of America As Represented By The Administrator Of The National Aeronautics And Space Administration Analog hardware for delta-backpropagation neural networks
EP0768610B1 (en) 1995-10-13 2000-08-09 STMicroelectronics S.r.l. Low-voltage, very-low-power neural network
CN101364594B (zh) * 2007-08-09 2010-06-02 中国科学院半导体研究所 硅基单电子神经元量子电路
US7814038B1 (en) 2007-12-06 2010-10-12 Dominic John Repici Feedback-tolerant method and device producing weight-adjustment factors for pre-synaptic neurons in artificial neural networks
WO2010133925A1 (en) 2009-05-20 2010-11-25 Universite Paris Sud (Paris 11) Teaching method for a neuronal nano- block
US9665822B2 (en) 2010-06-30 2017-05-30 International Business Machines Corporation Canonical spiking neuron network for spatiotemporal associative memory
US8694452B2 (en) 2010-07-07 2014-04-08 Qualcomm Incorporated Methods and systems for CMOS implementation of neuron synapse
US8676734B2 (en) 2010-07-07 2014-03-18 Qualcomm, Incorporated Methods and systems for replaceable synaptic weight storage in neuro-processors
US8655813B2 (en) 2010-12-30 2014-02-18 International Business Machines Corporation Synaptic weight normalized spiking neuronal networks
US8832010B2 (en) 2012-01-04 2014-09-09 International Business Machines Corporation Electronic synapses from stochastic binary memory devices
US20160364643A1 (en) 2012-03-08 2016-12-15 Hrl Laboratories Llc Scalable integrated circuit with synaptic electronics and cmos integrated memristors
US20150120631A1 (en) 2012-05-10 2015-04-30 Consejo Superior de Investagaciones Cientificas (CSIC) Method and System for Converting Pulsed-Processing Neural Network with Instantaneous Integration Synapses into Dynamic Integration Synapses
GB201209174D0 (en) 2012-05-25 2012-07-04 Univ Bath Neural network
US8811060B2 (en) 2012-05-31 2014-08-19 International Business Machines Corporation Non-volatile memory crosspoint repair
US9424514B2 (en) 2012-07-25 2016-08-23 Board Of Trustees Of Michigan State University Synapse maintenance in the developmental networks
US9218564B2 (en) 2012-07-30 2015-12-22 International Business Machines Corporation Providing transposable access to a synapse array using a recursive array layout
US20150019468A1 (en) 2013-07-09 2015-01-15 Knowmtech, Llc Thermodynamic computing
US9753959B2 (en) 2013-10-16 2017-09-05 University Of Tennessee Research Foundation Method and apparatus for constructing a neuroscience-inspired artificial neural network with visualization of neural pathways
CN203661036U (zh) * 2013-12-20 2014-06-18 浙江大学城市学院 一种基于浮栅技术的二值动态BiCMOS与门电路
US9619749B2 (en) 2014-03-06 2017-04-11 Progress, Inc. Neural network and method of neural network training
US9990580B2 (en) 2015-03-13 2018-06-05 International Business Machines Corporation Neuromorphic synapses
US10460237B2 (en) * 2015-11-30 2019-10-29 International Business Machines Corporation Neuron-centric local learning rate for artificial neural networks to increase performance, learning rate margin, and reduce power consumption
US10891543B2 (en) 2015-12-28 2021-01-12 Samsung Electronics Co., Ltd. LUT based synapse weight update scheme in STDP neuromorphic systems
US10614355B2 (en) 2015-12-30 2020-04-07 SK Hynix Inc. Method for updating weights of synapses of a neuromorphic device
US9715656B1 (en) 2016-09-12 2017-07-25 International Business Machines Corporation Killing asymmetric resistive processing units for neural network training
US11074499B2 (en) 2017-11-20 2021-07-27 International Business Machines Corporation Synaptic weight transfer between conductance pairs with polarity inversion for reducing fixed device asymmetries

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2013534677A (ja) 2010-07-07 2013-09-05 クゥアルコム・インコーポレイテッド 生物規範型ネットワークのための伝達及びシナプス訓練方法及びハードウェア
WO2013108299A1 (ja) 2012-01-20 2013-07-25 パナソニック株式会社 ニューラルネットワーク回路の学習方法
JP2020528611A (ja) 2017-07-26 2020-09-24 インターナショナル・ビジネス・マシーンズ・コーポレーションInternational Business Machines Corporation 異なる重みの符号付きアナログ・コンダクタンス対によって人工ニューロン回路網に対するシナプス・ウェイトを構成するためのシステムおよび方法

Also Published As

Publication number Publication date
DE112018004992T5 (de) 2020-06-10
CN111373414B (zh) 2023-08-25
GB2581310A (en) 2020-08-12
GB2581310B (en) 2020-12-09
CN111373414A (zh) 2020-07-03
US20210287090A1 (en) 2021-09-16
US11074499B2 (en) 2021-07-27
GB202008959D0 (en) 2020-07-29
JP2021503649A (ja) 2021-02-12
DE112018004992B4 (de) 2023-11-02
WO2019097494A1 (en) 2019-05-23
US11915132B2 (en) 2024-02-27
US20190156194A1 (en) 2019-05-23

Similar Documents

Publication Publication Date Title
US11915132B2 (en) Synaptic weight transfer between conductance pairs with polarity inversion for reducing fixed device asymmetries
JP7095069B2 (ja) 異なる重みの符号付きアナログ・コンダクタンス対によって人工ニューロン回路網に対するシナプス・ウェイトを構成するためのシステムおよび方法
US11157810B2 (en) Resistive processing unit architecture with separate weight update and inference circuitry
US10956815B2 (en) Killing asymmetric resistive processing units for neural network training
Nandakumar et al. Mixed-precision architecture based on computational memory for training deep neural networks
JP7336819B2 (ja) 抵抗処理ユニット・アレイのクロスポイント・デバイスに重みを記憶するための方法、そのクロスポイント・デバイス、ニューラル・ネットワークを実施するためのクロスポイント・アレイ、そのシステム、およびニューラル・ネットワークを実施するための方法
US10248907B2 (en) Resistive processing unit
US10460237B2 (en) Neuron-centric local learning rate for artificial neural networks to increase performance, learning rate margin, and reduce power consumption
JP7357079B2 (ja) 相変化メモリの閉ループ・プログラミング
US10453527B1 (en) In-cell differential read-out circuitry for reading signed weight values in resistive processing unit architecture
US20180253642A1 (en) Resistive processing unit with hysteretic updates for neural network training
US12229680B2 (en) Neural network accelerators resilient to conductance drift
US11562249B2 (en) DNN training with asymmetric RPU devices
US11056185B2 (en) Apparatus for deep learning operations on resistive crossbar array
JP2021503649A5 (cg-RX-API-DMAC7.html)
Woźniak et al. Neuromorphic architecture with 1M memristive synapses for detection of weakly correlated inputs
CN117136363B (zh) 深度神经网络训练

Legal Events

Date Code Title Description
A521 Request for written amendment filed

Free format text: JAPANESE INTERMEDIATE CODE: A523

Effective date: 20210122

A621 Written request for application examination

Free format text: JAPANESE INTERMEDIATE CODE: A621

Effective date: 20210423

RD04 Notification of resignation of power of attorney

Free format text: JAPANESE INTERMEDIATE CODE: A7424

Effective date: 20220502

A131 Notification of reasons for refusal

Free format text: JAPANESE INTERMEDIATE CODE: A131

Effective date: 20220517

A521 Request for written amendment filed

Free format text: JAPANESE INTERMEDIATE CODE: A523

Effective date: 20220810

TRDD Decision of grant or rejection written
A01 Written decision to grant a patent or to grant a registration (utility model)

Free format text: JAPANESE INTERMEDIATE CODE: A01

Effective date: 20220830

A61 First payment of annual fees (during grant procedure)

Free format text: JAPANESE INTERMEDIATE CODE: A61

Effective date: 20220907

R150 Certificate of patent or registration of utility model

Ref document number: 7139426

Country of ref document: JP

Free format text: JAPANESE INTERMEDIATE CODE: R150