CN105229675B - 尖峰网络的高效硬件实现 - Google Patents

尖峰网络的高效硬件实现 Download PDF

Info

Publication number
CN105229675B
CN105229675B CN201480029027.1A CN201480029027A CN105229675B CN 105229675 B CN105229675 B CN 105229675B CN 201480029027 A CN201480029027 A CN 201480029027A CN 105229675 B CN105229675 B CN 105229675B
Authority
CN
China
Prior art keywords
memory
artificial
nervous system
neuron
neurons
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201480029027.1A
Other languages
English (en)
Chinese (zh)
Other versions
CN105229675A (zh
Inventor
J·A·莱文
V·兰甘
E·C·马隆
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Qualcomm Inc
Original Assignee
Qualcomm Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Qualcomm Inc filed Critical Qualcomm Inc
Publication of CN105229675A publication Critical patent/CN105229675A/zh
Application granted granted Critical
Publication of CN105229675B publication Critical patent/CN105229675B/zh
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • GPHYSICS
    • G06COMPUTING OR CALCULATING; COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/06Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons
    • G06N3/063Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons using electronic means
    • GPHYSICS
    • G06COMPUTING OR CALCULATING; COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/04Architecture, e.g. interconnection topology
    • G06N3/049Temporal neural networks, e.g. delay elements, oscillating neurons or pulsed inputs
    • GPHYSICS
    • G06COMPUTING OR CALCULATING; COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/04Architecture, e.g. interconnection topology
    • G06N3/0495Quantised networks; Sparse networks; Compressed networks
    • GPHYSICS
    • G06COMPUTING OR CALCULATING; COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/04Architecture, e.g. interconnection topology
    • G06N3/0499Feedforward networks
    • GPHYSICS
    • G06COMPUTING OR CALCULATING; COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/08Learning methods
    • GPHYSICS
    • G06COMPUTING OR CALCULATING; COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/08Learning methods
    • G06N3/082Learning methods modifying the architecture, e.g. adding, deleting or silencing nodes or connections

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • General Health & Medical Sciences (AREA)
  • General Physics & Mathematics (AREA)
  • Evolutionary Computation (AREA)
  • Computational Linguistics (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Artificial Intelligence (AREA)
  • Neurology (AREA)
  • Image Analysis (AREA)
  • Stored Programmes (AREA)
CN201480029027.1A 2013-05-21 2014-05-20 尖峰网络的高效硬件实现 Active CN105229675B (zh)

Applications Claiming Priority (13)

Application Number Priority Date Filing Date Title
US201361825657P 2013-05-21 2013-05-21
US61/825,657 2013-05-21
US201361862741P 2013-08-06 2013-08-06
US201361862734P 2013-08-06 2013-08-06
US201361862714P 2013-08-06 2013-08-06
US61/862,741 2013-08-06
US61/862,714 2013-08-06
US61/862,734 2013-08-06
US201461932364P 2014-01-28 2014-01-28
US61/932,364 2014-01-28
US14/267,005 2014-05-01
US14/267,005 US9542643B2 (en) 2013-05-21 2014-05-01 Efficient hardware implementation of spiking networks
PCT/US2014/038841 WO2014189970A2 (en) 2013-05-21 2014-05-20 Efficient hardware implementation of spiking networks

Publications (2)

Publication Number Publication Date
CN105229675A CN105229675A (zh) 2016-01-06
CN105229675B true CN105229675B (zh) 2018-02-06

Family

ID=51022429

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201480029027.1A Active CN105229675B (zh) 2013-05-21 2014-05-20 尖峰网络的高效硬件实现

Country Status (6)

Country Link
US (1) US9542643B2 (enExample)
EP (1) EP3000076A2 (enExample)
JP (1) JP6130056B2 (enExample)
KR (1) KR101793011B1 (enExample)
CN (1) CN105229675B (enExample)
WO (1) WO2014189970A2 (enExample)

Families Citing this family (65)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8712940B2 (en) * 2011-05-31 2014-04-29 International Business Machines Corporation Structural plasticity in spiking neural networks with symmetric dual of an electronic neuron
US9087301B2 (en) 2012-12-21 2015-07-21 International Business Machines Corporation Hardware architecture for simulating a neural network of neurons
US9053429B2 (en) 2012-12-21 2015-06-09 International Business Machines Corporation Mapping neural dynamics of a neural model on to a coarsely grained look-up table
US9330355B2 (en) 2013-08-06 2016-05-03 Qualcomm Incorporated Computed synapses for neuromorphic systems
EP3035249B1 (en) 2014-12-19 2019-11-27 Intel Corporation Method and apparatus for distributed and cooperative computation in artificial neural networks
US10552731B2 (en) 2015-12-28 2020-02-04 International Business Machines Corporation Digital STDP synapse and LIF neuron-based neuromorphic system
US10891543B2 (en) 2015-12-28 2021-01-12 Samsung Electronics Co., Ltd. LUT based synapse weight update scheme in STDP neuromorphic systems
US10748058B2 (en) 2015-12-28 2020-08-18 International Business Machines Corporation LUT based neuron membrane potential update scheme in STDP neuromorphic systems
KR102285800B1 (ko) * 2016-03-18 2021-08-04 가부시키가이샤 한도오따이 에네루기 켄큐쇼 반도체 장치 및 이를 사용한 시스템
US11003984B2 (en) 2016-05-31 2021-05-11 Samsung Electronics Co., Ltd. Timing sequence for digital STDP synapse and LIF neuron-based neuromorphic system
CN111310893B (zh) * 2016-08-05 2023-11-21 中科寒武纪科技股份有限公司 一种用于执行神经网络运算的装置及方法
US11501130B2 (en) * 2016-09-09 2022-11-15 SK Hynix Inc. Neural network hardware accelerator architectures and operating method thereof
US10586147B2 (en) * 2016-09-22 2020-03-10 Intel Corporation Neuromorphic computing device, memory device, system, and method to maintain a spike history for neurons in a neuromorphic computing environment
US11281963B2 (en) 2016-09-26 2022-03-22 Intel Corporation Programmable neuron core with on-chip learning and stochastic time step control
US10489702B2 (en) * 2016-10-14 2019-11-26 Intel Corporation Hybrid compression scheme for efficient storage of synaptic weights in hardware neuromorphic cores
KR101997975B1 (ko) * 2016-12-01 2019-07-08 한국과학기술원 신경망 시스템을 이용한 정보의 장기, 단기, 및 하이브리드 기억을 위한 방법
US20180174027A1 (en) * 2016-12-20 2018-06-21 Michael I. Davies Multi-compartment dendrites in neuromorphic computing
US10248906B2 (en) * 2016-12-28 2019-04-02 Intel Corporation Neuromorphic circuits for storing and generating connectivity information
US10140574B2 (en) * 2016-12-31 2018-11-27 Via Alliance Semiconductor Co., Ltd Neural network unit with segmentable array width rotator and re-shapeable weight memory to match segment width to provide common weights to multiple rotator segments
KR101918102B1 (ko) 2017-02-14 2019-02-08 한국과학기술연구원 플로팅 게이트 적분기를 이용한 고집적 저전력 소모 인공뉴런 회로 및 이를 포함하는 뉴로모르픽 시스템, 이의 제어 방법
US10922608B2 (en) * 2017-03-08 2021-02-16 Arm Ltd Spiking neural network
CN106934457B (zh) * 2017-03-08 2019-12-06 杭州领芯电子有限公司 一种可灵活时分复用的脉冲神经元实现架构
FR3064384B1 (fr) * 2017-03-23 2019-05-03 Stmicroelectronics Sa Bloc refractaire pour dispositif integre de neurone artificiel
US10679119B2 (en) * 2017-03-24 2020-06-09 Intel Corporation Handling signal saturation in spiking neural networks
US12430543B2 (en) 2017-04-04 2025-09-30 Hailo Technologies Ltd. Structured sparsity guided training in an artificial neural network
US11238334B2 (en) 2017-04-04 2022-02-01 Hailo Technologies Ltd. System and method of input alignment for efficient vector operations in an artificial neural network
US10387298B2 (en) 2017-04-04 2019-08-20 Hailo Technologies Ltd Artificial neural network incorporating emphasis and focus techniques
US11544545B2 (en) 2017-04-04 2023-01-03 Hailo Technologies Ltd. Structured activation based sparsity in an artificial neural network
US11615297B2 (en) 2017-04-04 2023-03-28 Hailo Technologies Ltd. Structured weight based sparsity in an artificial neural network compiler
US11551028B2 (en) 2017-04-04 2023-01-10 Hailo Technologies Ltd. Structured weight based sparsity in an artificial neural network
US10909449B2 (en) * 2017-04-14 2021-02-02 Samsung Electronics Co., Ltd. Monolithic multi-bit weight cell for neuromorphic computing
US11308387B2 (en) 2017-05-09 2022-04-19 Samsung Electronics Co., Ltd. STDP with synaptic fatigue for learning of spike-time-coded patterns in the presence of parallel rate-coding
US10671349B2 (en) 2017-07-24 2020-06-02 Tesla, Inc. Accelerated mathematical engine
US11157287B2 (en) 2017-07-24 2021-10-26 Tesla, Inc. Computational array microprocessor system with variable latency memory access
US11409692B2 (en) 2017-07-24 2022-08-09 Tesla, Inc. Vector computational unit
US11157441B2 (en) 2017-07-24 2021-10-26 Tesla, Inc. Computational array microprocessor system using non-consecutive data formatting
US11893393B2 (en) 2017-07-24 2024-02-06 Tesla, Inc. Computational array microprocessor system with hardware arbiter managing memory requests
GB2568776B (en) * 2017-08-11 2020-10-28 Google Llc Neural network accelerator with parameters resident on chip
CN107742153B (zh) * 2017-10-20 2020-02-21 华中科技大学 一种基于忆阻器的具有稳态可塑性的神经元电路
US10452540B2 (en) * 2017-10-20 2019-10-22 International Business Machines Corporation Memory-mapped interface for message passing computing systems
US11561791B2 (en) 2018-02-01 2023-01-24 Tesla, Inc. Vector computational unit receiving data elements in parallel from a last row of a computational array
US11366998B2 (en) 2018-03-27 2022-06-21 Intel Corporation Neuromorphic accelerator multitasking
JP7490572B2 (ja) * 2018-05-08 2024-05-27 三星電子株式会社 ニューラル・ネットワーク処理要素
EP3605401A1 (en) * 2018-07-31 2020-02-05 GrAl Matter Labs S.A.S. Data processing module, data processing system and data processing method
US11200484B2 (en) 2018-09-06 2021-12-14 International Business Machines Corporation Probability propagation over factor graphs
CN109886416A (zh) * 2019-02-01 2019-06-14 京微齐力(北京)科技有限公司 集成人工智能模块的系统芯片及机器学习方法
EP3928319A4 (en) 2019-02-22 2022-11-30 Micron Technology, Inc. STORAGE DEVICE INTERFACE AND METHOD
KR102351087B1 (ko) 2019-06-04 2022-01-14 주식회사 딥엑스 인공신경망의 데이터 로컬리티 기반의 데이터 캐슁을 이용하여 고속의 인공신경망 오퍼레이션을 지원하는 데이터 관리 장치
KR102301041B1 (ko) * 2019-07-04 2021-09-14 한국과학기술연구원 뉴로모픽 장치
US11934946B2 (en) 2019-08-01 2024-03-19 International Business Machines Corporation Learning and recall in spiking neural networks
US11054997B2 (en) * 2019-08-12 2021-07-06 Micron Technology, Inc. Artificial neural networks in memory
KR102832599B1 (ko) 2019-11-15 2025-07-14 삼성전자주식회사 메모리 기반의 뉴로모픽 장치
CN115398448A (zh) 2019-12-27 2022-11-25 美光科技公司 神经形态存储器装置和方法
CN114930452A (zh) 2019-12-30 2022-08-19 美光科技公司 存储器装置接口和方法
CN114902332A (zh) 2019-12-31 2022-08-12 美光科技公司 存储器模块多端口缓冲技术
CN111461312B (zh) * 2020-03-26 2022-04-22 华中科技大学 一种基于忆阻器随机丢弃神经元
KR102860886B1 (ko) 2020-06-01 2025-09-18 삼성전자주식회사 스케줄러, 스케줄러의 동작 방법 및 이를 포함한 가속기 시스템
KR102204107B1 (ko) * 2020-07-16 2021-01-18 세종대학교산학협력단 뉴런의 임계값에 STDP(Spiking Timing Dependent Plasticity)를 적용한 스파이킹 뉴럴 네트워크(Spiking Neural Network)
US11221929B1 (en) 2020-09-29 2022-01-11 Hailo Technologies Ltd. Data stream fault detection mechanism in an artificial neural network processor
US11263077B1 (en) 2020-09-29 2022-03-01 Hailo Technologies Ltd. Neural network intermediate results safety mechanism in an artificial neural network processor
US11237894B1 (en) 2020-09-29 2022-02-01 Hailo Technologies Ltd. Layer control unit instruction addressing safety mechanism in an artificial neural network processor
US11874900B2 (en) 2020-09-29 2024-01-16 Hailo Technologies Ltd. Cluster interlayer safety mechanism in an artificial neural network processor
US11811421B2 (en) 2020-09-29 2023-11-07 Hailo Technologies Ltd. Weights safety mechanism in an artificial neural network processor
US12248367B2 (en) 2020-09-29 2025-03-11 Hailo Technologies Ltd. Software defined redundant allocation safety mechanism in an artificial neural network processor
CN114037050B (zh) * 2021-10-21 2022-08-16 大连理工大学 一种基于脉冲神经网络内在可塑性的机器人退化环境避障方法

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2006000103A1 (en) * 2004-06-29 2006-01-05 Universite De Sherbrooke Spiking neural network and use thereof
US7043466B2 (en) * 1990-01-24 2006-05-09 Renesas Technology Corp. Neural network processing system using semiconductor memories
CN102341815A (zh) * 2009-05-21 2012-02-01 国际商业机器公司 使用记忆切换元件的具有尖峰-时间依赖塑性的电子学习突触
CN102906767A (zh) * 2010-06-30 2013-01-30 国际商业机器公司 用于时空联合存储器的标准尖峰神经网络

Family Cites Families (30)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7085749B2 (en) 2001-05-31 2006-08-01 Canon Kabushiki Kaisha Pulse signal circuit, parallel processing circuit, pattern recognition system, and image input system
US7430546B1 (en) 2003-06-07 2008-09-30 Roland Erwin Suri Applications of an algorithm that mimics cortical processing
US7627540B2 (en) 2005-06-28 2009-12-01 Neurosciences Research Foundation, Inc. Addressing scheme for neural modeling and brain-based devices using special purpose processor
US8103602B2 (en) 2006-12-29 2012-01-24 Neurosciences Research Foundation, Inc. Solving the distal reward problem through linkage of STDP and dopamine signaling
US7818273B2 (en) 2007-09-18 2010-10-19 International Business Machines Corporation System and method for cortical simulation
GB0811057D0 (en) * 2008-06-17 2008-07-23 Univ Ulster Artificial neural network architecture
WO2010025130A1 (en) 2008-08-28 2010-03-04 Massachusetts Institute Of Technology Circuits and methods representative of spike timing dependent plasticity of neurons
US8630966B2 (en) 2009-01-27 2014-01-14 Salk Institute For Biological Studies Temporally dynamic artificial neural networks
US8510244B2 (en) 2009-03-20 2013-08-13 ISC8 Inc. Apparatus comprising artificial neuronal assembly
US20100312736A1 (en) 2009-06-05 2010-12-09 The Regents Of The University Of California Critical Branching Neural Computation Apparatus and Methods
US8527438B2 (en) 2009-12-22 2013-09-03 International Business Machines Corporation Producing spike-timing dependent plasticity in an ultra-dense synapse cross-bar array
US9269042B2 (en) 2010-09-30 2016-02-23 International Business Machines Corporation Producing spike-timing dependent plasticity in a neuromorphic network utilizing phase change synaptic devices
US8655813B2 (en) 2010-12-30 2014-02-18 International Business Machines Corporation Synaptic weight normalized spiking neuronal networks
US9147156B2 (en) 2011-09-21 2015-09-29 Qualcomm Technologies Inc. Apparatus and methods for synaptic update in a pulse-coded network
KR101888468B1 (ko) 2011-06-08 2018-08-16 삼성전자주식회사 Stdp 기능 셀을 위한 시냅스, stdp 기능 셀 및 stdp 기능 셀을 이용한 뉴로모픽 회로
US9104973B2 (en) 2011-09-21 2015-08-11 Qualcomm Technologies Inc. Elementary network description for neuromorphic systems with plurality of doublets wherein doublet events rules are executed in parallel
US9460387B2 (en) * 2011-09-21 2016-10-04 Qualcomm Technologies Inc. Apparatus and methods for implementing event-based updates in neuron networks
US8725662B2 (en) 2011-09-21 2014-05-13 Brain Corporation Apparatus and method for partial evaluation of synaptic updates based on system events
US9412064B2 (en) 2011-08-17 2016-08-09 Qualcomm Technologies Inc. Event-based communication in spiking neuron networks communicating a neural activity payload with an efficacy update
US8909576B2 (en) 2011-09-16 2014-12-09 International Business Machines Corporation Neuromorphic event-driven neural computing architecture in a scalable neural network
US9092735B2 (en) * 2011-09-21 2015-07-28 Qualcomm Incorporated Method and apparatus for structural delay plasticity in spiking neural networks
US9111222B2 (en) * 2011-11-09 2015-08-18 Qualcomm Incorporated Method and apparatus for switching the binary state of a location in memory in a probabilistic manner to store synaptic weights of a neural network
US8832010B2 (en) 2012-01-04 2014-09-09 International Business Machines Corporation Electronic synapses from stochastic binary memory devices
US9218563B2 (en) 2012-10-25 2015-12-22 Brain Corporation Spiking neuron sensory processing apparatus and methods for saliency detection
US9177245B2 (en) 2013-02-08 2015-11-03 Qualcomm Technologies Inc. Spiking network apparatus and method with bimodal spike-timing dependent plasticity
US9436908B2 (en) 2013-02-22 2016-09-06 Qualcomm Technologies Inc. Apparatus and methods for rate-modulated plasticity in a neuron network
US20140351186A1 (en) 2013-05-21 2014-11-27 Qualcomm Incorporated Spike time windowing for implementing spike-timing dependent plasticity (stdp)
US9436909B2 (en) 2013-06-19 2016-09-06 Brain Corporation Increased dynamic range artificial neuron network apparatus and methods
US9330355B2 (en) 2013-08-06 2016-05-03 Qualcomm Incorporated Computed synapses for neuromorphic systems
US20150046381A1 (en) 2013-08-06 2015-02-12 Qualcomm Incorporated Implementing delays between neurons in an artificial nervous system

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7043466B2 (en) * 1990-01-24 2006-05-09 Renesas Technology Corp. Neural network processing system using semiconductor memories
WO2006000103A1 (en) * 2004-06-29 2006-01-05 Universite De Sherbrooke Spiking neural network and use thereof
CN102341815A (zh) * 2009-05-21 2012-02-01 国际商业机器公司 使用记忆切换元件的具有尖峰-时间依赖塑性的电子学习突触
CN102906767A (zh) * 2010-06-30 2013-01-30 国际商业机器公司 用于时空联合存储器的标准尖峰神经网络

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
"A SIMD/Dataflow Architecture for a Neurocomputer for Spike-Processing Neural Networks (NESPINN)";Axel Jahnke etal.;《IEEE》;19960228;第232-237页 *

Also Published As

Publication number Publication date
US9542643B2 (en) 2017-01-10
WO2014189970A3 (en) 2015-04-09
JP2016522495A (ja) 2016-07-28
US20140351190A1 (en) 2014-11-27
KR101793011B1 (ko) 2017-11-02
WO2014189970A2 (en) 2014-11-27
KR20160010477A (ko) 2016-01-27
JP6130056B2 (ja) 2017-05-17
CN105229675A (zh) 2016-01-06
EP3000076A2 (en) 2016-03-30

Similar Documents

Publication Publication Date Title
CN105229675B (zh) 尖峰网络的高效硬件实现
CN105934766B (zh) 用阴影网络来监视神经网络
US10339041B2 (en) Shared memory architecture for a neural simulator
CN105659262A (zh) 在尖峰神经网络中使用重放来实现突触学习
CN105580031B (zh) 在多维范围上对包括可分离子系统的系统的评估
CN106030622A (zh) 原位神经网络协同处理
CN105874477A (zh) 配置稀疏神经网络
CN105556543B (zh) 针对神经模型的群组标签的实现方案的方法和装置
CN106164940A (zh) 在尖峰神经网络中通过全局标量值来调制可塑性
US9672464B2 (en) Method and apparatus for efficient implementation of common neuron models
US20150278685A1 (en) Probabilistic representation of large sequences using spiking neural network
CN106104585A (zh) 经由阈下调制的模拟信号重构和识别
CN106796667A (zh) 动态空间目标选择
CN106133763B (zh) 可塑性突触管理
CN105659261A (zh) 尖峰神经元网络中的拥塞避免
CN105659260B (zh) 动态地指派和检查突触延迟
US9342782B2 (en) Stochastic delay plasticity
US9418332B2 (en) Post ghost plasticity
WO2014197175A2 (en) Efficient implementation of neural population diversity in neural system
CN105612536A (zh) 远程地控制和监视神经模型执行的方法和装置
US20150278684A1 (en) Time synchronization of spiking neuron models on multiple nodes

Legal Events

Date Code Title Description
C06 Publication
PB01 Publication
C10 Entry into substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant