GB2606600B - An efficient method for VLSI implementation of useful neural network activation functions - Google Patents

An efficient method for VLSI implementation of useful neural network activation functions Download PDF

Info

Publication number
GB2606600B
GB2606600B GB2116839.8A GB202116839A GB2606600B GB 2606600 B GB2606600 B GB 2606600B GB 202116839 A GB202116839 A GB 202116839A GB 2606600 B GB2606600 B GB 2606600B
Authority
GB
United Kingdom
Prior art keywords
neural network
efficient method
activation functions
network activation
vlsi implementation
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
GB2116839.8A
Other versions
GB202116839D0 (en
GB2606600A (en
Inventor
Sawada Jun
D Flickner Myron
Stephen Cassidy Andrew
Vernon Arthur John
Datta Pallab
S Modha Dharmendra
Kyle Esser Steven
Seisho Taba Brian
Klamo Jennifer
Appuswamy Rathinakumar
Akopyan Filipp
Ortega Otero Carlos
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
International Business Machines Corp
Original Assignee
International Business Machines Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by International Business Machines Corp filed Critical International Business Machines Corp
Publication of GB202116839D0 publication Critical patent/GB202116839D0/en
Publication of GB2606600A publication Critical patent/GB2606600A/en
Application granted granted Critical
Publication of GB2606600B publication Critical patent/GB2606600B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/06—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons
    • G06N3/063—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons using electronic means
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F15/00—Digital computers in general; Data processing equipment in general
    • G06F15/16—Combinations of two or more digital computers each having at least an arithmetic unit, a program unit and a register, e.g. for a simultaneous processing of several programs
    • G06F15/163—Interprocessor communication
    • G06F15/173—Interprocessor communication using an interconnection network, e.g. matrix, shuffle, pyramid, star, snowflake
    • G06F15/17356—Indirect interconnection networks
    • G06F15/17368—Indirect interconnection networks non hierarchical topologies
    • G06F15/17381—Two dimensional, e.g. mesh, torus
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F15/00—Digital computers in general; Data processing equipment in general
    • G06F15/76—Architectures of general purpose stored program computers
    • G06F15/78—Architectures of general purpose stored program computers comprising a single central processing unit
    • G06F15/7807—System on chip, i.e. computer system on a single chip; System in package, i.e. computer system on one or more chips in a single package
    • G06F15/7825—Globally asynchronous, locally synchronous, e.g. network on chip
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/042—Knowledge-based neural networks; Logical representations of neural networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/048—Activation functions
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N5/00—Computing arrangements using knowledge-based models
    • G06N5/04—Inference or reasoning models

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • General Engineering & Computer Science (AREA)
  • Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • General Physics & Mathematics (AREA)
  • Mathematical Physics (AREA)
  • Computing Systems (AREA)
  • Software Systems (AREA)
  • Evolutionary Computation (AREA)
  • Data Mining & Analysis (AREA)
  • Computational Linguistics (AREA)
  • Artificial Intelligence (AREA)
  • General Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Computer Hardware Design (AREA)
  • Neurology (AREA)
  • Microelectronics & Electronic Packaging (AREA)
  • Complex Calculations (AREA)
  • Image Processing (AREA)
GB2116839.8A 2020-12-08 2021-11-23 An efficient method for VLSI implementation of useful neural network activation functions Active GB2606600B (en)

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
US17/115,285 US20220180177A1 (en) 2020-12-08 2020-12-08 An efficient method for vlsi implementation of useful neural network activation functions

Publications (3)

Publication Number Publication Date
GB202116839D0 GB202116839D0 (en) 2022-01-05
GB2606600A GB2606600A (en) 2022-11-16
GB2606600B true GB2606600B (en) 2024-05-08

Family

ID=79163998

Family Applications (1)

Application Number Title Priority Date Filing Date
GB2116839.8A Active GB2606600B (en) 2020-12-08 2021-11-23 An efficient method for VLSI implementation of useful neural network activation functions

Country Status (5)

Country Link
US (1) US20220180177A1 (en)
JP (1) JP2022091126A (en)
CN (1) CN114611682A (en)
DE (1) DE102021128932A1 (en)
GB (1) GB2606600B (en)

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20170102921A1 (en) * 2015-10-08 2017-04-13 Via Alliance Semiconductor Co., Ltd. Apparatus employing user-specified binary point fixed point arithmetic
KR20190051755A (en) * 2017-11-07 2019-05-15 삼성전자주식회사 Method and apparatus for learning low-precision neural network

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10037306B2 (en) * 2016-09-01 2018-07-31 Qualcomm Incorporated Approximation of non-linear functions in fixed point using look-up tables
US11816446B2 (en) * 2019-11-27 2023-11-14 Amazon Technologies, Inc. Systolic array component combining multiple integer and floating-point data types

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20170102921A1 (en) * 2015-10-08 2017-04-13 Via Alliance Semiconductor Co., Ltd. Apparatus employing user-specified binary point fixed point arithmetic
KR20190051755A (en) * 2017-11-07 2019-05-15 삼성전자주식회사 Method and apparatus for learning low-precision neural network
US11270187B2 (en) * 2017-11-07 2022-03-08 Samsung Electronics Co., Ltd Method and apparatus for learning low-precision neural network that combines weight quantization and activation quantization

Also Published As

Publication number Publication date
GB202116839D0 (en) 2022-01-05
GB2606600A (en) 2022-11-16
JP2022091126A (en) 2022-06-20
CN114611682A (en) 2022-06-10
US20220180177A1 (en) 2022-06-09
DE102021128932A1 (en) 2022-06-09

Similar Documents

Publication Publication Date Title
EP4080416A4 (en) Adaptive search method and apparatus for neural network
EP3817462A4 (en) Method and apparatus for implementing network synchronization
EP3780498A4 (en) Network deployment method and apparatus
EP3676974A4 (en) System and method for prediction of communications link quality
EP3939351A4 (en) Apparatus and method for fast communication between master and slave
EP3466020B8 (en) A method and an apparatus for publishing assertions in a distributed database of a mobile telecommunication network and for personalising internet of things devices.
EP4128580A4 (en) System and method for adaptive communications
KR102214837B9 (en) Convolution neural network parameter optimization method neural network computing method and apparatus
EP3817286A4 (en) Method and apparatus for generating network topology
EP3598704A4 (en) Method and apparatus for establishing domain-level topology and network system
EP3860161A4 (en) Internet of vehicles communication method, positioning method, and internet of vehicles communication apparatus
KR102247896B9 (en) Convolution neural network parameter optimization method neural network computing method and apparatus
EP4236232A4 (en) Method for configuring network parameter of internet of things device, internet of things device and internet of things system
EP4131871A4 (en) Method and apparatus for generating network topology
GB2577317B (en) A method of, and apparatus for, improved satellite communications
GB2606600B (en) An efficient method for VLSI implementation of useful neural network activation functions
EP3907662A4 (en) Method and apparatus for processing neural network model
KR102192461B9 (en) Apparatus and method for learning neural network capable of modeling uncertainty
IL308059A (en) System and method for secure internet communications
EP4249625A4 (en) Method and apparatus for treating metallic member
EP4163827A4 (en) Method and device for acquiring neural network
EP4112158A4 (en) Method and apparatus for producing emulsion
EP3750271A4 (en) Method and apparatus for adaptive scheduling and transmission
SG10201807451WA (en) System and method for simulating a network of vehicles
GB2610430B (en) Method and apparatus for coating