GB2606600B - An efficient method for VLSI implementation of useful neural network activation functions - Google Patents

An efficient method for VLSI implementation of useful neural network activation functions Download PDF

Info

Publication number
GB2606600B
GB2606600B GB2116839.8A GB202116839A GB2606600B GB 2606600 B GB2606600 B GB 2606600B GB 202116839 A GB202116839 A GB 202116839A GB 2606600 B GB2606600 B GB 2606600B
Authority
GB
United Kingdom
Prior art keywords
neural network
efficient method
activation functions
network activation
vlsi implementation
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
GB2116839.8A
Other versions
GB2606600A (en
GB202116839D0 (en
Inventor
Sawada Jun
D Flickner Myron
Stephen Cassidy Andrew
Vernon Arthur John
Datta Pallab
S Modha Dharmendra
Kyle Esser Steven
Seisho Taba Brian
Klamo Jennifer
Appuswamy Rathinakumar
Akopyan Filipp
Ortega Otero Carlos
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
International Business Machines Corp
Original Assignee
International Business Machines Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by International Business Machines Corp filed Critical International Business Machines Corp
Publication of GB202116839D0 publication Critical patent/GB202116839D0/en
Publication of GB2606600A publication Critical patent/GB2606600A/en
Application granted granted Critical
Publication of GB2606600B publication Critical patent/GB2606600B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/06—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons
    • G06N3/063—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons using electronic means
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F15/00—Digital computers in general; Data processing equipment in general
    • G06F15/16—Combinations of two or more digital computers each having at least an arithmetic unit, a program unit and a register, e.g. for a simultaneous processing of several programs
    • G06F15/163—Interprocessor communication
    • G06F15/173—Interprocessor communication using an interconnection network, e.g. matrix, shuffle, pyramid, star, snowflake
    • G06F15/17356—Indirect interconnection networks
    • G06F15/17368—Indirect interconnection networks non hierarchical topologies
    • G06F15/17381—Two dimensional, e.g. mesh, torus
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F15/00—Digital computers in general; Data processing equipment in general
    • G06F15/76—Architectures of general purpose stored program computers
    • G06F15/78—Architectures of general purpose stored program computers comprising a single central processing unit
    • G06F15/7807—System on chip, i.e. computer system on a single chip; System in package, i.e. computer system on one or more chips in a single package
    • G06F15/7825—Globally asynchronous, locally synchronous, e.g. network on chip
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/042—Knowledge-based neural networks; Logical representations of neural networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/048—Activation functions
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N5/00—Computing arrangements using knowledge-based models
    • G06N5/04—Inference or reasoning models

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Biomedical Technology (AREA)
  • Health & Medical Sciences (AREA)
  • Biophysics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Software Systems (AREA)
  • Mathematical Physics (AREA)
  • Computing Systems (AREA)
  • Data Mining & Analysis (AREA)
  • Computational Linguistics (AREA)
  • Evolutionary Computation (AREA)
  • Artificial Intelligence (AREA)
  • Molecular Biology (AREA)
  • General Health & Medical Sciences (AREA)
  • Computer Hardware Design (AREA)
  • Neurology (AREA)
  • Microelectronics & Electronic Packaging (AREA)
  • Complex Calculations (AREA)
  • Image Processing (AREA)
GB2116839.8A 2020-12-08 2021-11-23 An efficient method for VLSI implementation of useful neural network activation functions Active GB2606600B (en)

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
US17/115,285 US20220180177A1 (en) 2020-12-08 2020-12-08 An efficient method for vlsi implementation of useful neural network activation functions

Publications (3)

Publication Number Publication Date
GB202116839D0 GB202116839D0 (en) 2022-01-05
GB2606600A GB2606600A (en) 2022-11-16
GB2606600B true GB2606600B (en) 2024-05-08

Family

ID=79163998

Family Applications (1)

Application Number Title Priority Date Filing Date
GB2116839.8A Active GB2606600B (en) 2020-12-08 2021-11-23 An efficient method for VLSI implementation of useful neural network activation functions

Country Status (5)

Country Link
US (1) US20220180177A1 (en)
JP (1) JP2022091126A (en)
CN (1) CN114611682A (en)
DE (1) DE102021128932A1 (en)
GB (1) GB2606600B (en)

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20170102921A1 (en) * 2015-10-08 2017-04-13 Via Alliance Semiconductor Co., Ltd. Apparatus employing user-specified binary point fixed point arithmetic
KR20190051755A (en) * 2017-11-07 2019-05-15 삼성전자주식회사 Method and apparatus for learning low-precision neural network

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10037306B2 (en) * 2016-09-01 2018-07-31 Qualcomm Incorporated Approximation of non-linear functions in fixed point using look-up tables
US10127494B1 (en) * 2017-08-02 2018-11-13 Google Llc Neural network crossbar stack
US11816446B2 (en) * 2019-11-27 2023-11-14 Amazon Technologies, Inc. Systolic array component combining multiple integer and floating-point data types

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20170102921A1 (en) * 2015-10-08 2017-04-13 Via Alliance Semiconductor Co., Ltd. Apparatus employing user-specified binary point fixed point arithmetic
KR20190051755A (en) * 2017-11-07 2019-05-15 삼성전자주식회사 Method and apparatus for learning low-precision neural network
US11270187B2 (en) * 2017-11-07 2022-03-08 Samsung Electronics Co., Ltd Method and apparatus for learning low-precision neural network that combines weight quantization and activation quantization

Also Published As

Publication number Publication date
GB2606600A (en) 2022-11-16
GB202116839D0 (en) 2022-01-05
DE102021128932A1 (en) 2022-06-09
JP2022091126A (en) 2022-06-20
CN114611682A (en) 2022-06-10
US20220180177A1 (en) 2022-06-09

Similar Documents

Publication Publication Date Title
EP4080416A4 (en) Adaptive search method and apparatus for neural network
EP3817462A4 (en) Method and apparatus for implementing network synchronization
EP3596880A4 (en) Method and apparatus for access control in distributed blockchain-based internet of things (iot) network
EP4123513A4 (en) Fixed-point method and apparatus for neural network
EP3939351C0 (en) Apparatus and method for fast communication between master and slave
EP3780498A4 (en) Network deployment method and apparatus
EP3676974A4 (en) System and method for prediction of communications link quality
EP3637885A4 (en) Method and apparatus for processing uplink frequency shift of communication system
EP3466020B8 (en) A method and an apparatus for publishing assertions in a distributed database of a mobile telecommunication network and for personalising internet of things devices.
EP4358594A4 (en) Network camping-on method and apparatus
KR102214837B9 (en) Convolution neural network parameter optimization method neural network computing method and apparatus
EP3817286A4 (en) Method and apparatus for generating network topology
KR102247896B9 (en) Convolution neural network parameter optimization method neural network computing method and apparatus
EP3739909A4 (en) Method and apparatus for network self-organization
EP4131871A4 (en) Method and apparatus for generating network topology
EP4024659A4 (en) Method and apparatus for generating physical network topological graph
GB2606600B (en) An efficient method for VLSI implementation of useful neural network activation functions
EP3907662A4 (en) Method and apparatus for processing neural network model
EP3444758A4 (en) Discrete data representation-supporting apparatus and method for back-training of artificial neural network
EP4148627A4 (en) Neural network scheduling method and apparatus
GB202208958D0 (en) Hardware implementation of an attention-based neural network
EP4249625A4 (en) Method and apparatus for treating metallic member
EP3914561A4 (en) Method and apparatus for producing hydroxyl radicals
EP3750271A4 (en) Method and apparatus for adaptive scheduling and transmission
GB2610430B (en) Method and apparatus for coating

Legal Events

Date Code Title Description
746 Register noted 'licences of right' (sect. 46/1977)

Effective date: 20240530