KR102578508B1 - 호스트 전달되는 병합된 가중치들 및 계층별 명령어들의 패키지를 사용한 뉴럴 네트워크 가속기에 의한 다중 계층 뉴럴 네트워크 프로세싱 - Google Patents

호스트 전달되는 병합된 가중치들 및 계층별 명령어들의 패키지를 사용한 뉴럴 네트워크 가속기에 의한 다중 계층 뉴럴 네트워크 프로세싱 Download PDF

Info

Publication number
KR102578508B1
KR102578508B1 KR1020207013441A KR20207013441A KR102578508B1 KR 102578508 B1 KR102578508 B1 KR 102578508B1 KR 1020207013441 A KR1020207013441 A KR 1020207013441A KR 20207013441 A KR20207013441 A KR 20207013441A KR 102578508 B1 KR102578508 B1 KR 102578508B1
Authority
KR
South Korea
Prior art keywords
neural network
layer
processing
shared memory
instructions
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
KR1020207013441A
Other languages
English (en)
Korean (ko)
Other versions
KR20200069338A (ko
Inventor
아아론 엔쥐
엘리엇 델라예
에산 가세미
시아오 텡
진드리치 제즈다
용준 우
션 세틀
아시쉬 시라사오
Original Assignee
자일링크스 인코포레이티드
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 자일링크스 인코포레이티드 filed Critical 자일링크스 인코포레이티드
Publication of KR20200069338A publication Critical patent/KR20200069338A/ko
Application granted granted Critical
Publication of KR102578508B1 publication Critical patent/KR102578508B1/ko
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/06—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons
    • G06N3/063—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons using electronic means
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/0464—Convolutional networks [CNN, ConvNet]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Evolutionary Computation (AREA)
  • General Engineering & Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Artificial Intelligence (AREA)
  • General Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • Computational Linguistics (AREA)
  • General Physics & Mathematics (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Neurology (AREA)
  • Advance Control (AREA)
  • Image Analysis (AREA)
  • Complex Calculations (AREA)
KR1020207013441A 2017-10-17 2018-10-16 호스트 전달되는 병합된 가중치들 및 계층별 명령어들의 패키지를 사용한 뉴럴 네트워크 가속기에 의한 다중 계층 뉴럴 네트워크 프로세싱 Active KR102578508B1 (ko)

Applications Claiming Priority (3)

Application Number Priority Date Filing Date Title
US15/785,800 2017-10-17
US15/785,800 US11620490B2 (en) 2017-10-17 2017-10-17 Multi-layer neural network processing by a neural network accelerator using host communicated merged weights and a package of per-layer instructions
PCT/US2018/056112 WO2019079319A1 (en) 2017-10-17 2018-10-16 NEURONAL MULTICOUCHE NETWORK PROCESSING BY A NEURONAL NETWORK ACCELERATOR USING CONTAINED HOST COMMUNICATION WEIGHTS AND A LAYERED INSTRUCTION PACKAGE

Publications (2)

Publication Number Publication Date
KR20200069338A KR20200069338A (ko) 2020-06-16
KR102578508B1 true KR102578508B1 (ko) 2023-09-13

Family

ID=64110172

Family Applications (1)

Application Number Title Priority Date Filing Date
KR1020207013441A Active KR102578508B1 (ko) 2017-10-17 2018-10-16 호스트 전달되는 병합된 가중치들 및 계층별 명령어들의 패키지를 사용한 뉴럴 네트워크 가속기에 의한 다중 계층 뉴럴 네트워크 프로세싱

Country Status (6)

Country Link
US (1) US11620490B2 (https=)
EP (1) EP3698296B1 (https=)
JP (1) JP7196167B2 (https=)
KR (1) KR102578508B1 (https=)
CN (1) CN111226231A (https=)
WO (1) WO2019079319A1 (https=)

Families Citing this family (26)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11037330B2 (en) * 2017-04-08 2021-06-15 Intel Corporation Low rank matrix compression
US11386644B2 (en) * 2017-10-17 2022-07-12 Xilinx, Inc. Image preprocessing for generalized image processing
US10565285B2 (en) * 2017-12-18 2020-02-18 International Business Machines Corporation Processor and memory transparent convolutional lowering and auto zero padding for deep neural network implementations
US11250107B2 (en) * 2019-07-15 2022-02-15 International Business Machines Corporation Method for interfacing with hardware accelerators
US11573828B2 (en) * 2019-09-16 2023-02-07 Nec Corporation Efficient and scalable enclave protection for machine learning programs
US11501145B1 (en) * 2019-09-17 2022-11-15 Amazon Technologies, Inc. Memory operation for systolic array
KR102463123B1 (ko) * 2019-11-29 2022-11-04 한국전자기술연구원 뉴럴 네트워크 가속기의 효율적인 제어, 모니터링 및 소프트웨어 디버깅 방법
US12554962B2 (en) * 2019-12-24 2026-02-17 Intel Corporation Configurable processor element arrays for implementing convolutional neural networks
US11132594B2 (en) * 2020-01-03 2021-09-28 Capital One Services, Llc Systems and methods for producing non-standard shaped cards
US11182159B2 (en) * 2020-02-26 2021-11-23 Google Llc Vector reductions using shared scratchpad memory
CN111461316A (zh) * 2020-03-31 2020-07-28 中科寒武纪科技股份有限公司 计算神经网络的方法、装置、板卡及计算机可读存储介质
CN111461315A (zh) * 2020-03-31 2020-07-28 中科寒武纪科技股份有限公司 计算神经网络的方法、装置、板卡及计算机可读存储介质
US11783163B2 (en) * 2020-06-15 2023-10-10 Arm Limited Hardware accelerator for IM2COL operation
KR102860333B1 (ko) * 2020-06-22 2025-09-16 삼성전자주식회사 가속기, 가속기의 동작 방법 및 이를 포함한 가속기 시스템
KR102859455B1 (ko) * 2020-08-31 2025-09-12 삼성전자주식회사 가속기, 가속기의 동작 방법 및 이를 포함한 전자 장치
KR102893029B1 (ko) * 2020-09-11 2025-11-27 삼성전자주식회사 스토리지 장치 및 스토리지 장치의 동작 방법
CN113485762B (zh) * 2020-09-19 2024-07-26 广东高云半导体科技股份有限公司 用可配置器件卸载计算任务以提高系统性能的方法和装置
US20220108209A1 (en) * 2020-10-05 2022-04-07 Microsoft Technology Licensing, Llc Shared memory spaces in data and model parallelism
CN112613605A (zh) * 2020-12-07 2021-04-06 深兰人工智能(深圳)有限公司 神经网络加速控制方法、装置、电子设备及存储介质
US20220179703A1 (en) * 2020-12-07 2022-06-09 Nvidia Corporation Application programming interface for neural network computation
US20230325648A1 (en) * 2020-12-10 2023-10-12 Neuronix AI Labs Inc. Neural networks processing units activation sparsity removal
CN112580787B (zh) * 2020-12-25 2023-11-17 北京百度网讯科技有限公司 神经网络加速器的数据处理方法、装置、设备及存储介质
US12094531B2 (en) * 2021-01-11 2024-09-17 Micron Technology, Inc. Caching techniques for deep learning accelerator
DE102021202934A1 (de) * 2021-03-25 2022-09-29 Robert Bosch Gesellschaft mit beschränkter Haftung Verfolgung mehrerer Objekte mit neuronalen Netzwerken, lokalen Speichern und einem gemeinsamen Speicher
CN113326479A (zh) * 2021-05-28 2021-08-31 哈尔滨理工大学 一种基于fpga的k均值算法的实现方法
CN121902881A (zh) 2023-04-06 2026-04-21 墨子国际有限公司 用于神经网络加速器的分层片上网络（noc）

Family Cites Families (16)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6346825B1 (en) 2000-10-06 2002-02-12 Xilinx, Inc. Block RAM with configurable data width and parity for use in a field programmable gate array
WO2014204615A2 (en) * 2013-05-22 2014-12-24 Neurala, Inc. Methods and apparatus for iterative nonspecific distributed runtime architecture and its application to cloud intelligence
US10339041B2 (en) * 2013-10-11 2019-07-02 Qualcomm Incorporated Shared memory architecture for a neural simulator
US11099918B2 (en) * 2015-05-11 2021-08-24 Xilinx, Inc. Accelerating algorithms and applications on FPGAs
US10083395B2 (en) * 2015-05-21 2018-09-25 Google Llc Batch processing in a neural network processor
US10417555B2 (en) * 2015-05-29 2019-09-17 Samsung Electronics Co., Ltd. Data-optimized neural network traversal
KR102628902B1 (ko) * 2015-10-28 2024-01-24 구글 엘엘씨 계산 그래프들 프로세싱
US9875104B2 (en) * 2016-02-03 2018-01-23 Google Llc Accessing data in multi-dimensional tensors
US10891538B2 (en) * 2016-08-11 2021-01-12 Nvidia Corporation Sparse convolutional neural network accelerator
CN107239823A (zh) 2016-08-12 2017-10-10 北京深鉴科技有限公司 一种用于实现稀疏神经网络的装置和方法
US10802992B2 (en) * 2016-08-12 2020-10-13 Xilinx Technology Beijing Limited Combining CPU and special accelerator for implementing an artificial neural network
US10489702B2 (en) * 2016-10-14 2019-11-26 Intel Corporation Hybrid compression scheme for efficient storage of synaptic weights in hardware neuromorphic cores
US10175980B2 (en) * 2016-10-27 2019-01-08 Google Llc Neural network compute tile
US10949736B2 (en) * 2016-11-03 2021-03-16 Intel Corporation Flexible neural network accelerator and methods therefor
JP6961011B2 (ja) * 2016-12-09 2021-11-05 ベイジン ホライズン インフォメーション テクノロジー カンパニー リミテッド データ管理のためのシステム及び方法
GB2568776B (en) * 2017-08-11 2020-10-28 Google Llc Neural network accelerator with parameters resident on chip

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
L. R. Long. "A Framework for FPGA-based Acceleration of Neural Network Inference with Limited Numerical Precision via High-level Synthesis with Streaming Functionality". Thesis, University of Toronto*
S Liu 등. "Cambricon: An Instruction Set Architecture for Neural Networks". 2016 ACM/IEEE 43rd Annual International Symposium on Computer Architecture. IEEE*

Also Published As

Publication number Publication date
EP3698296A1 (en) 2020-08-26
US20190114529A1 (en) 2019-04-18
JP7196167B2 (ja) 2022-12-26
WO2019079319A1 (en) 2019-04-25
US11620490B2 (en) 2023-04-04
KR20200069338A (ko) 2020-06-16
JP2020537785A (ja) 2020-12-24
CN111226231A (zh) 2020-06-02
EP3698296B1 (en) 2024-07-17

Similar Documents

Publication Publication Date Title
KR102578508B1 (ko) 호스트 전달되는 병합된 가중치들 및 계층별 명령어들의 패키지를 사용한 뉴럴 네트워크 가속기에 의한 다중 계층 뉴럴 네트워크 프로세싱
KR102562715B1 (ko) 다수의 프로세서들 및 뉴럴 네트워크 가속기를 갖는 뉴럴 네트워크 프로세싱 시스템
KR102697368B1 (ko) 일반화된 이미지 프로세싱을 위한 이미지 프리프로세싱
US11429848B2 (en) Host-directed multi-layer neural network processing via per-layer work requests
US11568218B2 (en) Neural network processing system having host controlled kernel acclerators
US10515135B1 (en) Data format suitable for fast massively parallel general matrix multiplication in a programmable IC
JP7337053B2 (ja) 超並列ソフトウェア定義ハードウェアシステムにおける静的ブロックスケジューリング
EP3698294B1 (en) Machine learning runtime library for neural network acceleration
US10354733B1 (en) Software-defined memory bandwidth reduction by hierarchical stream buffering for general matrix multiplication in a programmable IC
US10984500B1 (en) Inline image preprocessing for convolution operations using a matrix multiplier on an integrated circuit
US11204747B1 (en) Re-targetable interface for data exchange between heterogeneous systems and accelerator abstraction into software instructions
US10943039B1 (en) Software-driven design optimization for fixed-point multiply-accumulate circuitry
US11036827B1 (en) Software-defined buffer/transposer for general matrix multiplication in a programmable IC

Legal Events

Date Code Title Description
PA0105 International application

Patent event date: 20200511

Patent event code: PA01051R01D

Comment text: International Patent Application

PG1501 Laying open of application
A201 Request for examination
PA0201 Request for examination

Patent event code: PA02012R01D

Patent event date: 20211008

Comment text: Request for Examination of Application

E902 Notification of reason for refusal
PE0902 Notice of grounds for rejection

Comment text: Notification of reason for refusal

Patent event date: 20230227

Patent event code: PE09021S01D

E701 Decision to grant or registration of patent right
PE0701 Decision of registration

Patent event code: PE07011S01D

Comment text: Decision to Grant Registration

Patent event date: 20230828

GRNT Written decision to grant
PR0701 Registration of establishment

Comment text: Registration of Establishment

Patent event date: 20230911

Patent event code: PR07011E01D

PR1002 Payment of registration fee

Payment date: 20230911

End annual number: 3

Start annual number: 1

PG1601 Publication of registration