KR102197539B1 - 신경 네트워크들에서 프로세싱하기 - Google Patents

신경 네트워크들에서 프로세싱하기 Download PDF

Info

Publication number
KR102197539B1
KR102197539B1 KR1020180125335A KR20180125335A KR102197539B1 KR 102197539 B1 KR102197539 B1 KR 102197539B1 KR 1020180125335 A KR1020180125335 A KR 1020180125335A KR 20180125335 A KR20180125335 A KR 20180125335A KR 102197539 B1 KR102197539 B1 KR 102197539B1
Authority
KR
South Korea
Prior art keywords
values
source operand
instructions
execution unit
masking
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
KR1020180125335A
Other languages
English (en)
Korean (ko)
Other versions
KR20190044549A (ko
Inventor
스테판 펠릭스
시몬 크리스티안 노우레스
코스타 고드프리 다
Original Assignee
그래프코어 리미티드
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 그래프코어 리미티드 filed Critical 그래프코어 리미티드
Publication of KR20190044549A publication Critical patent/KR20190044549A/ko
Application granted granted Critical
Publication of KR102197539B1 publication Critical patent/KR102197539B1/ko
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/30—Arrangements for executing machine instructions, e.g. instruction decode
    • G06F9/30003—Arrangements for executing specific machine instructions
    • G06F9/30007—Arrangements for executing specific machine instructions to perform operations on data operands
    • G06F9/30036—Instructions to perform operations on packed data, e.g. vector, tile or matrix operations
    • G06F9/30038—Instructions to perform operations on packed data, e.g. vector, tile or matrix operations using a mask
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F7/00—Methods or arrangements for processing data by operating upon the order or content of the data handled
    • G06F7/58—Random or pseudo-random number generators
    • G06F7/582—Pseudo-random number generators
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/30—Arrangements for executing machine instructions, e.g. instruction decode
    • G06F9/30003—Arrangements for executing specific machine instructions
    • G06F9/30007—Arrangements for executing specific machine instructions to perform operations on data operands
    • G06F9/30018—Bit or string instructions
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/30—Arrangements for executing machine instructions, e.g. instruction decode
    • G06F9/30003—Arrangements for executing specific machine instructions
    • G06F9/30007—Arrangements for executing specific machine instructions to perform operations on data operands
    • G06F9/30036—Instructions to perform operations on packed data, e.g. vector, tile or matrix operations
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/06—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons
    • G06N3/061—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons using biological neurons, e.g. biological neurons connected to an integrated circuit
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/06—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons
    • G06N3/063—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons using electronic means
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/082—Learning methods modifying the architecture, e.g. adding, deleting or silencing nodes or connections

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Software Systems (AREA)
  • General Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • Mathematical Physics (AREA)
  • Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • General Health & Medical Sciences (AREA)
  • Evolutionary Computation (AREA)
  • Data Mining & Analysis (AREA)
  • Computational Linguistics (AREA)
  • Artificial Intelligence (AREA)
  • Neurology (AREA)
  • Microelectronics & Electronic Packaging (AREA)
  • Computational Mathematics (AREA)
  • Mathematical Analysis (AREA)
  • Mathematical Optimization (AREA)
  • Pure & Applied Mathematics (AREA)
  • Executing Machine-Instructions (AREA)
  • Advance Control (AREA)
  • Complex Calculations (AREA)
KR1020180125335A 2017-10-20 2018-10-19 신경 네트워크들에서 프로세싱하기 Active KR102197539B1 (ko)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
GB1717306.3 2017-10-20
GB1717306.3A GB2568230B (en) 2017-10-20 2017-10-20 Processing in neural networks

Publications (2)

Publication Number Publication Date
KR20190044549A KR20190044549A (ko) 2019-04-30
KR102197539B1 true KR102197539B1 (ko) 2020-12-31

Family

ID=60481809

Family Applications (1)

Application Number Title Priority Date Filing Date
KR1020180125335A Active KR102197539B1 (ko) 2017-10-20 2018-10-19 신경 네트워크들에서 프로세싱하기

Country Status (8)

Country Link
US (1) US11900109B2 (enExample)
EP (1) EP3474193B1 (enExample)
JP (1) JP6709266B2 (enExample)
KR (1) KR102197539B1 (enExample)
CN (1) CN109697506B (enExample)
CA (1) CA3021426C (enExample)
GB (1) GB2568230B (enExample)
TW (1) TWI719348B (enExample)

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR102683757B1 (ko) * 2018-02-20 2024-07-10 삼성전자주식회사 심층 신경망의 학습을 수행시키는 방법 및 그에 대한 장치
KR102694572B1 (ko) * 2018-02-20 2024-08-13 삼성전자주식회사 완전 연결 네트워크의 데이터 입력 및 출력을 제어하는 방법 및 장치
KR102780709B1 (ko) * 2023-03-31 2025-03-12 한국과학기술원 부호형 비트 슬라이스 생성기 및 그 방법과, 부호형 비트 슬라이스 연산기와, 이들을 적용한 인공지능 신경망 가속장치

Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2016541038A (ja) * 2013-10-02 2016-12-28 クゥアルコム・インコーポレイテッドＱｕａｌｃｏｍｍ Ｉｎｃｏｒｐｏｒａｔｅｄ 擬似乱数のプログラム可能な確率分布関数を生成するための方法および装置

Family Cites Families (27)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
AU641418B2 (en) * 1989-09-20 1993-09-23 Fujitsu Limited A parallel data processing system for processing and transmitting data concurrently
JPH11500547A (ja) * 1994-12-01 1999-01-12 インテル・コーポレーション 乗算を有するマイクロプロセッサ
US6473522B1 (en) * 2000-03-14 2002-10-29 Intel Corporation Estimating text color and segmentation of images
JP3621695B2 (ja) 2002-07-29 2005-02-16 株式会社東芝 半導体装置及び素子形成用基板
US20040230626A1 (en) * 2003-05-12 2004-11-18 International Business Machines Corporation Computer system method for a one cycle implementation of test under mask instructions
US7961873B2 (en) * 2004-03-03 2011-06-14 King Fahd University Of Petroleum And Minerals Password protocols using XZ-elliptic curve cryptography
US7502763B2 (en) * 2005-07-29 2009-03-10 The Florida International University Board Of Trustees Artificial neural network design and evaluation tool
US9529592B2 (en) * 2007-12-27 2016-12-27 Intel Corporation Vector mask memory access instructions to perform individual and sequential memory access operations if an exception occurs during a full width memory access operation
US9477467B2 (en) * 2013-03-30 2016-10-25 Intel Corporation Processors, methods, and systems to implement partial register accesses with masked full register accesses
CN104751227B (zh) * 2013-12-31 2018-03-06 科大讯飞股份有限公司 用于语音识别的深度神经网络的构建方法及系统
CN106030513A (zh) * 2014-03-27 2016-10-12 英特尔公司 用于采用到被屏蔽结果元素的传播将连续源元素存储到未屏蔽结果元素的处理器、方法、系统和指令
EP3123300A1 (en) 2014-03-28 2017-02-01 Intel Corporation Processors, methods, systems, and instructions to store source elements to corresponding unmasked result elements with propagation to masked result elements
US20160026912A1 (en) * 2014-07-22 2016-01-28 Intel Corporation Weight-shifting mechanism for convolutional neural networks
US10133570B2 (en) * 2014-09-19 2018-11-20 Intel Corporation Processors, methods, systems, and instructions to select and consolidate active data elements in a register under mask into a least significant portion of result, and to indicate a number of data elements consolidated
EP3035205A1 (en) * 2014-12-19 2016-06-22 Intel Corporation Reconfigurable functional unit and method for artificial neural networks
US10373054B2 (en) * 2015-04-19 2019-08-06 International Business Machines Corporation Annealed dropout training of neural networks
US10192162B2 (en) * 2015-05-21 2019-01-29 Google Llc Vector computation unit in a neural network processor
CN106528047B (zh) * 2015-10-08 2019-04-09 上海兆芯集成电路有限公司 一种处理器、神经网络单元及其运作方法
JP2017097585A (ja) * 2015-11-24 2017-06-01 株式会社リコー 学習装置、プログラム及び学習方法
CN106127301B (zh) * 2016-01-16 2019-01-11 上海大学 一种随机神经网络硬件实现装置
CN106796668B (zh) * 2016-03-16 2019-06-14 香港应用科技研究院有限公司 用于人工神经网络中比特深度减少的方法和系统
US10360496B2 (en) * 2016-04-01 2019-07-23 Intel Corporation Apparatus and method for a digital neuromorphic processor
US10891538B2 (en) * 2016-08-11 2021-01-12 Nvidia Corporation Sparse convolutional neural network accelerator
US11068781B2 (en) * 2016-10-07 2021-07-20 Nvidia Corporation Temporal ensembling for semi-supervised learning
US10528321B2 (en) * 2016-12-07 2020-01-07 Microsoft Technology Licensing, Llc Block floating point for neural network implementations
CA3051990C (en) * 2017-02-23 2021-03-23 Cerebras Systems Inc. Accelerated deep learning
US10776697B2 (en) * 2017-04-18 2020-09-15 Huawei Technologies Co., Ltd. System and method for training a neural network

Patent Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2016541038A (ja) * 2013-10-02 2016-12-28 クゥアルコム・インコーポレイテッドＱｕａｌｃｏｍｍ Ｉｎｃｏｒｐｏｒａｔｅｄ 擬似乱数のプログラム可能な確率分布関数を生成するための方法および装置

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
Li Wan 외 4명. ‘Regularization of Neural Networks using DropConnect’, Proceedings of the 30th International Conference on Machine Learning (ICML’13), 2013.
ogdan Penkovsky. ‘Theory and modeling of complex nonlinear delay dynamics applied to neuromorphic computing’, 2017.06.21.

Also Published As

Publication number Publication date
TWI719348B (zh) 2021-02-21
CA3021426A1 (en) 2019-04-20
GB2568230A (en) 2019-05-15
KR20190044549A (ko) 2019-04-30
CA3021426C (en) 2023-08-01
JP6709266B2 (ja) 2020-06-10
JP2019079524A (ja) 2019-05-23
US11900109B2 (en) 2024-02-13
CN109697506A (zh) 2019-04-30
GB201717306D0 (en) 2017-12-06
TW201931107A (zh) 2019-08-01
GB2568230B (en) 2020-06-03
CN109697506B (zh) 2023-07-14
EP3474193A1 (en) 2019-04-24
US20190121639A1 (en) 2019-04-25
EP3474193B1 (en) 2025-09-10

Similar Documents

Publication Publication Date Title
US11169778B2 (en) Converting floating point numbers to reduce the precision
TWI501148B (zh) 條件式比較指令
US11449309B2 (en) Hardware module for converting numbers
US11294635B2 (en) Pseudo-random number generator
TW201805835A (zh) 一種能支援不同位元寬運算資料的運算單元、方法及裝置
US11334320B2 (en) Generating randomness in neural networks
GB2582145A (en) Execution unit in Processor
KR102197539B1 (ko) 신경 네트워크들에서 프로세싱하기
CN111260025A (zh) 用于执行lstm神经网络运算的装置和运算方法
CN114764620B (zh) 一种量子卷积操作器
CN110998512B (zh) 在数据处理设备中对连续值进行匹配
WO2024220262A1 (en) Adaptation of task performable by pre-trained model into parallel hardware
Aizaz et al. FeNN: A RISC-V vector processor for Spiking Neural Network acceleration
US20190073584A1 (en) Apparatus and methods for forward propagation in neural networks supporting discrete data
US11630667B2 (en) Dedicated vector sub-processor system
US12468534B2 (en) Overflow event counter
US20230244488A1 (en) Dual-Mode Floating Point Processor Operation
Gandhi et al. Quantum Concepts
JP2024525798A (ja) 述語技術
EP4264457A1 (en) Protection of neural networks by obfuscation of neural network operations and architecture
Kussul et al. Hardware for Neural Networks

Legal Events

Date Code Title Description
E13-X000 Pre-grant limitation requested

St.27 status event code: A-2-3-E10-E13-lim-X000

PA0109 Patent application

St.27 status event code: A-0-1-A10-A12-nap-PA0109

P11-X000 Amendment of application requested

St.27 status event code: A-2-2-P10-P11-nap-X000

P13-X000 Application amended

St.27 status event code: A-2-2-P10-P13-nap-X000

A201 Request for examination
PA0201 Request for examination

St.27 status event code: A-1-2-D10-D11-exm-PA0201

PG1501 Laying open of application

St.27 status event code: A-1-1-Q10-Q12-nap-PG1501

D13-X000 Search requested

St.27 status event code: A-1-2-D10-D13-srh-X000

D14-X000 Search report completed

St.27 status event code: A-1-2-D10-D14-srh-X000

E902 Notification of reason for refusal
PE0902 Notice of grounds for rejection

St.27 status event code: A-1-2-D10-D21-exm-PE0902

E13-X000 Pre-grant limitation requested

St.27 status event code: A-2-3-E10-E13-lim-X000

P11-X000 Amendment of application requested

St.27 status event code: A-2-2-P10-P11-nap-X000

P13-X000 Application amended

St.27 status event code: A-2-2-P10-P13-nap-X000

E902 Notification of reason for refusal
PE0902 Notice of grounds for rejection

St.27 status event code: A-1-2-D10-D21-exm-PE0902

E13-X000 Pre-grant limitation requested

St.27 status event code: A-2-3-E10-E13-lim-X000

P11-X000 Amendment of application requested

St.27 status event code: A-2-2-P10-P11-nap-X000

P13-X000 Application amended

St.27 status event code: A-2-2-P10-P13-nap-X000

E701 Decision to grant or registration of patent right
PE0701 Decision of registration

St.27 status event code: A-1-2-D10-D22-exm-PE0701

PR0701 Registration of establishment

St.27 status event code: A-2-4-F10-F11-exm-PR0701

PR1002 Payment of registration fee

St.27 status event code: A-2-2-U10-U11-oth-PR1002

Fee payment year number: 1

PG1601 Publication of registration

St.27 status event code: A-4-4-Q10-Q13-nap-PG1601

PR1001 Payment of annual fee

St.27 status event code: A-4-4-U10-U11-oth-PR1001

Fee payment year number: 4

PR1001 Payment of annual fee

St.27 status event code: A-4-4-U10-U11-oth-PR1001

Fee payment year number: 5

P22-X000 Classification modified

St.27 status event code: A-4-4-P10-P22-nap-X000

PR1001 Payment of annual fee

St.27 status event code: A-4-4-U10-U11-oth-PR1001

Fee payment year number: 6