TWI913392B - 階層和共享指數浮點數資料類型 - Google Patents

階層和共享指數浮點數資料類型

Info

Publication number
TWI913392B
TWI913392B TW111100914A TW111100914A TWI913392B TW I913392 B TWI913392 B TW I913392B TW 111100914 A TW111100914 A TW 111100914A TW 111100914 A TW111100914 A TW 111100914A TW I913392 B TWI913392 B TW I913392B
Authority
TW
Taiwan
Prior art keywords
value
shared
floating
index value
point
Prior art date
Application number
TW111100914A
Other languages
English (en)
Chinese (zh)
Other versions
TW202234229A (zh
Inventor
比塔 達菲奇魯哈尼
芬默吉爾 伊蘭格
拉索爾 沙菲普爾
傑瑞米 弗爾斯
明剛 劉
靜文 席
道格拉斯Ｃ 伯格
艾瑞克Ｓ 鍾
Original Assignee
美商微軟技術授權有限責任公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 美商微軟技術授權有限責任公司 filed Critical 美商微軟技術授權有限責任公司
Publication of TW202234229A publication Critical patent/TW202234229A/zh
Application granted granted Critical
Publication of TWI913392B publication Critical patent/TWI913392B/zh

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F7/00—Methods or arrangements for processing data by operating upon the order or content of the data handled
    • G06F7/38—Methods or arrangements for performing computations using exclusively denominational number representation, e.g. using binary, ternary, decimal representation
    • G06F7/48—Methods or arrangements for performing computations using exclusively denominational number representation, e.g. using binary, ternary, decimal representation using non-contact-making devices, e.g. tube, solid state device; using unspecified devices
    • G06F7/483—Computations with numbers represented by a non-linear combination of denominational numbers, e.g. rational numbers, logarithmic number system or floating-point numbers
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/06—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons
    • G06N3/063—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons using electronic means
    • H—ELECTRICITY
    • H03—ELECTRONIC CIRCUITRY
    • H03M—CODING; DECODING; CODE CONVERSION IN GENERAL
    • H03M7/00—Conversion of a code where information is represented by a given sequence or number of digits to a code where the same, similar or subset of information is represented by a different sequence or number of digits
    • H03M7/14—Conversion to or from non-weighted codes
    • H03M7/24—Conversion to or from floating-point codes
    • H—ELECTRICITY
    • H03—ELECTRONIC CIRCUITRY
    • H03M—CODING; DECODING; CODE CONVERSION IN GENERAL
    • H03M7/00—Conversion of a code where information is represented by a given sequence or number of digits to a code where the same, similar or subset of information is represented by a different sequence or number of digits
    • H03M7/30—Compression; Expansion; Suppression of unnecessary data, e.g. redundancy reduction

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Computing Systems (AREA)
  • Computational Mathematics (AREA)
  • Mathematical Analysis (AREA)
  • Mathematical Optimization (AREA)
  • Pure & Applied Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • Nonlinear Science (AREA)
  • Health & Medical Sciences (AREA)
  • Biophysics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Biomedical Technology (AREA)
  • Computational Linguistics (AREA)
  • Neurology (AREA)
  • Artificial Intelligence (AREA)
  • Data Mining & Analysis (AREA)
  • Evolutionary Computation (AREA)
  • General Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Complex Calculations (AREA)
  • Electromagnetism (AREA)
TW111100914A 2021-02-10 2022-01-10 階層和共享指數浮點數資料類型 TWI913392B (zh)

Applications Claiming Priority (4)

Application Number Priority Date Filing Date Title
US202163148086P 2021-02-10 2021-02-10
US63/148,086 2021-02-10
US17/361,263 2021-06-28
US17/361,263 US11886833B2 (en) 2021-02-10 2021-06-28 Hierarchical and shared exponent floating point data types

Publications (2)

Publication Number Publication Date
TW202234229A TW202234229A (zh) 2022-09-01
TWI913392B true TWI913392B (zh) 2026-02-01

Family

ID=82704967

Family Applications (1)

Application Number Title Priority Date Filing Date
TW111100914A TWI913392B (zh) 2021-02-10 2022-01-10 階層和共享指數浮點數資料類型

Country Status (6)

Country Link
US (1) US11886833B2 (enExample)
EP (1) EP4291979A1 (enExample)
JP (1) JP7899185B2 (enExample)
KR (1) KR20230137356A (enExample)
CN (1) CN116830077A (enExample)
TW (1) TWI913392B (enExample)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20240402993A1 (en) * 2023-05-30 2024-12-05 Microsoft Technology Licensing, Llc Determining shared exponent values for shared exponent floating point data types

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
TW201322118A (zh) 2011-06-30 2013-06-01 Samplify Systems Inc 浮點資料之壓縮
TW201617857A (zh) 2014-07-02 2016-05-16 上海兆芯集成電路有限公司 非不可分割分路融合乘積-累加之微處理器與方法
US20190347072A1 (en) 2018-05-08 2019-11-14 Microsoft Technology Licensing, Llc Block floating point computations using shared exponents
CN111796795A (zh) 2019-04-04 2020-10-20 英特尔公司 用于执行单精度浮点扩展数学运算的机制

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8301803B2 (en) 2009-10-23 2012-10-30 Samplify Systems, Inc. Block floating point compression of signal data
US12205035B2 (en) 2018-06-08 2025-01-21 Intel Corporation Artificial neural network training using flexible floating point tensors
US10747502B2 (en) 2018-09-19 2020-08-18 Xilinx, Inc. Multiply and accumulate circuit
US12141689B2 (en) 2019-03-18 2024-11-12 Nvidia Corporation Data compression for a neural network

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
TW201322118A (zh) 2011-06-30 2013-06-01 Samplify Systems Inc 浮點資料之壓縮
TW201617857A (zh) 2014-07-02 2016-05-16 上海兆芯集成電路有限公司 非不可分割分路融合乘積-累加之微處理器與方法
US20190347072A1 (en) 2018-05-08 2019-11-14 Microsoft Technology Licensing, Llc Block floating point computations using shared exponents
CN111796795A (zh) 2019-04-04 2020-10-20 英特尔公司 用于执行单精度浮点扩展数学运算的机制

Also Published As

Publication number Publication date
US20220253281A1 (en) 2022-08-11
EP4291979A1 (en) 2023-12-20
JP7899185B2 (ja) 2026-08-03
US11886833B2 (en) 2024-01-30
TW202234229A (zh) 2022-09-01
CN116830077A (zh) 2023-09-29
JP2024508596A (ja) 2024-02-28
KR20230137356A (ko) 2023-10-04

Similar Documents

Publication Publication Date Title
CN110880038B (zh) 基于fpga的加速卷积计算的系统、卷积神经网络
JP7525237B2 (ja) 効率的パラメータ・アップデートのための削減された精度パラメータを有する機械学習ハードウェア
JP7296709B2 (ja) ニューラルネットワークのパラメータを量子化する装置及びその方法
JP7325158B2 (ja) ニューラル・ネットワーク・コアにおける動的精度のためのデータ表現
US10491239B1 (en) Large-scale computations using an adaptive numerical format
CN109063825B (zh) 卷积神经网络加速装置
CN113508402B (zh) 从量化的固件神经网络层得出一致的软件神经网络层
KR20200004700A (ko) 뉴럴 네트워크에서 파라미터를 처리하는 방법 및 장치
CN113222102A (zh) 用于神经网络模型量化的优化方法
TW202234232A (zh) 用於標準化功能的數位電路系統
CN115983324A (zh) 神经网络量化方法、装置及电子设备
US11526761B2 (en) Neural network training with decreased memory consumption and processor utilization
TW202328985A (zh) 融合用於神經網路硬體加速器的算子
KR20240021853A (ko) 신경망에 대한 좁은 데이터 형식의 희소화
CN115965048A (zh) 数据处理装置、数据处理方法和电子设备
TW202234229A (zh) 階層和共享指數浮點數資料類型
US20250315664A1 (en) Method and device for calculating self-attention module
CN120953039A (zh) 原生支持fp8数制块量化大模型的张量核部件及芯片
TW202503508A (zh) 確定共享指數浮點資料類型的共享指數值
CN119884571A (zh) 矩阵乘法器和包括矩阵乘法器的矩阵乘法器件的操作方法
WO2020177863A1 (en) Training of algorithms
WO2022173572A1 (en) Hierarchical and shared exponent floating point data types
JP7632286B2 (ja) 情報処理装置、情報処理システム及び情報処理方法
JP2024517707A (ja) 半導体ハードウェアにおいてニューラルネットワークを処理するための実装および方法
CN109416757B (zh) 用于处理数值数据的方法、设备和计算机可读存储介质