CN116830077A - 分层和共享指数浮点数据类型 - Google Patents

分层和共享指数浮点数据类型 Download PDF

Info

Publication number
CN116830077A
CN116830077A CN202280014048.0A CN202280014048A CN116830077A CN 116830077 A CN116830077 A CN 116830077A CN 202280014048 A CN202280014048 A CN 202280014048A CN 116830077 A CN116830077 A CN 116830077A
Authority
CN
China
Prior art keywords
value
exponent
shared
floating point
values
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202280014048.0A
Other languages
English (en)
Chinese (zh)
Inventor
B·达尔维什·鲁哈尼
V·埃兰戈
R·沙菲普尔
J·弗沃斯
刘明罡
奚锦文
D·C·伯格
E·S·钟
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Microsoft Technology Licensing LLC
Original Assignee
Microsoft Technology Licensing LLC
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Microsoft Technology Licensing LLC filed Critical Microsoft Technology Licensing LLC
Priority claimed from PCT/US2022/013086 external-priority patent/WO2022173572A1/en
Publication of CN116830077A publication Critical patent/CN116830077A/zh
Pending legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F7/00—Methods or arrangements for processing data by operating upon the order or content of the data handled
    • G06F7/38—Methods or arrangements for performing computations using exclusively denominational number representation, e.g. using binary, ternary, decimal representation
    • G06F7/48—Methods or arrangements for performing computations using exclusively denominational number representation, e.g. using binary, ternary, decimal representation using non-contact-making devices, e.g. tube, solid state device; using unspecified devices
    • G06F7/483—Computations with numbers represented by a non-linear combination of denominational numbers, e.g. rational numbers, logarithmic number system or floating-point numbers
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/06—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons
    • G06N3/063—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons using electronic means
    • H—ELECTRICITY
    • H03—ELECTRONIC CIRCUITRY
    • H03M—CODING; DECODING; CODE CONVERSION IN GENERAL
    • H03M7/00—Conversion of a code where information is represented by a given sequence or number of digits to a code where the same, similar or subset of information is represented by a different sequence or number of digits
    • H03M7/14—Conversion to or from non-weighted codes
    • H03M7/24—Conversion to or from floating-point codes
    • H—ELECTRICITY
    • H03—ELECTRONIC CIRCUITRY
    • H03M—CODING; DECODING; CODE CONVERSION IN GENERAL
    • H03M7/00—Conversion of a code where information is represented by a given sequence or number of digits to a code where the same, similar or subset of information is represented by a different sequence or number of digits
    • H03M7/30—Compression; Expansion; Suppression of unnecessary data, e.g. redundancy reduction

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Computing Systems (AREA)
  • Computational Mathematics (AREA)
  • Mathematical Analysis (AREA)
  • Mathematical Optimization (AREA)
  • Pure & Applied Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • Nonlinear Science (AREA)
  • Health & Medical Sciences (AREA)
  • Biophysics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Biomedical Technology (AREA)
  • Computational Linguistics (AREA)
  • Neurology (AREA)
  • Artificial Intelligence (AREA)
  • Data Mining & Analysis (AREA)
  • Evolutionary Computation (AREA)
  • General Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Complex Calculations (AREA)
  • Electromagnetism (AREA)
CN202280014048.0A 2021-02-10 2022-01-20 分层和共享指数浮点数据类型 Pending CN116830077A (zh)

Applications Claiming Priority (5)

Application Number Priority Date Filing Date Title
US202163148086P 2021-02-10 2021-02-10
US63/148,086 2021-02-10
US17/361,263 2021-06-28
US17/361,263 US11886833B2 (en) 2021-02-10 2021-06-28 Hierarchical and shared exponent floating point data types
PCT/US2022/013086 WO2022173572A1 (en) 2021-02-10 2022-01-20 Hierarchical and shared exponent floating point data types

Publications (1)

Publication Number Publication Date
CN116830077A true CN116830077A (zh) 2023-09-29

Family

ID=82704967

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202280014048.0A Pending CN116830077A (zh) 2021-02-10 2022-01-20 分层和共享指数浮点数据类型

Country Status (6)

Country Link
US (1) US11886833B2 (enExample)
EP (1) EP4291979A1 (enExample)
JP (1) JP7899185B2 (enExample)
KR (1) KR20230137356A (enExample)
CN (1) CN116830077A (enExample)
TW (1) TWI913392B (enExample)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20240402993A1 (en) * 2023-05-30 2024-12-05 Microsoft Technology Licensing, Llc Determining shared exponent values for shared exponent floating point data types

Family Cites Families (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8301803B2 (en) 2009-10-23 2012-10-30 Samplify Systems, Inc. Block floating point compression of signal data
WO2013003479A2 (en) 2011-06-30 2013-01-03 Samplify Systems, Inc. Compression of floating-point data
US9891887B2 (en) 2014-07-02 2018-02-13 Via Alliance Semiconductor Co., Ltd Subdivision of a fused compound arithmetic operation
US10579334B2 (en) * 2018-05-08 2020-03-03 Microsoft Technology Licensing, Llc Block floating point computations using shared exponents
US12205035B2 (en) 2018-06-08 2025-01-21 Intel Corporation Artificial neural network training using flexible floating point tensors
US10747502B2 (en) 2018-09-19 2020-08-18 Xilinx, Inc. Multiply and accumulate circuit
US12141689B2 (en) 2019-03-18 2024-11-12 Nvidia Corporation Data compression for a neural network
US11934797B2 (en) 2019-04-04 2024-03-19 Intel Corporation Mechanism to perform single precision floating point extended math operations

Also Published As

Publication number Publication date
US20220253281A1 (en) 2022-08-11
EP4291979A1 (en) 2023-12-20
JP7899185B2 (ja) 2026-08-03
TWI913392B (zh) 2026-02-01
US11886833B2 (en) 2024-01-30
TW202234229A (zh) 2022-09-01
JP2024508596A (ja) 2024-02-28
KR20230137356A (ko) 2023-10-04

Similar Documents

Publication Publication Date Title
CN110880038B (zh) 基于fpga的加速卷积计算的系统、卷积神经网络
CN110222821B (zh) 基于权重分布的卷积神经网络低位宽量化方法
US10491239B1 (en) Large-scale computations using an adaptive numerical format
KR102381770B1 (ko) 정밀 지수 및 정밀 소프트맥스 계산
EP3915056B1 (en) Neural network activation compression with non-uniform mantissas
WO2020074989A1 (en) Data representation for dynamic precision in neural network cores
CA3044660C (en) Information processing device and information processing method
EP3931758A1 (en) Neural network layer processing with scaled quantization
EP3931756A1 (en) Neural network layer processing with normalization and transformation of data
CN114418057A (zh) 卷积神经网络的运算方法及相关设备
EP4022773B1 (en) Compression of data that exhibits mixed compressibility
CN116187413A (zh) 基于动态精度量化的神经网络模型训练加速方法及系统
KR20240021853A (ko) 신경망에 대한 좁은 데이터 형식의 희소화
TWI913392B (zh) 階層和共享指數浮點數資料類型
CN117574966A (zh) 模型量化方法、装置、电子设备及存储介质
CN115965048A (zh) 数据处理装置、数据处理方法和电子设备
US20240402993A1 (en) Determining shared exponent values for shared exponent floating point data types
WO2022173572A1 (en) Hierarchical and shared exponent floating point data types
US20210216867A1 (en) Information processing apparatus, neural network computation program, and neural network computation method
CN119884571A (zh) 矩阵乘法器和包括矩阵乘法器的矩阵乘法器件的操作方法
WO2020177863A1 (en) Training of algorithms
CN114207609B (zh) 信息处理装置、信息处理系统和信息处理方法
EP4455943A1 (en) Optimizing method and computing system for deep learning network
CN115965047A (zh) 数据处理器、数据处理方法和电子设备
WO2026016482A1 (zh) 数据处理方法、装置、芯片、服务器及系统

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination