KR20230137356A - 계층적 및 공유 지수 부동 소수점 데이터 타입 - Google Patents

계층적 및 공유 지수 부동 소수점 데이터 타입 Download PDF

Info

Publication number
KR20230137356A
KR20230137356A KR1020237027167A KR20237027167A KR20230137356A KR 20230137356 A KR20230137356 A KR 20230137356A KR 1020237027167 A KR1020237027167 A KR 1020237027167A KR 20237027167 A KR20237027167 A KR 20237027167A KR 20230137356 A KR20230137356 A KR 20230137356A
Authority
KR
South Korea
Prior art keywords
value
floating point
shared
exponent
values
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
KR1020237027167A
Other languages
English (en)
Korean (ko)
Inventor
로우하니 비타 다르비쉬
벤무길 엘란고
라소울 샤피포어
제레미 포워스
밍 강 리우
진웬 시
더글라스 씨 버거
에릭 에스 충
Original Assignee
마이크로소프트 테크놀로지 라이센싱, 엘엘씨
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 마이크로소프트 테크놀로지 라이센싱, 엘엘씨 filed Critical 마이크로소프트 테크놀로지 라이센싱, 엘엘씨
Priority claimed from PCT/US2022/013086 external-priority patent/WO2022173572A1/en
Publication of KR20230137356A publication Critical patent/KR20230137356A/ko
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F7/00—Methods or arrangements for processing data by operating upon the order or content of the data handled
    • G06F7/38—Methods or arrangements for performing computations using exclusively denominational number representation, e.g. using binary, ternary, decimal representation
    • G06F7/48—Methods or arrangements for performing computations using exclusively denominational number representation, e.g. using binary, ternary, decimal representation using non-contact-making devices, e.g. tube, solid state device; using unspecified devices
    • G06F7/483—Computations with numbers represented by a non-linear combination of denominational numbers, e.g. rational numbers, logarithmic number system or floating-point numbers
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/06—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons
    • G06N3/063—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons using electronic means
    • H—ELECTRICITY
    • H03—ELECTRONIC CIRCUITRY
    • H03M—CODING; DECODING; CODE CONVERSION IN GENERAL
    • H03M7/00—Conversion of a code where information is represented by a given sequence or number of digits to a code where the same, similar or subset of information is represented by a different sequence or number of digits
    • H03M7/14—Conversion to or from non-weighted codes
    • H03M7/24—Conversion to or from floating-point codes
    • H—ELECTRICITY
    • H03—ELECTRONIC CIRCUITRY
    • H03M—CODING; DECODING; CODE CONVERSION IN GENERAL
    • H03M7/00—Conversion of a code where information is represented by a given sequence or number of digits to a code where the same, similar or subset of information is represented by a different sequence or number of digits
    • H03M7/30—Compression; Expansion; Suppression of unnecessary data, e.g. redundancy reduction

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Computing Systems (AREA)
  • Computational Mathematics (AREA)
  • Mathematical Analysis (AREA)
  • Mathematical Optimization (AREA)
  • Pure & Applied Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • Nonlinear Science (AREA)
  • Health & Medical Sciences (AREA)
  • Biophysics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Biomedical Technology (AREA)
  • Computational Linguistics (AREA)
  • Neurology (AREA)
  • Artificial Intelligence (AREA)
  • Data Mining & Analysis (AREA)
  • Evolutionary Computation (AREA)
  • General Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Complex Calculations (AREA)
  • Electromagnetism (AREA)
KR1020237027167A 2021-02-10 2022-01-20 계층적 및 공유 지수 부동 소수점 데이터 타입 Pending KR20230137356A (ko)

Applications Claiming Priority (5)

Application Number Priority Date Filing Date Title
US202163148086P 2021-02-10 2021-02-10
US63/148,086 2021-02-10
US17/361,263 2021-06-28
US17/361,263 US11886833B2 (en) 2021-02-10 2021-06-28 Hierarchical and shared exponent floating point data types
PCT/US2022/013086 WO2022173572A1 (en) 2021-02-10 2022-01-20 Hierarchical and shared exponent floating point data types

Publications (1)

Publication Number Publication Date
KR20230137356A true KR20230137356A (ko) 2023-10-04

Family

ID=82704967

Family Applications (1)

Application Number Title Priority Date Filing Date
KR1020237027167A Pending KR20230137356A (ko) 2021-02-10 2022-01-20 계층적 및 공유 지수 부동 소수점 데이터 타입

Country Status (6)

Country Link
US (1) US11886833B2 (enExample)
EP (1) EP4291979A1 (enExample)
JP (1) JP7899185B2 (enExample)
KR (1) KR20230137356A (enExample)
CN (1) CN116830077A (enExample)
TW (1) TWI913392B (enExample)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20240402993A1 (en) * 2023-05-30 2024-12-05 Microsoft Technology Licensing, Llc Determining shared exponent values for shared exponent floating point data types

Family Cites Families (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8301803B2 (en) 2009-10-23 2012-10-30 Samplify Systems, Inc. Block floating point compression of signal data
WO2013003479A2 (en) 2011-06-30 2013-01-03 Samplify Systems, Inc. Compression of floating-point data
US9891887B2 (en) 2014-07-02 2018-02-13 Via Alliance Semiconductor Co., Ltd Subdivision of a fused compound arithmetic operation
US10579334B2 (en) * 2018-05-08 2020-03-03 Microsoft Technology Licensing, Llc Block floating point computations using shared exponents
US12205035B2 (en) 2018-06-08 2025-01-21 Intel Corporation Artificial neural network training using flexible floating point tensors
US10747502B2 (en) 2018-09-19 2020-08-18 Xilinx, Inc. Multiply and accumulate circuit
US12141689B2 (en) 2019-03-18 2024-11-12 Nvidia Corporation Data compression for a neural network
US11934797B2 (en) 2019-04-04 2024-03-19 Intel Corporation Mechanism to perform single precision floating point extended math operations

Also Published As

Publication number Publication date
US20220253281A1 (en) 2022-08-11
EP4291979A1 (en) 2023-12-20
JP7899185B2 (ja) 2026-08-03
TWI913392B (zh) 2026-02-01
US11886833B2 (en) 2024-01-30
TW202234229A (zh) 2022-09-01
CN116830077A (zh) 2023-09-29
JP2024508596A (ja) 2024-02-28

Similar Documents

Publication Publication Date Title
CN110880038B (zh) 基于fpga的加速卷积计算的系统、卷积神经网络
CN112074806B (zh) 用于块浮点计算的系统、方法和计算机存储介质
CN112088354B (zh) 使用共享指数的块浮点计算
CN112673383B (zh) 神经网络核中动态精度的数据表示
KR20200004700A (ko) 뉴럴 네트워크에서 파라미터를 처리하는 방법 및 장치
JP2019164793A5 (enExample)
US11783200B2 (en) Artificial neural network implementation in field-programmable gate arrays
KR20190034985A (ko) 인공 신경망의 양자화 방법 및 장치
WO2020176248A1 (en) Neural network layer processing with scaled quantization
EP3931756A1 (en) Neural network layer processing with normalization and transformation of data
CN114626516A (zh) 一种基于对数块浮点量化的神经网络加速系统
KR20230132795A (ko) 정규화 기능을 위한 디지털 회로
KR20240021853A (ko) 신경망에 대한 좁은 데이터 형식의 희소화
CN116187413A (zh) 基于动态精度量化的神经网络模型训练加速方法及系统
US20240403644A1 (en) Optimizing deep neural network models based on sparsification and quantization
US11886833B2 (en) Hierarchical and shared exponent floating point data types
EP4591218A1 (en) Quantized neural network architecture
WO2023201424A1 (en) System and method for adaptation of containers for floating-point data for training of a machine learning model
CN121002477A (zh) 确定用于共享指数浮点数据类型的共享指数值
US20210216867A1 (en) Information processing apparatus, neural network computation program, and neural network computation method
KR20240105373A (ko) 곱셈 및 누산 유닛의 모듈로 공간 프로세싱
WO2022173572A1 (en) Hierarchical and shared exponent floating point data types
US20230385600A1 (en) Optimizing method and computing apparatus for deep learning network and computer-readable storage medium
WO2020177863A1 (en) Training of algorithms
KR20260039700A (ko) 기계 학습 모델 파라미터들의 혼합된 정밀도 양자화

Legal Events

Date Code Title Description
PA0105 International application

St.27 status event code: A-0-1-A10-A15-nap-PA0105

PG1501 Laying open of application

St.27 status event code: A-1-1-Q10-Q12-nap-PG1501

A201 Request for examination
P11-X000 Amendment of application requested

St.27 status event code: A-2-2-P10-P11-nap-X000

P13-X000 Application amended

St.27 status event code: A-2-2-P10-P13-nap-X000

PA0201 Request for examination

St.27 status event code: A-1-2-D10-D11-exm-PA0201

D13 Search requested

Free format text: ST27 STATUS EVENT CODE: A-1-2-D10-D13-SRH-X000 (AS PROVIDED BY THE NATIONAL OFFICE)

D13-X000 Search requested

St.27 status event code: A-1-2-D10-D13-srh-X000

D14 Search report completed

Free format text: ST27 STATUS EVENT CODE: A-1-2-D10-D14-SRH-X000 (AS PROVIDED BY THE NATIONAL OFFICE)

D14-X000 Search report completed

St.27 status event code: A-1-2-D10-D14-srh-X000

PE0701 Decision of registration

St.27 status event code: A-1-2-D10-D22-exm-PE0701