PH12022552241A1 - Parallelized rate-distortion optimized quantization using deep learning - Google Patents

Parallelized rate-distortion optimized quantization using deep learning

Info

Publication number
PH12022552241A1
PH12022552241A1 PH1/2022/552241A PH12022552241A PH12022552241A1 PH 12022552241 A1 PH12022552241 A1 PH 12022552241A1 PH 12022552241 A PH12022552241 A PH 12022552241A PH 12022552241 A1 PH12022552241 A1 PH 12022552241A1
Authority
PH
Philippines
Prior art keywords
block
video encoder
coefficients
probabilities
transform coefficient
Prior art date
Application number
PH1/2022/552241A
Other languages
English (en)
Inventor
Taco Sebastiaan Cohen
Dana Kianfar
Reza Pourreza
Amir Said
Auke Joris Wiggers
Original Assignee
Qualcomm Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Qualcomm Inc filed Critical Qualcomm Inc
Publication of PH12022552241A1 publication Critical patent/PH12022552241A1/en

Links

Classifications

    • GPHYSICS
    • G05CONTROLLING; REGULATING
    • G05BCONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
    • G05B13/00Adaptive control systems, i.e. systems automatically adjusting themselves to have a performance which is optimum according to some preassigned criterion
    • GPHYSICS
    • G06COMPUTING OR CALCULATING; COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F30/00Computer-aided design [CAD]
    • G06F30/20Design optimisation, verification or simulation
    • G06F30/27Design optimisation, verification or simulation using machine learning, e.g. artificial intelligence, neural networks, support vector machines [SVM] or training a model
    • GPHYSICS
    • G06COMPUTING OR CALCULATING; COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • GPHYSICS
    • G06COMPUTING OR CALCULATING; COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/04Architecture, e.g. interconnection topology
    • G06N3/045Combinations of networks
    • GPHYSICS
    • G06COMPUTING OR CALCULATING; COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/04Architecture, e.g. interconnection topology
    • G06N3/0464Convolutional networks [CNN, ConvNet]
    • GPHYSICS
    • G06COMPUTING OR CALCULATING; COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/04Architecture, e.g. interconnection topology
    • G06N3/047Probabilistic or stochastic networks
    • GPHYSICS
    • G06COMPUTING OR CALCULATING; COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/08Learning methods
    • G06N3/082Learning methods modifying the architecture, e.g. adding, deleting or silencing nodes or connections
    • GPHYSICS
    • G06COMPUTING OR CALCULATING; COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/08Learning methods
    • G06N3/084Backpropagation, e.g. using gradient descent
    • GPHYSICS
    • G06COMPUTING OR CALCULATING; COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/08Learning methods
    • G06N3/09Supervised learning
    • GPHYSICS
    • G06COMPUTING OR CALCULATING; COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N5/00Computing arrangements using knowledge-based models
    • G06N5/01Dynamic search techniques; Heuristics; Dynamic trees; Branch-and-bound
    • GPHYSICS
    • G06COMPUTING OR CALCULATING; COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T9/00Image coding
    • G06T9/002Image coding using neural networks
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/117Filters, e.g. for pre-processing or post-processing
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/124Quantisation
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/13Adaptive entropy coding, e.g. adaptive variable length coding [AVLC] or context adaptive binary arithmetic coding [CABAC]
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/146Data rate or code amount at the encoder output
    • H04N19/147Data rate or code amount at the encoder output according to rate distortion criteria
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/157Assigned coding mode, i.e. the coding mode being predefined or preselected to be further used for selection of another element or parameter
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/17Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
    • H04N19/176Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a block, e.g. a macroblock
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/18Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being a set of transform coefficients
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/46Embedding additional information in the video signal during the compression process
    • H04N19/463Embedding additional information in the video signal during the compression process by compressing encoding parameters before transmission
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/48Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using compressed domain processing techniques other than decoding, e.g. modification of transform coefficients, variable length coding [VLC] data or run-length data
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/60Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding
    • H04N19/61Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding in combination with predictive coding
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/80Details of filtering operations specially adapted for video compression, e.g. for pixel interpolation
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04QSELECTING
    • H04Q2213/00Indexing scheme relating to selecting arrangements in general and for multiplex systems
    • H04Q2213/13343Neural networks
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04QSELECTING
    • H04Q2213/00Indexing scheme relating to selecting arrangements in general and for multiplex systems
    • H04Q2213/343Neural network
    • YGENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y10TECHNICAL SUBJECTS COVERED BY FORMER USPC
    • Y10STECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y10S706/00Data processing: artificial intelligence

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Evolutionary Computation (AREA)
  • General Physics & Mathematics (AREA)
  • Artificial Intelligence (AREA)
  • Software Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Health & Medical Sciences (AREA)
  • Computational Linguistics (AREA)
  • Data Mining & Analysis (AREA)
  • Mathematical Physics (AREA)
  • Computing Systems (AREA)
  • Biophysics (AREA)
  • Molecular Biology (AREA)
  • General Health & Medical Sciences (AREA)
  • Biomedical Technology (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Medical Informatics (AREA)
  • Probability & Statistics with Applications (AREA)
  • Computer Hardware Design (AREA)
  • Geometry (AREA)
  • Automation & Control Theory (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)
  • Compression, Expansion, Code Conversion, And Decoders (AREA)
PH1/2022/552241A 2020-04-17 2021-03-23 Parallelized rate-distortion optimized quantization using deep learning PH12022552241A1 (en)

Applications Claiming Priority (4)

Application Number Priority Date Filing Date Title
US202063011685P 2020-04-17 2020-04-17
US202063034618P 2020-06-04 2020-06-04
US17/070,589 US12058348B2 (en) 2020-04-17 2020-10-14 Parallelized rate-distortion optimized quantization using deep learning
PCT/US2021/023680 WO2021211270A1 (en) 2020-04-17 2021-03-23 Parallelized rate-distortion optimized quantization using deep learning

Publications (1)

Publication Number Publication Date
PH12022552241A1 true PH12022552241A1 (en) 2024-03-11

Family

ID=78082393

Family Applications (1)

Application Number Title Priority Date Filing Date
PH1/2022/552241A PH12022552241A1 (en) 2020-04-17 2021-03-23 Parallelized rate-distortion optimized quantization using deep learning

Country Status (9)

Country Link
US (1) US12058348B2 (https=)
EP (1) EP4136837A1 (https=)
JP (1) JP7642671B2 (https=)
KR (1) KR20230007313A (https=)
CN (1) CN115336266B (https=)
BR (1) BR112022020125A2 (https=)
PH (1) PH12022552241A1 (https=)
TW (1) TW202145792A (https=)
WO (1) WO2021211270A1 (https=)

Families Citing this family (16)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11490083B2 (en) 2020-02-05 2022-11-01 Qualcomm Incorporated Learned low-complexity adaptive quantization for video compression
US20220215265A1 (en) * 2021-01-04 2022-07-07 Tencent America LLC Method and apparatus for end-to-end task-oriented latent compression with deep reinforcement learning
US12244792B2 (en) * 2021-03-30 2025-03-04 Sony Interactive Entertainment Europe Limited Processing image data
US11368349B1 (en) * 2021-11-15 2022-06-21 King Abdulaziz University Convolutional neural networks based computationally efficient method for equalization in FBMC-OQAM system
JP7825447B2 (ja) * 2022-02-14 2026-03-06 日本放送協会 符号化装置、プログラム、及びモデル生成方法
WO2023169501A1 (en) * 2022-03-09 2023-09-14 Beijing Bytedance Network Technology Co., Ltd. Method, apparatus, and medium for visual data processing
US20230306239A1 (en) * 2022-03-25 2023-09-28 Tencent America LLC Online training-based encoder tuning in neural image compression
US20230316588A1 (en) * 2022-03-29 2023-10-05 Tencent America LLC Online training-based encoder tuning with multi model selection in neural image compression
US12231183B2 (en) * 2022-04-29 2025-02-18 Qualcomm Incorporated Machine learning for beam predictions with confidence indications
CN114708436B (zh) * 2022-06-02 2022-09-02 深圳比特微电子科技有限公司 语义分割模型的训练方法、语义分割方法、装置和介质
CN115209147B (zh) * 2022-09-15 2022-12-27 深圳沛喆微电子有限公司 摄像头视频传输带宽优化方法、装置、设备及存储介质
CN116366846B (zh) * 2023-03-14 2025-11-11 北京百度网讯科技有限公司 视频编码方法、装置以及设备
CN117764192A (zh) * 2023-07-31 2024-03-26 中国银联股份有限公司 构建对比矩阵的方法和系统以及层次分析方法和系统
WO2025114549A1 (en) * 2023-12-01 2025-06-05 Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. Block-based codec supporting transform coefficient prediction and/or transform improvement
US20250307133A1 (en) * 2024-03-28 2025-10-02 Advanced Micro Devices, Inc. Offloading Quantization of Directional Blocked Data Formats to Near-Memory Units
WO2025238505A1 (en) * 2024-05-13 2025-11-20 Imax Corporation Large multimodal model-based video encoding optimization

Family Cites Families (23)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR20030009575A (ko) * 2001-06-26 2003-02-05 박광훈 신경망 분류기를 이용한 동영상 전송률 제어 장치 및 그방법
US7620103B2 (en) 2004-12-10 2009-11-17 Lsi Corporation Programmable quantization dead zone and threshold for standard-based H.264 and/or VC1 video encoding
US7889790B2 (en) 2005-12-20 2011-02-15 Sharp Laboratories Of America, Inc. Method and apparatus for dynamically adjusting quantization offset values
US7995649B2 (en) 2006-04-07 2011-08-09 Microsoft Corporation Quantization adjustment based on texture level
US8767834B2 (en) 2007-03-09 2014-07-01 Sharp Laboratories Of America, Inc. Methods and systems for scalable-to-non-scalable bit-stream rewriting
ES2681209T3 (es) 2009-09-10 2018-09-12 Guangdong Oppo Mobile Telecommunications Corp., Ltd. Técnicas de aceleración para una cuantificación optimizada de tasa de distorsión
US8170110B2 (en) * 2009-10-16 2012-05-01 Hong Kong Applied Science and Technology Research Institute Company Limited Method and apparatus for zoom motion estimation
KR101492930B1 (ko) 2010-09-14 2015-02-23 블랙베리 리미티드 변환 도메인 내의 어댑티브 필터링을 이용한 데이터 압축 방법 및 장치
BR112013007023A2 (pt) * 2010-09-28 2017-07-25 Samsung Electronics Co Ltd método de codificação de vídeo e método de decodificação de vídeo
US8553769B2 (en) * 2011-01-19 2013-10-08 Blackberry Limited Method and device for improved multi-layer data compression
US9521410B2 (en) 2012-04-26 2016-12-13 Qualcomm Incorporated Quantization parameter (QP) coding in video coding
US9213556B2 (en) * 2012-07-30 2015-12-15 Vmware, Inc. Application directed user interface remoting using video encoding techniques
US9560386B2 (en) 2013-02-21 2017-01-31 Mozilla Corporation Pyramid vector quantization for video coding
US9294766B2 (en) 2013-09-09 2016-03-22 Apple Inc. Chroma quantization in video coding
US10057578B2 (en) * 2014-10-07 2018-08-21 Qualcomm Incorporated QP derivation and offset for adaptive color transform in video coding
EP3545679B1 (en) * 2016-12-02 2022-08-24 Huawei Technologies Co., Ltd. Apparatus and method for encoding an image
US10721471B2 (en) * 2017-10-26 2020-07-21 Intel Corporation Deep learning based quantization parameter estimation for video encoding
KR102941657B1 (ko) * 2018-02-08 2026-03-20 한국전자통신연구원 신경망에 기반하는 비디오 부호화 및 비디오 복호화를 위한 방법 및 장치
EP3633990B1 (en) * 2018-10-02 2021-10-27 Nokia Technologies Oy An apparatus and method for using a neural network in video coding
JP2020088740A (ja) * 2018-11-29 2020-06-04 ピクシブ株式会社 画像処理装置、画像処理方法及び画像処理プログラム
US12505580B2 (en) * 2019-07-02 2025-12-23 Telefonaktiebolaget Lm Ericsson (Publ) Inference processing of data
US11496769B2 (en) * 2019-09-27 2022-11-08 Apple Inc. Neural network based image set compression
CN112819699B (zh) * 2019-11-15 2024-11-05 北京金山云网络技术有限公司 视频处理方法、装置及电子设备

Also Published As

Publication number Publication date
US12058348B2 (en) 2024-08-06
JP2023522575A (ja) 2023-05-31
CN115336266B (zh) 2025-09-23
BR112022020125A2 (pt) 2022-11-29
WO2021211270A1 (en) 2021-10-21
KR20230007313A (ko) 2023-01-12
EP4136837A1 (en) 2023-02-22
JP7642671B2 (ja) 2025-03-10
TW202145792A (zh) 2021-12-01
CN115336266A (zh) 2022-11-11
US20210329267A1 (en) 2021-10-21

Similar Documents

Publication Publication Date Title
PH12022552241A1 (en) Parallelized rate-distortion optimized quantization using deep learning
US20250278595A1 (en) Methods and apparatuses for compressing parameters of neural networks
US11990148B2 (en) Compressing audio waveforms using neural networks and vector quantizers
EP3970371B1 (en) Content adaptive optimization for neural data compression
US11403528B2 (en) Self-tuning incremental model compression solution in deep neural network with guaranteed accuracy performance
US12087024B2 (en) Image compression using normalizing flows
CN111641832B (zh) 编码方法、解码方法、装置、电子设备及存储介质
CN106203624A (zh) 基于深度神经网络的矢量量化系统及方法
US12166995B2 (en) Image encoding method and image decoding method
CN115361559A (zh) 图像编码方法、图像解码方法、装置以及存储介质
US20200293895A1 (en) Information processing method and apparatus
CN110753225A (zh) 一种视频压缩方法、装置及终端设备
Bilen et al. Solving time-domain audio inverse problems using nonnegative tensor factorization
Zhe et al. Rate-distortion optimized coding for efficient cnn compression
CN109361922B (zh) 预测量化编码方法
Yoshimura et al. WaveNet-based zero-delay lossless speech coding
CN111161363A (zh) 一种图像编码模型训练方法及装置
Shin et al. On the Potential of Entropy-Constrained Vector Quantization in Semantic Communication
Shin et al. Conditional Entropy-Constrained Multi-Stage Vector Quantization for Semantic Communication
Faundez-Zanuy Non-linear predictive vector quantization of speech
JPH09200778A (ja) 映像信号符号化方法及び映像信号符号化装置
JP2914546B2 (ja) 特異値展開画像符号化装置
Omara et al. From a sparse vector to a sparse symmetric matrix for efficient lossy speech compression
KR100214377B1 (ko) 개선된 신경회로망 데이터 콤프레서 및 이를 이용한 데이터 처리 방법
WO2024229038A3 (en) Systems and methods for dependent quantization based on current dequantization states