JP2024520682A - 機械向け映像符号化（ｖｃｍ）のためのエンコーダ及びデコーダ - Google Patents

機械向け映像符号化（ｖｃｍ）のためのエンコーダ及びデコーダ Download PDF

Info

Publication number
JP2024520682A
JP2024520682A JP2023574502A JP2023574502A JP2024520682A JP 2024520682 A JP2024520682 A JP 2024520682A JP 2023574502 A JP2023574502 A JP 2023574502A JP 2023574502 A JP2023574502 A JP 2023574502A JP 2024520682 A JP2024520682 A JP 2024520682A
Authority
JP
Japan
Prior art keywords
encoder
feature
video
vcm
bitstream
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
JP2023574502A
Other languages
English (en)
Japanese (ja)
Other versions
JP2024520682A5 (https=
Inventor
カルバ、ハリ
フルト、ボリヴォジェ
アジッチ、ベリボル
Original Assignee
オーピー ソリューションズ， エルエルシー
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by オーピー ソリューションズ， エルエルシー filed Critical オーピー ソリューションズ， エルエルシー
Publication of JP2024520682A publication Critical patent/JP2024520682A/ja
Publication of JP2024520682A5 publication Critical patent/JP2024520682A5/ja
Pending legal-status Critical Current

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/235—Processing of additional data, e.g. scrambling of additional data or processing content descriptors
    • H04N21/2355—Processing of additional data, e.g. scrambling of additional data or processing content descriptors involving reformatting operations of additional data, e.g. HTML pages
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/44—Decoders specially adapted therefor, e.g. video decoders which are asymmetric with respect to the encoder
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/0464—Convolutional networks [CNN, ConvNet]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
    • G06V10/7715—Feature extraction, e.g. by transforming the feature space, e.g. multi-dimensional scaling [MDS]; Mappings, e.g. subspace methods
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/40—Scenes; Scene-specific elements in video content
    • G06V20/46—Extracting features or characteristics from the video content, e.g. video fingerprints, representative shots or key frames
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/189—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the adaptation method, adaptation tool or adaptation type used for the adaptive coding
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/46—Embedding additional information in the video signal during the compression process
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/234—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs
    • H04N21/23418—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving operations for analysing video streams, e.g. detecting features or characteristics
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/234—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs
    • H04N21/2343—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/435—Processing of additional data, e.g. decrypting of additional data, reconstructing software from modules extracted from the transport stream
    • H04N21/4355—Processing of additional data, e.g. decrypting of additional data, reconstructing software from modules extracted from the transport stream involving reformatting operations of additional data, e.g. HTML pages on a television screen
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
    • H04N21/4402—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Theoretical Computer Science (AREA)
  • Signal Processing (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Evolutionary Computation (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • General Health & Medical Sciences (AREA)
  • Software Systems (AREA)
  • Computing Systems (AREA)
  • Artificial Intelligence (AREA)
  • Health & Medical Sciences (AREA)
  • Medical Informatics (AREA)
  • Databases & Information Systems (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • Data Mining & Analysis (AREA)
  • Molecular Biology (AREA)
  • General Engineering & Computer Science (AREA)
  • Mathematical Physics (AREA)
  • Biomedical Technology (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)
JP2023574502A 2021-06-07 2022-06-03 機械向け映像符号化（ｖｃｍ）のためのエンコーダ及びデコーダ Pending JP2024520682A (ja)

Applications Claiming Priority (3)

Application Number Priority Date Filing Date Title
US202163197834P 2021-06-07 2021-06-07
US63/197,834 2021-06-07
PCT/US2022/032048 WO2022260934A1 (en) 2021-06-07 2022-06-03 Encoder and decoder for video coding for machines (vcm)

Publications (2)

Publication Number Publication Date
JP2024520682A true JP2024520682A (ja) 2024-05-24
JP2024520682A5 JP2024520682A5 (https=) 2025-05-29

Family

ID=84425308

Family Applications (1)

Application Number Title Priority Date Filing Date
JP2023574502A Pending JP2024520682A (ja) 2021-06-07 2022-06-03 機械向け映像符号化（ｖｃｍ）のためのエンコーダ及びデコーダ

Country Status (6)

Country Link
US (1) US20240107088A1 (https=)
EP (1) EP4352701A4 (https=)
JP (1) JP2024520682A (https=)
KR (1) KR20240051076A (https=)
CN (1) CN117897736A (https=)
WO (1) WO2022260934A1 (https=)

Families Citing this family (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP4427459A4 (en) * 2021-11-04 2026-01-28 Op Solutions Llc SYSTEMS AND METHODS FOR TRANSFERRING MOTION INFORMATION FROM A VISUAL DOMAIN WITH CHARACTERISTICS AND FINE-FINING MOTION VECTOR CONTROL ON THE DECODER SIDE BASED ON CHARACTERISTICS
CN120419187A (zh) * 2023-01-03 2025-08-01 Lg 电子株式会社 编码/解码方法和装置以及存储比特流的记录介质
US12309404B2 (en) * 2023-03-07 2025-05-20 Disney Enterprises, Inc. Contextual video compression framework with spatial-temporal cross-covariance transformers
CN120917753A (zh) * 2023-04-12 2025-11-07 Lg 电子株式会社 编码/解码方法、设备和存储比特流的记录介质
WO2025221719A1 (en) * 2024-04-15 2025-10-23 Op Solutions, Llc Systems, methods and bitstreams for removing non-essential feature map information in machine-based applications using regions of interest
WO2025254466A1 (ko) * 2024-06-05 2025-12-11 한화비전 주식회사 스케일링 파라미터를 이용하는 부호화/복호화 방법, 장치 및 기록 매체

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2020055279A1 (en) * 2018-09-10 2020-03-19 Huawei Technologies Co., Ltd. Hybrid video and feature coding and decoding
WO2021095245A1 (ja) * 2019-11-15 2021-05-20 日本電信電話株式会社 画像処理方法、データ処理方法、画像処理装置、およびプログラム

Family Cites Families (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20170249739A1 (en) * 2016-02-26 2017-08-31 Biomediq A/S Computer analysis of mammograms
US11019355B2 (en) * 2018-04-03 2021-05-25 Electronics And Telecommunications Research Institute Inter-prediction method and apparatus using reference frame generated based on deep learning
US11410275B2 (en) * 2019-09-23 2022-08-09 Tencent America LLC Video coding for machine (VCM) based system and method for video super resolution (SR)
WO2021172956A1 (ko) * 2020-02-28 2021-09-02 엘지전자 주식회사 영상 특징 정보 시그널링을 위한 영상 부호화/복호화 방법, 장치 및 비트스트림을 전송하는 방법
EP4136848A4 (en) * 2020-04-16 2024-04-03 INTEL Corporation PATCH-BASED VIDEO CODING FOR MACHINES
US11516478B2 (en) * 2020-12-30 2022-11-29 Hyundai Motor Company Method and apparatus for coding machine vision data using prediction

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2020055279A1 (en) * 2018-09-10 2020-03-19 Huawei Technologies Co., Ltd. Hybrid video and feature coding and decoding
WO2021095245A1 (ja) * 2019-11-15 2021-05-20 日本電信電話株式会社 画像処理方法、データ処理方法、画像処理装置、およびプログラム

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
XIANG ZHANG ET AL.: ""A Joint Compression Scheme of Video Feature Descriptors and Visual Content"", IEEE TRANSACTIONS ON IMAGE PROCESSING, vol. 26, no. 2, JPN7026000115, 1 February 2017 (2017-02-01), US, pages 635 - 636, ISSN: 0005775460 *
XIANG ZHANG ET AL.: ""From Visual Search to Video Compression: A Compact Representation Framework for Video Feature Descr", 2016 DATA COMPRESSION CONFERENCE (DCC), JPN6026000779, 19 December 2016 (2016-12-19), US, pages 408 - 409, ISSN: 0005775459 *

Also Published As

Publication number Publication date
CN117897736A (zh) 2024-04-16
KR20240051076A (ko) 2024-04-19
WO2022260934A1 (en) 2022-12-15
EP4352701A4 (en) 2025-04-09
EP4352701A1 (en) 2024-04-17
US20240107088A1 (en) 2024-03-28

Similar Documents

Publication Publication Date Title
US20240107088A1 (en) Encoder and decoder for video coding for machines (vcm)
US20240357142A1 (en) Video and feature coding for multi-task machine learning
US20240283942A1 (en) Systems and methods for object and event detection and feature-based rate-distortion optimization for video coding
CN119301609A (zh) 用于使用生成式对抗模型对图像数据进行编码和解码的系统和方法
US20240185572A1 (en) Systems and methods for joint optimization training and encoder side downsampling
CN119032558A (zh) 使用自编码器进行面向机器的视频编码的系统和方法
US20240267531A1 (en) Systems and methods for optimizing a loss function for video coding for machines
US20240340391A1 (en) Intelligent multi-stream video coding for video surveillance
US20240236342A1 (en) Systems and methods for scalable video coding for machines
US20240357107A1 (en) Systems and methods for video coding of features using subpictures
US20240291999A1 (en) Systems and methods for motion information transfer from visual to feature domain and feature-based decoder-side motion vector refinement control
CN118414829A (zh) 用于对象和事件检测以及用于视频编码的基于特征的率失真优化的系统和方法
CN118614062A (zh) 用于从视觉到特征域的运动信息传递的系统和方法
CN118119951A (zh) 用于联合优化训练和编码器侧下采样的系统和方法
CN121713217A (zh) 用于机器视频编码的解码后的帧增强的系统和方法
WO2025015116A2 (en) Systems and method for decoded frame augmentation for video coding for machines with dct filtering for map
CN118742904A (zh) 用于多任务机器学习的视频和特征编码

Legal Events

Date Code Title Description
A521 Request for written amendment filed

Free format text: JAPANESE INTERMEDIATE CODE: A523

Effective date: 20250521

A621 Written request for application examination

Free format text: JAPANESE INTERMEDIATE CODE: A621

Effective date: 20250521

A977 Report on retrieval

Free format text: JAPANESE INTERMEDIATE CODE: A971007

Effective date: 20251226

A131 Notification of reasons for refusal

Free format text: JAPANESE INTERMEDIATE CODE: A131

Effective date: 20260120

A601 Written request for extension of time

Free format text: JAPANESE INTERMEDIATE CODE: A601

Effective date: 20260420