EP4393148A1 - Appareil, procédé et programme informatique pour vidéo volumétrique - Google Patents

Appareil, procédé et programme informatique pour vidéo volumétrique

Info

Publication number
EP4393148A1
EP4393148A1 EP22860687.7A EP22860687A EP4393148A1 EP 4393148 A1 EP4393148 A1 EP 4393148A1 EP 22860687 A EP22860687 A EP 22860687A EP 4393148 A1 EP4393148 A1 EP 4393148A1
Authority
EP
European Patent Office
Prior art keywords
format
volumetric
bitstream
encoded
volumetric representation
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
EP22860687.7A
Other languages
German (de)
English (en)
Inventor
Lukasz Kondrad
Lauri Aleksi ILOLA
Emre Baris Aksu
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Nokia Technologies Oy
Original Assignee
Nokia Technologies Oy
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Nokia Technologies Oy filed Critical Nokia Technologies Oy
Publication of EP4393148A1 publication Critical patent/EP4393148A1/fr
Pending legal-status Critical Current

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
    • H04N21/81—Monomedia components thereof
    • H04N21/816—Monomedia components thereof involving special video data, e.g 3D video
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T9/00—Image coding
    • G06T9/001—Model-based coding, e.g. wire frame
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/10—Processing, recording or transmission of stereoscopic or multi-view image signals
    • H04N13/106—Processing image signals
    • H04N13/161—Encoding, multiplexing or demultiplexing different image signal components
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/20—Image signal generators
    • H04N13/282—Image signal generators for generating image signals corresponding to three or more geometrical viewpoints, e.g. multi-view systems
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/46—Embedding additional information in the video signal during the compression process
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/597—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding specially adapted for multi-view video sequence encoding
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/60—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding
    • H04N19/61—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding in combination with predictive coding
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/70—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by syntax aspects related to video coding, e.g. related to compression standards
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/234—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/236—Assembling of a multiplex stream, e.g. transport stream, by combining a video stream with other content or additional data, e.g. inserting a URL [Uniform Resource Locator] into a video stream, multiplexing software data into a video stream; Remultiplexing of multiplex streams; Insertion of stuffing bits into the multiplex stream, e.g. to obtain a constant bit-rate; Assembling of a packetised elementary stream
    • H04N21/2365—Multiplexing of several video streams
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs

Definitions

  • the present invention relates to an apparatus, a method and a computer program for volumetric video coding.
  • Visual volumetric video-based coding (V3C; defined in ISO/IEC DIS 23090-5) provides a generic syntax and mechanism for volumetric video coding.
  • the generic syntax can be used by applications targeting volumetric content, such as point clouds, immersive video with depth, and mesh representations of volumetric frames.
  • the purpose of the specification is to define how to decode and interpret the associated data (atlas data in ISO/IEC 23090-5) which tells a Tenderer how to interpret 2D frames for reconstructing volumetric frames.
  • V3C i.e. video-based point cloud compression (V-PCC; defined in ISO/IEC 23090-5) and MPEG immersive video (MIV; defined in ISO/IEC 23090-12)
  • V-PCC video-based point cloud compression
  • MIV MPEG immersive video
  • MPEG 3DG ISO SC29 WG7
  • a method comprising obtaining bitstreams of at least two 3D volumetric representations, wherein a first 3D volumetric representation is encoded according to a first format and a second 3D volumetric representation is encoded according to a second format; analyzing content and semantics of syntax elements of said at least two 3D volumetric representations; determining one or more signaling elements, based on the analysis, to indicate differences in the semantics of the syntax elements between the first format and the second format; and merging the bitstreams of said at least two 3D volumetric representations into a multi-format volumetric representation.
  • An apparatus comprises means for obtaining bitstreams of at least two 3D volumetric representations, wherein a first 3D volumetric representation is encoded according to a first format and a second 3D volumetric representation is encoded according to a second format; means for analyzing content and semantics of syntax elements of said at least two 3D volumetric representations; means for determining one or more signaling elements, based on the analysis, to indicate differences in the semantics of the syntax elements between the first format and the second format; and means for merging the bitstreams of said at least two 3D volumetric representations into a bitstream of a multi-format volumetric representation.
  • the apparatus comprises means for adjusting said first and second format-specific sub-encoders to use a same coordinate space.
  • a signaling of differences in the semantics of the syntax elements of the video components of the first format and the second format is configured to be carried out by at least one syntax element included in an atlas frame parameter set extension data syntax structure.
  • said means for merging the bitstreams of said at least two 3D volumetric representations is configured to further merge at least patch data of the first format and the second format into the bitstream of the multi-format volumetric representation.
  • An apparatus comprises means for receiving a bitstream in a decoder, said bitstream comprising at least two 3D volumetric representations, wherein a first 3D volumetric representation is encoded according to a first format and a second 3D volumetric representation is encoded according to a second format; means for receiving, either in said bitstream or in a further bitstream, one or more signaling elements indicating differences in semantics of syntax elements between the first format and the second format; means for separating, from said bitstream, the encoded first 3D volumetric representation to a first format-specific sub-decoder and the encoded second 3D volumetric representation to a second format-specific sub-decoder; and means for decoding the encoded first 3D volumetric representation with the first format-specific subdecoder and the encoded second 3D volumetric representation with the second formatspecific sub-decoder at least partly based on said one or more syntax elements.
  • Figs, la and lb show an encoder and decoder for encoding and decoding 2D pictures
  • FIG. 7 shows an exemplified block chart of an apparatus according to another embodiment
  • FIG. 8 shows an example of input V3C bitstreams and multi-format V3C bitstream according to an embodiment
  • Fig. 10 shows a flow chart for decoding method according to an embodiment.
  • a video codec comprises an encoder that transforms the input video into a compressed representation suited for storage/transmission, and a decoder that can un- compress the compressed video representation back into a viewable form.
  • An encoder may discard some information in the original video sequence in order to represent the video in a more compact form (i.e. at lower bitrate).
  • FIGs, la and lb show an encoder and decoder for encoding and decoding the 2D texture pictures, geometry pictures and/or auxiliary pictures.
  • a video codec consists of an encoder that transforms an input video into a compressed representation suited for storage/transmission and a decoder that can uncompress the compressed video representation back into a viewable form. Typically, the encoder discards and/or loses some information in the original video sequence in order to represent the video in a more compact form (that is, at lower bitrate).
  • An example of an encoding process is illustrated in Figure la.
  • Figure 7 shows an exemplified block chart of an apparatus according to such embodiment.
  • the apparatus obtains the three different 3D volumetric representations in one or more volumetric frames with mixed content.
  • the apparatus analyses the content at least to the extent that the three different 3D volumetric representations can be split and mapped to their format-specific sub-encoders.
  • the example of Figure 7 discloses a first sub-encoder for V3C mesh format, a second subencoder for V3C MIV format and a third sub-encoder for V3C V-PCC format.
  • a vmf content format id may instead be used to signal predefined values.
  • the index of a sub-profile ptl_sub_profile_idc syntax element could be linked in for loop of profile_tier_level( ) syntax structure.
  • said encoding may comprise one or more of the following: encoding source image data into a bitstream, encapsulating the encoded bitstream in a container file and/or in packet(s) or stream(s) of a communication protocol, and announcing or describing the bitstream in a content description, such as the Media Presentation Description (MPD) of ISO/IEC 23009-1 (known as MPEG-DASH) or the IETF Session Description Protocol (SDP).
  • MPD Media Presentation Description
  • SDP IETF Session Description Protocol
  • the various embodiments of the invention may be implemented in hardware or special purpose circuits or any combination thereof. While various aspects of the invention may be illustrated and described as block diagrams or using some other pictorial representation, it is well understood that these blocks, apparatus, systems, techniques or methods described herein may be implemented in, as non-limiting examples, hardware, software, firmware, special purpose circuits or logic, general purpose hardware or controller or other computing devices, or some combination thereof.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)

Abstract

Les modes de réalisation concernent un procédé et un équipement technique permettant de mettre en œuvre le procédé. Le procédé comprend l'obtention de flux binaires d'au moins deux représentations volumétriques 3D, une première représentation volumétrique 3D étant codée selon un premier format et une seconde représentation volumétrique 3D étant codée selon un second format (500) ; l'analyse du contenu et de la sémantique d'éléments de syntaxe desdites au moins deux représentations volumétriques 3D (502) ; la détermination d'un ou plusieurs éléments de signalisation, sur la base de l'analyse, pour indiquer des différences dans la sémantique des éléments de syntaxe entre le premier format et le second format (504) ; et le codage des flux binaires desdites au moins deux représentations volumétriques 3D en une représentation volumétrique multi-format (506).
EP22860687.7A 2021-08-26 2022-06-22 Appareil, procédé et programme informatique pour vidéo volumétrique Pending EP4393148A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
FI20215899 2021-08-26
PCT/FI2022/050445 WO2023025980A1 (fr) 2021-08-26 2022-06-22 Appareil, procédé et programme informatique pour vidéo volumétrique

Publications (1)

Publication Number Publication Date
EP4393148A1 true EP4393148A1 (fr) 2024-07-03

Family

ID=85322537

Family Applications (1)

Application Number Title Priority Date Filing Date
EP22860687.7A Pending EP4393148A1 (fr) 2021-08-26 2022-06-22 Appareil, procédé et programme informatique pour vidéo volumétrique

Country Status (2)

Country Link
EP (1) EP4393148A1 (fr)
WO (1) WO2023025980A1 (fr)

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9036700B2 (en) * 2009-07-15 2015-05-19 Google Technology Holdings LLC Simulcast of stereoviews for 3D TV
JP7436364B2 (ja) * 2018-08-03 2024-02-21 パナソニック インテレクチュアル プロパティ コーポレーション オブ アメリカ 三次元データ符号化方法、三次元データ復号方法、三次元データ符号化装置、及び三次元データ復号装置

Also Published As

Publication number Publication date
WO2023025980A1 (fr) 2023-03-02

Similar Documents

Publication Publication Date Title
EP3614674A1 (fr) Appareil, procédé et programme informatique pour vidéo volumétrique
EP3926960A1 (fr) Dispositif de transmission de données de nuage de points, procédé de transmission de données de nuage de points, dispositif de réception de données de nuage de points et procédé de réception de données de nuage de points
US20210144404A1 (en) Apparatus, a method and a computer program for volumetric video
US12101457B2 (en) Apparatus, a method and a computer program for volumetric video
WO2019135024A1 (fr) Appareil, procédé et programme informatique pour vidéo volumétrique
WO2019158821A1 (fr) Appareil, procédé et programme informatique de vidéo volumétrique
WO2019243663A1 (fr) Appareil, procédé et programme informatique pour vidéo volumétrique
WO2019229293A1 (fr) Appareil, procédé et programme d'ordinateur pour vidéo volumétrique
CN114946179A (zh) 点云数据发送装置、点云数据发送方法、点云数据接收装置和点云数据接收方法
US20220335978A1 (en) An apparatus, a method and a computer program for video coding and decoding
US11895341B2 (en) Point cloud data transmission device, point cloud data transmission method, point cloud data reception device, and point cloud data reception method
CN115398890A (zh) 点云数据发送装置、点云数据发送方法、点云数据接收装置和点云数据接收方法
EP4399877A1 (fr) Appareil, procédé et programme informatique destinés à une vidéo volumétrique
EP4240014A1 (fr) Dispositif de transmission de données de nuage de points, procédé de transmission de données de nuage de points, dispositif de réception de données de nuage de points et procédé de réception de données de nuage de points
US11974026B2 (en) Apparatus, a method and a computer program for volumetric video
US12047604B2 (en) Apparatus, a method and a computer program for volumetric video
US12069314B2 (en) Apparatus, a method and a computer program for volumetric video
WO2021170906A1 (fr) Appareil, procédé et programme informatique pour vidéo volumétrique
JP7416820B2 (ja) ビデオコーディングにおけるヌルタイルコーディング
EP3699867A1 (fr) Appareil, procédé et programme informatique pour vidéo volumétrique
EP3987774A1 (fr) Appareil, procédé et programme informatique pour vidéo volumétrique
WO2021165566A1 (fr) Appareil, procédé et programme informatique pour vidéo volumétrique
WO2023025980A1 (fr) Appareil, procédé et programme informatique pour vidéo volumétrique
EP4145832A1 (fr) Appareil, procédé et programme informatique pour vidéo volumétrique
WO2023041838A1 (fr) Appareil, procédé et programme informatique pour vidéo volumétrique

Legal Events

Date Code Title Description
STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: THE INTERNATIONAL PUBLICATION HAS BEEN MADE

PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: REQUEST FOR EXAMINATION WAS MADE

17P Request for examination filed

Effective date: 20240326

AK Designated contracting states

Kind code of ref document: A1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR