EP1820090A2 - Prise en charge d'extensions frext dans un format de fichier a codec video evolue - Google Patents

Prise en charge d'extensions frext dans un format de fichier a codec video evolue

Info

Publication number
EP1820090A2
EP1820090A2 EP05811841A EP05811841A EP1820090A2 EP 1820090 A2 EP1820090 A2 EP 1820090A2 EP 05811841 A EP05811841 A EP 05811841A EP 05811841 A EP05811841 A EP 05811841A EP 1820090 A2 EP1820090 A2 EP 1820090A2
Authority
EP
European Patent Office
Prior art keywords
parameter set
multimedia data
bit depth
chroma
chroma format
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Withdrawn
Application number
EP05811841A
Other languages
German (de)
English (en)
Inventor
Mohammed Zubair Visharam
Ali Tabatabai
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Sony Electronics Inc
Original Assignee
Sony Electronics Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority claimed from US11/255,853 external-priority patent/US20070098083A1/en
Application filed by Sony Electronics Inc filed Critical Sony Electronics Inc
Publication of EP1820090A2 publication Critical patent/EP1820090A2/fr
Withdrawn legal-status Critical Current

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N11/00—Colour television systems
    • H04N11/04—Colour television systems using pulse code modulation
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/186—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being a colour or a chrominance component
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/46—Embedding additional information in the video signal during the compression process

Definitions

  • the invention relates generally to the storage and retrieval of audiovisual content in a multimedia file format and particularly to file formats compatible with the ISO media file format.
  • the ISO file format was, in turn, used as a template for two standard file formats: (1) the MPEG-4 file format developed by the Moving Picture Experts Group, known as MP4 (ISO/IEC 14496-14, Information Technology ⁇ Coding of audio-visual objects ⁇ Part 14: MP4 File Format); and (2) a file format for JPEG 2000 (ISO/IEC 15444-1), developed by Joint Photographic Experts Group (JPEG).
  • the ISO media file format is a hierarchical data structure.
  • the data structures contain metadata providing declarative, structural and temporal information about the actual media data.
  • the media data itself may be located within the data structure or in the same file or externally in a different file.
  • Each metadata stream is called a track.
  • the metadata within this, track contains the structural information providing references to the externally framed media data.
  • the media data referred to by a meta-data track can be of various types (e.g., video data, audio data, binary format screen representations (BIFS), etc.).
  • the externally framed media data is divided into samples (also known as access units or pictures.
  • a sample represents a unit of media data at a particular time point and is the smallest data entity which can be represented by timing, location, and other metadata information.
  • Each metadata track thereby contains various sample entries and descriptions which provide information about the type of media data being referred to, followed by their timing and location and size information.
  • JVT Joint Video Team
  • AVC Advanced Video Codec
  • VCL Video Coding Layer
  • NAL Network Abstraction Layer
  • the VCL contains the coding related parts of the codec, such as motion compensation, transform coding of coefficients, and entropy coding.
  • the output of the VCL is slices, each of which contains a series of video macroblocks and associated header information.
  • the NAL abstracts the VCL from the details of the transport layer used to carry the VCL data.
  • the NAL defines a generic and transport independent representation for information, and defines the interface between the video codec itself and the outside world.
  • the JVT codec design specifies a set of NAL units, each of which contains different types of data. [0009] In many existing video coding formats, the coded stream data includes various kinds of headers containing parameters that control the decoding process.
  • the MPEG-2 video standard includes sequence headers, enhanced group of pictures (GOP), and picture headers before the video data corresponding to those items.
  • GOP enhanced group of pictures
  • JVT the information needed to decode VCL data is grouped into parameter sets, and JVT defines an NAL unit that transports the parameter sets to the decoder.
  • the parameter set NAL units may be sent in the same stream as the video NAL units (in-band) or in a separate stream (out-of-band).
  • FRExt fidelity range extensions
  • FRExt also specifies extra color spaces, such as the International Commission on Illumination (CIE) XYZ and RBG (red, green, blue) color spaces, in addition to the previously supported YCbCr (yellow, chroma-blue, chroma-red) color space.
  • CIE International Commission on Illumination
  • RBG red, green, blue
  • a parameter set is created to specify chroma format, luma bit depth, and chroma bit depth for a portion of multimedia data.
  • the parameter set is encoded into a metadata file that is associated with the multimedia data.
  • the parameter set is extracted from the metadata file if a decoder configuration record contains fields corresponding to the parameter set.
  • the decoder configuration record is created with fields corresponding to the parameter set.
  • Figure 1 is a block diagram of one embodiment of an encoding system
  • Figure 2 is a block diagram of one embodiment of a decoding system
  • FIG. 3 is a block diagram of a computer environment suitable for practicing the invention.
  • Figure 4 is a flow diagram of a method for storing parameter set metadata at an encoding system.
  • Figure 5 is a flow diagram of a method for utilizing parameter set metadata at a decoding system.
  • the decoder configuration record in the AVC file format is extended to specify the chroma format, luma bit depth, and chroma bit depth for a portion of multimedia data.
  • the parameter set associated with a FRExt profiles is encoded into a metadata file that is associated with the multimedia data.
  • the parameter set is extracted from the metadata file if the decoder configuration record contains fields corresponding to the presence of FRExt data.
  • FIG. 1 illustrates one embodiment of an encoding system 100 that generates parameter set metadata.
  • the encoding system 100 includes a media encoder 104, a metadata generator 106 and a file creator 108.
  • the media encoder 104 receives media data that may include video data (e.g., video objects created from a natural source video scene and other external video objects), audio data (e.g., audio objects created from a natural source audio scene and other external audio objects), synthetic objects, or any combination of the above.
  • the media encoder 104 may consist of a number of individual encoders or include sub- encoders to process various types of media data.
  • the media encoder 104 codes the media data and passes it to the metadata generator 106.
  • the metadata generator 106 generates metadata that provides information about the media data.
  • the metadata is formatted as parameter set NAL units.
  • the file creator 108 stores the metadata in a file whose structure is defined by the media file format.
  • the media file format may specify that the metadata is stored in-band or entirely or partially out-of band. Coded media data is linked to the out-ot-band metadata by references contained in the metadata file (e.g., via URLs).
  • the file created by the file creator 108 is available on a channel 110 for storage or transmission.
  • Figure 2 illustrates one embodiment of a decoding system 200 that extracts parameter set metadata.
  • the decoding system 200 includes a metadata extractor 204, a media data stream processor 206, a media decoder 210, a compositor 212 and a Tenderer 214.
  • the decoding system 200 may reside on a client device and be used for local playback. Alternatively, the decoding system 200 may be used for streaming data, with a server portion and a client portion communicating with each other over a network (e.g., Internet) 208.
  • the server portion may include the metadata extractor 204 and the media data stream processor 206.
  • the client portion may include the media decoder 210, the compositor 212 and the renderer 214.
  • the metadata extractor 204 is responsible for extracting metadata from a file stored in a database 216 or received over a network (e.g., from the encoding system 100).
  • a decoder configuration record specifies the metadata that the metadata extractor 204 is capable of handling. Any additional metadata that is not recognized is ignored.
  • the extracted metadata is passed to the media data stream processor 206 which also receives the associated coded media data.
  • the media data stream processor 206 uses the metadata to form a media data stream to be sent to the media decoder 210.
  • the media data stream is formed, it is sent to the media decoder 210 either directly (e.g., for local playback) or over a network 208 (e.g., for streaming data) for decoding.
  • the compositor 212 receives the output of the media decoder 210 and composes a scene which is then rendered on a user display device by the renderer 214.
  • the metadata may change between the time it is created and the time it is used to decode a corresponding portion of media data. If such a change occurs, the decoding system 200 receives a metadata update packet specifying the change. The state of the metadata before and after the update is applied is maintained in the metadata.
  • Figure 3 illustrates one embodiment of a computer system suitable for use as a metadata generator 106 and/or a file creator 108 of Figure 1, or a metadata extractor 204 and/or a media data stream processor 206 of Figure 2.
  • the computer system 340 includes a processor 350, memory 355 and input/output capability 360 coupled to a system bus 365.
  • the memory 355 is configured to store instructions which, when executed by the processor 350, perform the methods described herein.
  • Input/output 360 also encompasses various types of machine-readable media, including any type of storage device that is accessible by the processor 350.
  • machine-readable medium/media further encompasses a carrier wave that encodes a data signal.
  • the system 340 is controlled by operating system software executing in memory 355.
  • Input/output and related media 360 store the computer-executable instructions for the operating system and methods of the present invention.
  • Each of the metadata generator 106, the file creator 108, the metadata extractor 204 and the media data stream processor 206 that are shown in Figures 1 and 2 may be a separate component coupled to the processor 350, or may be embodied in computer-executable instructions executed by the processor 350.
  • the computer system 340 may be part of, or coupled to, an ISP (Internet Service Provider) through input/output 360 to transmit or receive media data over the Internet.
  • ISP Internet Service Provider
  • the computer system 340 is one example of many possible computer systems that have different architectures.
  • a typical computer system will usually include at least a processor, memory, and a bus coupling the memory to the processor.
  • One of skill in the art will immediately appreciate that the invention can be practiced with other computer system configurations, including multiprocessor systems, minicomputers, mainframe computers, and the like.
  • the invention can also be practiced in distributed computing environments where tasks are performed by remote processing devices that are linked through a communications network.
  • Figures 4 and 5 illustrate processes for storing and retrieving parameter set metadata that are performed by the encoding system 100 and the decoding system 200 respectively.
  • the processes may be performed by processing logic that may comprise hardware (e.g., circuitry, dedicated logic, etc.), software (such as run on a general purpose computer system or a dedicated machine), or a combination ot botn. t or s ⁇ ware- implemented processes, the description of a flow diagram enables one skilled in the art to develop such programs including instructions to carry out the processes on suitably configured computers (the processor of the computer executing the instructions from computer-readable media, including memory).
  • the computer-executable instructions may be written in a computer programming language or may be embodied in firmware logic. If written in a programming language conforming to a recognized standard, such instructions can be executed on a variety of hardware platforms and for interface to a variety of operating systems.
  • FIG. 4 is a flow diagram of one embodiment of a method 400 for creating parameter set metadata at the encoding system 100.
  • the processing logic of block 402 receives a file with encoded media data, which includes sets of encoding parameters that specify how to decode portions of the media data.
  • the processing logic examines the relationships between the sets of encoding parameters and the corresponding portions of the media data (block 404), and creates metadata defining the parameter sets and their associations with the media data portions (block 406).
  • the parameter set metadata is organized into a set of predefined data structures.
  • the set of predefined data structures may include a data structure containing descriptive information about the parameter sets, and a data structure containing information that defines associations between media data portions and corresponding parameter sets.
  • the processing logic determines whether any parameter set data structure contains a repeated sequence of data (block 408). If this determination is positive, the processing logic converts each repeated sequence of data into a reference to a sequence occurrence and the number of times the sequence occurs (block 410). This type of parameter set is referred to as a sequence parameter set.
  • the processing logic incorporates the parameter set metadata in a file associated with media data using a specific media file format (e.g., the AVC file format).
  • the parameter set metadata may be in-band or out-of- band.
  • FIG. 5 is a flow diagram of one embodiment of a method 500 for utilizing parameter set metadata at the decoding system 200.
  • the processing logic at block 502 receives a file associated with encoded media data.
  • the file may be received from a database (local or external), the encoding system 100, or from any other device on a network.
  • the file includes the parameter set metadata that defines parameter sets for the corresponding media data.
  • the processing logic of block 504 extracts the parameter set metadata from the file.
  • the processing logic at block 506 uses the extracted metadata to determine which parameter set is associated with a specific media data portion.
  • the information in the parameter set controls decoding and transmission time of media data portions and corresponding parameter sets.
  • chroma format and bit depth parameters have been created to incorporate the FRExt into the existing AVC sequence parameter sets by the JVT team. If a video sample is in one of the extended chroma formats such as YUV 4:2:2 or 4:4:4, a chroma format indicator, M chroma_format_idc," is included in the corresponding sequence parameter set by the metadata generator 106 of Figure 1 when executing blocks 406 through 410 of method 400.
  • the chroma_format_idc parameter specifies the chroma (hue and saturation) sampling relative to the luma (luminosity) sampling and has a value ranging from 0 to 3.
  • bit_depth_luma_minus8 specifies the bit depth of the luma samples
  • bit_depth_chroma_minus8 specifies the bit depth of the chroma samples.
  • the values of the bit_de ⁇ th_luma_minus8 and bit_depth_chroma_minus8 parameters range from O -to 4 according to the following formulas:
  • BitDepth 8+ bit_depth_luma_minus8 (1 )
  • BitDepth 8+ bit_depth_chroma_minus8 (2)
  • a value of zero corresponds to a bit depth of 8 bits, while a value of 4 corresponds to a bit depth of 12 bits.
  • chroma_format field contains the chroma format indicator defined by the parameter chroma_format_idc.
  • the other two fields contain the corresponding luma and chroma parameter values.
  • the modified decoder configuration record controls the extraction of the new FRExt parameters by the metadata extractor 204 as it executes block 505 of method 500.

Abstract

Selon l'invention, un ensemble de paramètres est créé pour spécifier un format de chrominance, une profondeur de bits de luminance et une profondeur de bits de chrominance pour une partie de données multimédia. Cet ensemble de paramètres est codé en un fichier de métadonnées associé aux données multimédia. L'ensemble de paramètres est extrait du fichier de métadonnées si un enregistrement de configuration de décodeur contient des champs correspondant à l'ensemble de paramètres. Dans un autre aspect, l'enregistrement de configuration de décodeur est créé avec des champs correspondant à l'ensemble de paramètres.
EP05811841A 2004-10-21 2005-10-21 Prise en charge d'extensions frext dans un format de fichier a codec video evolue Withdrawn EP1820090A2 (fr)

Applications Claiming Priority (3)

Application Number Priority Date Filing Date Title
US62075304P 2004-10-21 2004-10-21
US11/255,853 US20070098083A1 (en) 2005-10-20 2005-10-20 Supporting fidelity range extensions in advanced video codec file format
PCT/US2005/038255 WO2006047448A2 (fr) 2004-10-21 2005-10-21 Prise en charge d'extensions frext dans un format de fichier a codec video evolue

Publications (1)

Publication Number Publication Date
EP1820090A2 true EP1820090A2 (fr) 2007-08-22

Family

ID=36228345

Family Applications (1)

Application Number Title Priority Date Filing Date
EP05811841A Withdrawn EP1820090A2 (fr) 2004-10-21 2005-10-21 Prise en charge d'extensions frext dans un format de fichier a codec video evolue

Country Status (7)

Country Link
EP (1) EP1820090A2 (fr)
JP (1) JP2008518516A (fr)
KR (1) KR20070084442A (fr)
AU (1) AU2005299534A1 (fr)
CA (1) CA2584765A1 (fr)
RU (1) RU2007118660A (fr)
WO (1) WO2006047448A2 (fr)

Families Citing this family (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2008085885A2 (fr) * 2007-01-04 2008-07-17 Thomson Licensing Procédés et appareil pour information multi-vue portée en syntaxe de haut niveau
CN101578877B (zh) 2007-01-11 2016-05-04 汤姆森许可贸易公司 对mpeg-4avc高层编码中cavlc4:4:4帧内、高4:4:4帧内和高4:4:4预测简档使用针对coded_block_flag语法元素和coded_block_pattern语法元素的语法的方法和装置
KR101394154B1 (ko) 2007-10-16 2014-05-14 삼성전자주식회사 미디어 컨텐츠 및 메타데이터를 부호화하는 방법과 그 장치
KR101692829B1 (ko) * 2008-06-12 2017-01-05 톰슨 라이센싱 감소된 비트 깊이 갱신 모드와 감소된 크로마 샘플링 갱신 모드로 비디오 코딩 및 디코딩하기 위한 방법 및 장치
KR101748810B1 (ko) * 2011-06-24 2017-06-19 가부시키가이샤 엔.티.티.도코모 모션 보상 예측을 위한 방법 및 장치
US9648317B2 (en) * 2012-01-30 2017-05-09 Qualcomm Incorporated Method of coding video and storing video content
TW201342884A (zh) 2012-01-31 2013-10-16 Sony Corp 編碼裝置及編碼方法、以及解碼裝置及解碼方法
US9813705B2 (en) * 2012-04-26 2017-11-07 Qualcomm Incorporated Parameter set coding
TWI652937B (zh) * 2013-10-07 2019-03-01 Ｖｉｄ衡器股份有限公司 多層視訊編碼組合可調能力處理方法
US11184602B2 (en) * 2017-02-13 2021-11-23 Sony Corporation Image processing apparatus and image processing method

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5325423A (en) * 1992-11-13 1994-06-28 Multimedia Systems Corporation Interactive multimedia communication system
US6639945B2 (en) * 1997-03-14 2003-10-28 Microsoft Corporation Method and apparatus for implementing motion detection in video compression
EP1258837A1 (fr) * 2001-05-14 2002-11-20 Thomson Licensing S.A. Procédé pour causer un effet photométrique
US8978090B2 (en) * 2003-03-12 2015-03-10 Qualcomm Incorporated Multimedia transcoding proxy server for wireless telecommunication system

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See references of WO2006047448A3 *

Also Published As

Publication number Publication date
KR20070084442A (ko) 2007-08-24
CA2584765A1 (fr) 2006-05-04
WO2006047448A2 (fr) 2006-05-04
AU2005299534A1 (en) 2006-05-04
RU2007118660A (ru) 2008-11-27
WO2006047448A3 (fr) 2009-04-16
JP2008518516A (ja) 2008-05-29

Similar Documents

Publication Publication Date Title
WO2006047448A2 (fr) Prise en charge d'extensions frext dans un format de fichier a codec video evolue
KR102014628B1 (ko) 이미지 시퀀스 트랙을 처리하기 위한 방법, 장치 및 컴퓨터 프로그램 제품
AU2003237120B2 (en) Supporting advanced coding formats in media files
US7613727B2 (en) Method and apparatus for supporting advanced coding formats in media files
US20070098083A1 (en) Supporting fidelity range extensions in advanced video codec file format
US20040006575A1 (en) Method and apparatus for supporting advanced coding formats in media files
US9596430B2 (en) Data generation apparatus, data generating method, data reproduction apparatus, and data reproducing method
US9788020B2 (en) File generation apparatus, file generating method, file reproduction apparatus, and file reproducing method
US20040167925A1 (en) Method and apparatus for supporting advanced coding formats in media files
US10999605B2 (en) Signaling of important video information in file formats
US20060233247A1 (en) Storing SVC streams in the AVC file format
US20030163477A1 (en) Method and apparatus for supporting advanced coding formats in media files
US9918099B2 (en) File generation apparatus, file generating method, file reproduction apparatus, and file reproducing method
AU2003228734A1 (en) Generic adaptation layer for jvt video
AU2003213555B2 (en) Method and apparatus for supporting AVC in MP4
CN101416149A (zh) 以高级视频编解码器文件格式支持保真度范围扩展
AU2003219877A1 (en) Method and apparatus for supporting avc in mp4
US20080137733A1 (en) Encoding device, decoding device, recording device, audio/video data transmission system

Legal Events

Date Code Title Description
PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

17P Request for examination filed

Effective date: 20070504

AK Designated contracting states

Kind code of ref document: A2

Designated state(s): AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HU IE IS IT LI LT LU LV MC NL PL PT RO SE SI SK TR

AX Request for extension of the european patent

Extension state: AL BA HR MK YU

DAX Request for extension of the european patent (deleted)
RBV Designated contracting states (corrected)

Designated state(s): DE FR GB

R17D Deferred search report published (corrected)

Effective date: 20090416

RIC1 Information provided on ipc code assigned before grant

Ipc: H04N 7/12 20060101AFI20090423BHEP

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: THE APPLICATION HAS BEEN WITHDRAWN

18W Application withdrawn

Effective date: 20110428