EP1820090A2 - Unterstützung von erweiterungen des wiedergabetreuebereichs in einem erweiterten video-codec-dateiformat - Google Patents
Unterstützung von erweiterungen des wiedergabetreuebereichs in einem erweiterten video-codec-dateiformatInfo
- Publication number
- EP1820090A2 EP1820090A2 EP05811841A EP05811841A EP1820090A2 EP 1820090 A2 EP1820090 A2 EP 1820090A2 EP 05811841 A EP05811841 A EP 05811841A EP 05811841 A EP05811841 A EP 05811841A EP 1820090 A2 EP1820090 A2 EP 1820090A2
- Authority
- EP
- European Patent Office
- Prior art keywords
- parameter set
- multimedia data
- bit depth
- chroma
- chroma format
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Withdrawn
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N11/00—Colour television systems
- H04N11/04—Colour television systems using pulse code modulation
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
- H04N19/186—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being a colour or a chrominance component
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/46—Embedding additional information in the video signal during the compression process
Definitions
- the invention relates generally to the storage and retrieval of audiovisual content in a multimedia file format and particularly to file formats compatible with the ISO media file format.
- the ISO file format was, in turn, used as a template for two standard file formats: (1) the MPEG-4 file format developed by the Moving Picture Experts Group, known as MP4 (ISO/IEC 14496-14, Information Technology ⁇ Coding of audio-visual objects ⁇ Part 14: MP4 File Format); and (2) a file format for JPEG 2000 (ISO/IEC 15444-1), developed by Joint Photographic Experts Group (JPEG).
- the ISO media file format is a hierarchical data structure.
- the data structures contain metadata providing declarative, structural and temporal information about the actual media data.
- the media data itself may be located within the data structure or in the same file or externally in a different file.
- Each metadata stream is called a track.
- the metadata within this, track contains the structural information providing references to the externally framed media data.
- the media data referred to by a meta-data track can be of various types (e.g., video data, audio data, binary format screen representations (BIFS), etc.).
- the externally framed media data is divided into samples (also known as access units or pictures.
- a sample represents a unit of media data at a particular time point and is the smallest data entity which can be represented by timing, location, and other metadata information.
- Each metadata track thereby contains various sample entries and descriptions which provide information about the type of media data being referred to, followed by their timing and location and size information.
- JVT Joint Video Team
- AVC Advanced Video Codec
- VCL Video Coding Layer
- NAL Network Abstraction Layer
- the VCL contains the coding related parts of the codec, such as motion compensation, transform coding of coefficients, and entropy coding.
- the output of the VCL is slices, each of which contains a series of video macroblocks and associated header information.
- the NAL abstracts the VCL from the details of the transport layer used to carry the VCL data.
- the NAL defines a generic and transport independent representation for information, and defines the interface between the video codec itself and the outside world.
- the JVT codec design specifies a set of NAL units, each of which contains different types of data. [0009] In many existing video coding formats, the coded stream data includes various kinds of headers containing parameters that control the decoding process.
- the MPEG-2 video standard includes sequence headers, enhanced group of pictures (GOP), and picture headers before the video data corresponding to those items.
- GOP enhanced group of pictures
- JVT the information needed to decode VCL data is grouped into parameter sets, and JVT defines an NAL unit that transports the parameter sets to the decoder.
- the parameter set NAL units may be sent in the same stream as the video NAL units (in-band) or in a separate stream (out-of-band).
- FRExt fidelity range extensions
- FRExt also specifies extra color spaces, such as the International Commission on Illumination (CIE) XYZ and RBG (red, green, blue) color spaces, in addition to the previously supported YCbCr (yellow, chroma-blue, chroma-red) color space.
- CIE International Commission on Illumination
- RBG red, green, blue
- a parameter set is created to specify chroma format, luma bit depth, and chroma bit depth for a portion of multimedia data.
- the parameter set is encoded into a metadata file that is associated with the multimedia data.
- the parameter set is extracted from the metadata file if a decoder configuration record contains fields corresponding to the parameter set.
- the decoder configuration record is created with fields corresponding to the parameter set.
- Figure 1 is a block diagram of one embodiment of an encoding system
- Figure 2 is a block diagram of one embodiment of a decoding system
- FIG. 3 is a block diagram of a computer environment suitable for practicing the invention.
- Figure 4 is a flow diagram of a method for storing parameter set metadata at an encoding system.
- Figure 5 is a flow diagram of a method for utilizing parameter set metadata at a decoding system.
- the decoder configuration record in the AVC file format is extended to specify the chroma format, luma bit depth, and chroma bit depth for a portion of multimedia data.
- the parameter set associated with a FRExt profiles is encoded into a metadata file that is associated with the multimedia data.
- the parameter set is extracted from the metadata file if the decoder configuration record contains fields corresponding to the presence of FRExt data.
- FIG. 1 illustrates one embodiment of an encoding system 100 that generates parameter set metadata.
- the encoding system 100 includes a media encoder 104, a metadata generator 106 and a file creator 108.
- the media encoder 104 receives media data that may include video data (e.g., video objects created from a natural source video scene and other external video objects), audio data (e.g., audio objects created from a natural source audio scene and other external audio objects), synthetic objects, or any combination of the above.
- the media encoder 104 may consist of a number of individual encoders or include sub- encoders to process various types of media data.
- the media encoder 104 codes the media data and passes it to the metadata generator 106.
- the metadata generator 106 generates metadata that provides information about the media data.
- the metadata is formatted as parameter set NAL units.
- the file creator 108 stores the metadata in a file whose structure is defined by the media file format.
- the media file format may specify that the metadata is stored in-band or entirely or partially out-of band. Coded media data is linked to the out-ot-band metadata by references contained in the metadata file (e.g., via URLs).
- the file created by the file creator 108 is available on a channel 110 for storage or transmission.
- Figure 2 illustrates one embodiment of a decoding system 200 that extracts parameter set metadata.
- the decoding system 200 includes a metadata extractor 204, a media data stream processor 206, a media decoder 210, a compositor 212 and a Tenderer 214.
- the decoding system 200 may reside on a client device and be used for local playback. Alternatively, the decoding system 200 may be used for streaming data, with a server portion and a client portion communicating with each other over a network (e.g., Internet) 208.
- the server portion may include the metadata extractor 204 and the media data stream processor 206.
- the client portion may include the media decoder 210, the compositor 212 and the renderer 214.
- the metadata extractor 204 is responsible for extracting metadata from a file stored in a database 216 or received over a network (e.g., from the encoding system 100).
- a decoder configuration record specifies the metadata that the metadata extractor 204 is capable of handling. Any additional metadata that is not recognized is ignored.
- the extracted metadata is passed to the media data stream processor 206 which also receives the associated coded media data.
- the media data stream processor 206 uses the metadata to form a media data stream to be sent to the media decoder 210.
- the media data stream is formed, it is sent to the media decoder 210 either directly (e.g., for local playback) or over a network 208 (e.g., for streaming data) for decoding.
- the compositor 212 receives the output of the media decoder 210 and composes a scene which is then rendered on a user display device by the renderer 214.
- the metadata may change between the time it is created and the time it is used to decode a corresponding portion of media data. If such a change occurs, the decoding system 200 receives a metadata update packet specifying the change. The state of the metadata before and after the update is applied is maintained in the metadata.
- Figure 3 illustrates one embodiment of a computer system suitable for use as a metadata generator 106 and/or a file creator 108 of Figure 1, or a metadata extractor 204 and/or a media data stream processor 206 of Figure 2.
- the computer system 340 includes a processor 350, memory 355 and input/output capability 360 coupled to a system bus 365.
- the memory 355 is configured to store instructions which, when executed by the processor 350, perform the methods described herein.
- Input/output 360 also encompasses various types of machine-readable media, including any type of storage device that is accessible by the processor 350.
- machine-readable medium/media further encompasses a carrier wave that encodes a data signal.
- the system 340 is controlled by operating system software executing in memory 355.
- Input/output and related media 360 store the computer-executable instructions for the operating system and methods of the present invention.
- Each of the metadata generator 106, the file creator 108, the metadata extractor 204 and the media data stream processor 206 that are shown in Figures 1 and 2 may be a separate component coupled to the processor 350, or may be embodied in computer-executable instructions executed by the processor 350.
- the computer system 340 may be part of, or coupled to, an ISP (Internet Service Provider) through input/output 360 to transmit or receive media data over the Internet.
- ISP Internet Service Provider
- the computer system 340 is one example of many possible computer systems that have different architectures.
- a typical computer system will usually include at least a processor, memory, and a bus coupling the memory to the processor.
- One of skill in the art will immediately appreciate that the invention can be practiced with other computer system configurations, including multiprocessor systems, minicomputers, mainframe computers, and the like.
- the invention can also be practiced in distributed computing environments where tasks are performed by remote processing devices that are linked through a communications network.
- Figures 4 and 5 illustrate processes for storing and retrieving parameter set metadata that are performed by the encoding system 100 and the decoding system 200 respectively.
- the processes may be performed by processing logic that may comprise hardware (e.g., circuitry, dedicated logic, etc.), software (such as run on a general purpose computer system or a dedicated machine), or a combination ot botn. t or s ⁇ ware- implemented processes, the description of a flow diagram enables one skilled in the art to develop such programs including instructions to carry out the processes on suitably configured computers (the processor of the computer executing the instructions from computer-readable media, including memory).
- the computer-executable instructions may be written in a computer programming language or may be embodied in firmware logic. If written in a programming language conforming to a recognized standard, such instructions can be executed on a variety of hardware platforms and for interface to a variety of operating systems.
- FIG. 4 is a flow diagram of one embodiment of a method 400 for creating parameter set metadata at the encoding system 100.
- the processing logic of block 402 receives a file with encoded media data, which includes sets of encoding parameters that specify how to decode portions of the media data.
- the processing logic examines the relationships between the sets of encoding parameters and the corresponding portions of the media data (block 404), and creates metadata defining the parameter sets and their associations with the media data portions (block 406).
- the parameter set metadata is organized into a set of predefined data structures.
- the set of predefined data structures may include a data structure containing descriptive information about the parameter sets, and a data structure containing information that defines associations between media data portions and corresponding parameter sets.
- the processing logic determines whether any parameter set data structure contains a repeated sequence of data (block 408). If this determination is positive, the processing logic converts each repeated sequence of data into a reference to a sequence occurrence and the number of times the sequence occurs (block 410). This type of parameter set is referred to as a sequence parameter set.
- the processing logic incorporates the parameter set metadata in a file associated with media data using a specific media file format (e.g., the AVC file format).
- the parameter set metadata may be in-band or out-of- band.
- FIG. 5 is a flow diagram of one embodiment of a method 500 for utilizing parameter set metadata at the decoding system 200.
- the processing logic at block 502 receives a file associated with encoded media data.
- the file may be received from a database (local or external), the encoding system 100, or from any other device on a network.
- the file includes the parameter set metadata that defines parameter sets for the corresponding media data.
- the processing logic of block 504 extracts the parameter set metadata from the file.
- the processing logic at block 506 uses the extracted metadata to determine which parameter set is associated with a specific media data portion.
- the information in the parameter set controls decoding and transmission time of media data portions and corresponding parameter sets.
- chroma format and bit depth parameters have been created to incorporate the FRExt into the existing AVC sequence parameter sets by the JVT team. If a video sample is in one of the extended chroma formats such as YUV 4:2:2 or 4:4:4, a chroma format indicator, M chroma_format_idc," is included in the corresponding sequence parameter set by the metadata generator 106 of Figure 1 when executing blocks 406 through 410 of method 400.
- the chroma_format_idc parameter specifies the chroma (hue and saturation) sampling relative to the luma (luminosity) sampling and has a value ranging from 0 to 3.
- bit_depth_luma_minus8 specifies the bit depth of the luma samples
- bit_depth_chroma_minus8 specifies the bit depth of the chroma samples.
- the values of the bit_de ⁇ th_luma_minus8 and bit_depth_chroma_minus8 parameters range from O -to 4 according to the following formulas:
- BitDepth 8+ bit_depth_luma_minus8 (1 )
- BitDepth 8+ bit_depth_chroma_minus8 (2)
- a value of zero corresponds to a bit depth of 8 bits, while a value of 4 corresponds to a bit depth of 12 bits.
- chroma_format field contains the chroma format indicator defined by the parameter chroma_format_idc.
- the other two fields contain the corresponding luma and chroma parameter values.
- the modified decoder configuration record controls the extraction of the new FRExt parameters by the metadata extractor 204 as it executes block 505 of method 500.
Applications Claiming Priority (3)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US62075304P | 2004-10-21 | 2004-10-21 | |
US11/255,853 US20070098083A1 (en) | 2005-10-20 | 2005-10-20 | Supporting fidelity range extensions in advanced video codec file format |
PCT/US2005/038255 WO2006047448A2 (en) | 2004-10-21 | 2005-10-21 | Supporting fidelity range extensions in advanced video codec file format |
Publications (1)
Publication Number | Publication Date |
---|---|
EP1820090A2 true EP1820090A2 (de) | 2007-08-22 |
Family
ID=36228345
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
EP05811841A Withdrawn EP1820090A2 (de) | 2004-10-21 | 2005-10-21 | Unterstützung von erweiterungen des wiedergabetreuebereichs in einem erweiterten video-codec-dateiformat |
Country Status (7)
Country | Link |
---|---|
EP (1) | EP1820090A2 (de) |
JP (1) | JP2008518516A (de) |
KR (1) | KR20070084442A (de) |
AU (1) | AU2005299534A1 (de) |
CA (1) | CA2584765A1 (de) |
RU (1) | RU2007118660A (de) |
WO (1) | WO2006047448A2 (de) |
Families Citing this family (10)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
EP2116063B1 (de) * | 2007-01-04 | 2017-03-08 | Thomson Licensing | Verfahren und vorrichtung für in einer high-level-syntax gelieferte mehrfachansichtsinformationen |
CN101578877B (zh) | 2007-01-11 | 2016-05-04 | 汤姆森许可贸易公司 | 对mpeg-4avc高层编码中cavlc4:4:4帧内、高4:4:4帧内和高4:4:4预测简档使用针对coded_block_flag语法元素和coded_block_pattern语法元素的语法的方法和装置 |
KR101394154B1 (ko) | 2007-10-16 | 2014-05-14 | 삼성전자주식회사 | 미디어 컨텐츠 및 메타데이터를 부호화하는 방법과 그 장치 |
KR101692829B1 (ko) | 2008-06-12 | 2017-01-05 | 톰슨 라이센싱 | 감소된 비트 깊이 갱신 모드와 감소된 크로마 샘플링 갱신 모드로 비디오 코딩 및 디코딩하기 위한 방법 및 장치 |
ES2953082T3 (es) * | 2011-06-24 | 2023-11-08 | Ntt Docomo Inc | Métodos para predicción con compensación de movimiento |
US20130195172A1 (en) * | 2012-01-30 | 2013-08-01 | Qualcomm Incorporated | Method of coding video and storing video content |
TW201342884A (zh) | 2012-01-31 | 2013-10-16 | Sony Corp | 編碼裝置及編碼方法、以及解碼裝置及解碼方法 |
US9813705B2 (en) * | 2012-04-26 | 2017-11-07 | Qualcomm Incorporated | Parameter set coding |
JP2017501599A (ja) * | 2013-10-07 | 2017-01-12 | ヴィド スケール インコーポレイテッド | マルチレイヤビデオコーディングのコンバインドスケーラビリティ処理 |
DE112018000795T5 (de) * | 2017-02-13 | 2019-12-12 | Sony Corporation | Bildverarbeitungsvorrichtung und bildverarbeitungsverfahren |
Family Cites Families (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5325423A (en) * | 1992-11-13 | 1994-06-28 | Multimedia Systems Corporation | Interactive multimedia communication system |
US6639945B2 (en) * | 1997-03-14 | 2003-10-28 | Microsoft Corporation | Method and apparatus for implementing motion detection in video compression |
EP1258837A1 (de) * | 2001-05-14 | 2002-11-20 | Thomson Licensing S.A. | Verfahren zur Erzeugung eines gegenseitigen fotometrischen Effektes |
US8978090B2 (en) * | 2003-03-12 | 2015-03-10 | Qualcomm Incorporated | Multimedia transcoding proxy server for wireless telecommunication system |
-
2005
- 2005-10-21 JP JP2007538146A patent/JP2008518516A/ja active Pending
- 2005-10-21 EP EP05811841A patent/EP1820090A2/de not_active Withdrawn
- 2005-10-21 KR KR1020077011552A patent/KR20070084442A/ko not_active Application Discontinuation
- 2005-10-21 RU RU2007118660/09A patent/RU2007118660A/ru not_active Application Discontinuation
- 2005-10-21 WO PCT/US2005/038255 patent/WO2006047448A2/en active Application Filing
- 2005-10-21 AU AU2005299534A patent/AU2005299534A1/en not_active Abandoned
- 2005-10-21 CA CA002584765A patent/CA2584765A1/en not_active Abandoned
Non-Patent Citations (1)
Title |
---|
See references of WO2006047448A3 * |
Also Published As
Publication number | Publication date |
---|---|
KR20070084442A (ko) | 2007-08-24 |
AU2005299534A1 (en) | 2006-05-04 |
WO2006047448A2 (en) | 2006-05-04 |
RU2007118660A (ru) | 2008-11-27 |
CA2584765A1 (en) | 2006-05-04 |
JP2008518516A (ja) | 2008-05-29 |
WO2006047448A3 (en) | 2009-04-16 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
EP1820090A2 (de) | Unterstützung von erweiterungen des wiedergabetreuebereichs in einem erweiterten video-codec-dateiformat | |
KR102014628B1 (ko) | 이미지 시퀀스 트랙을 처리하기 위한 방법, 장치 및 컴퓨터 프로그램 제품 | |
AU2003237120B2 (en) | Supporting advanced coding formats in media files | |
US7613727B2 (en) | Method and apparatus for supporting advanced coding formats in media files | |
US20070098083A1 (en) | Supporting fidelity range extensions in advanced video codec file format | |
US20040006575A1 (en) | Method and apparatus for supporting advanced coding formats in media files | |
US9596430B2 (en) | Data generation apparatus, data generating method, data reproduction apparatus, and data reproducing method | |
US9788020B2 (en) | File generation apparatus, file generating method, file reproduction apparatus, and file reproducing method | |
US20040167925A1 (en) | Method and apparatus for supporting advanced coding formats in media files | |
US10999605B2 (en) | Signaling of important video information in file formats | |
US20060233247A1 (en) | Storing SVC streams in the AVC file format | |
AU2003213554B2 (en) | Method and apparatus for supporting AVC in MP4 | |
US9918099B2 (en) | File generation apparatus, file generating method, file reproduction apparatus, and file reproducing method | |
AU2003228734A1 (en) | Generic adaptation layer for jvt video | |
AU2003213555B2 (en) | Method and apparatus for supporting AVC in MP4 | |
CN101416149A (zh) | 以高级视频编解码器文件格式支持保真度范围扩展 | |
AU2003219877A1 (en) | Method and apparatus for supporting avc in mp4 | |
US20080137733A1 (en) | Encoding device, decoding device, recording device, audio/video data transmission system | |
Mohr | Motion JPEG 2000 and ISO Base Media File Format (Parts 3 and 12) |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PUAI | Public reference made under article 153(3) epc to a published international application that has entered the european phase |
Free format text: ORIGINAL CODE: 0009012 |
|
17P | Request for examination filed |
Effective date: 20070504 |
|
AK | Designated contracting states |
Kind code of ref document: A2 Designated state(s): AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HU IE IS IT LI LT LU LV MC NL PL PT RO SE SI SK TR |
|
AX | Request for extension of the european patent |
Extension state: AL BA HR MK YU |
|
DAX | Request for extension of the european patent (deleted) | ||
RBV | Designated contracting states (corrected) |
Designated state(s): DE FR GB |
|
R17D | Deferred search report published (corrected) |
Effective date: 20090416 |
|
RIC1 | Information provided on ipc code assigned before grant |
Ipc: H04N 7/12 20060101AFI20090423BHEP |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: THE APPLICATION HAS BEEN WITHDRAWN |
|
18W | Application withdrawn |
Effective date: 20110428 |