WO2009048235A2 - Système et procédé pour service de contenus multimédia 3d au moyen d'un format de fichier pour application multimédia - Google Patents

Système et procédé pour service de contenus multimédia 3d au moyen d'un format de fichier pour application multimédia Download PDF

Info

Publication number
WO2009048235A2
WO2009048235A2 PCT/KR2008/005791 KR2008005791W WO2009048235A2 WO 2009048235 A2 WO2009048235 A2 WO 2009048235A2 KR 2008005791 W KR2008005791 W KR 2008005791W WO 2009048235 A2 WO2009048235 A2 WO 2009048235A2
Authority
WO
WIPO (PCT)
Prior art keywords
content
information
image
file format
video
Prior art date
Application number
PCT/KR2008/005791
Other languages
English (en)
Other versions
WO2009048235A3 (fr
Inventor
Jeong Yeon Lim
Young Ho Joo
Won Jun Lee
Jae Won Moon
Ki Mun Kim
Min Seok Kim
Original Assignee
Sk Telecom Co., Ltd.
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Sk Telecom Co., Ltd. filed Critical Sk Telecom Co., Ltd.
Publication of WO2009048235A2 publication Critical patent/WO2009048235A2/fr
Publication of WO2009048235A3 publication Critical patent/WO2009048235A3/fr

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q50/00—Information and communication technology [ICT] specially adapted for implementation of business processes of specific business sectors, e.g. utilities or tourism
    • G06Q50/10—Services
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/4302—Content synchronisation processes, e.g. decoder synchronisation
    • H04N21/4307—Synchronising the rendering of multiple content streams or additional data on devices, e.g. synchronisation of audio on a mobile phone with the video output on the TV screen
    • H04N21/43074—Synchronising the rendering of multiple content streams or additional data on devices, e.g. synchronisation of audio on a mobile phone with the video output on the TV screen of additional data with content streams on the same device, e.g. of EPG data or interactive icon with a TV program
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/234—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs
    • H04N21/2343—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements
    • H04N21/234318—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements by decomposing into objects, e.g. MPEG-4 objects
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/10—Processing, recording or transmission of stereoscopic or multi-view image signals
    • H04N13/106—Processing image signals
    • H04N13/172—Processing image signals image signals comprising non-image signal components, e.g. headers or format information
    • H04N13/178—Metadata, e.g. disparity information
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/234—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs
    • H04N21/2343—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements
    • H04N21/23439—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements for generating different versions
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/25—Management operations performed by the server for facilitating the content distribution or administrating data related to end-users or client devices, e.g. end-user or client device authentication, learning user preferences for recommending movies
    • H04N21/266—Channel or content management, e.g. generation and management of keys and entitlement messages in a conditional access system, merging a VOD unicast channel into a multicast channel
    • H04N21/2662—Controlling the complexity of the video stream, e.g. by scaling the resolution or bitrate of the video stream based on the client capabilities
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
    • H04N21/81—Monomedia components thereof
    • H04N21/8146—Monomedia components thereof involving graphical data, e.g. 3D object, 2D graphics
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
    • H04N21/85—Assembly of content; Generation of multimedia applications
    • H04N21/854—Content authoring
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
    • H04N21/85—Assembly of content; Generation of multimedia applications
    • H04N21/854—Content authoring
    • H04N21/85406—Content authoring involving a specific file format, e.g. MP4 format
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
    • H04N21/85—Assembly of content; Generation of multimedia applications
    • H04N21/854—Content authoring
    • H04N21/8547—Content authoring involving timestamps for synchronizing content
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/10—Processing, recording or transmission of stereoscopic or multi-view image signals
    • H04N13/106—Processing image signals
    • H04N13/161—Encoding, multiplexing or demultiplexing different image signal components

Definitions

  • the present invention relates, in general, to a system for providing a 3-Dimensional
  • (3D) multimedia content service using an integrated multimedia file structure and, more particularly, to a system and method for providing a 3D multimedia content service using an integrated multimedia file structure, which provides an integrated multimedia file structure including information required to reproduce or modify 3D images.
  • multimedia application file format technology which is one of the fields of standardization that is currently being conducted by a Moving Picture Expert Group (MPEG) standardization group, is technology for an object-oriented standard configured to overcome the limitations of existing universal standards and to aim at a specific application service.
  • MPEG-A MPEG- Application
  • multimedia application format International Organization for Standardization (ISO)/ International Electrotechnical Commission (IEC) 23000
  • ISO International Organization for Standardization
  • IEC International Electrotechnical Commission
  • Standards for a multimedia application file format deviate from the definition of application tool sets having profile concepts using only technology for existing MPEG element standards, and provides a framework defined as a single integrated file application format in which non-MPEG component standards are included in a file format, so that various characteristics of respective element standards can be provided for an application service in various forms.
  • MAF is intended to provide rich media by allowing both multimedia content, which includes video, audio, an image or text, and metadata, which enables temporal synchronization or does not require temporal synchronization, to be included in a single file format, which enables the exchange, management, editing, and representation of media content, and to be used in various types of application services suitable for standards.
  • a service provider requires a file format which can be provided in the form of a single file, including information about cameras, such as the distance to and the locations of cameras, correction information, the number of cameras and depth information, and reproduction information required to combine a plurality of pieces of video, generate them into a single 3D video, and then reproduce the 3D video, together with composed 3D video content.
  • an object of the present invention is to provide a system and method for providing a 3D multimedia content service using an integrated multimedia file structure, which provides an integrated multimedia file structure including information required to reproduce or modify 3D content.
  • Another object of the present invention is to provide an integrated multimedia file structure, which includes content information required to reproduce 3D video in the form of 2D video when the 3D video is reproduced on a 2D reproduction terminal, and information about combination with 2D video or about modification of 3D video so that 3D video segments can be reproduced together with 2D video segments in a single file.
  • the present invention provides a system for providing a 3-Dimensional (3D) multimedia content service using an integrated multimedia file structure, the system including a 3D content authoring server for authoring an integrated multimedia content file format by containing any one of information used to reproduce 3D content, image conversion information used to select a single view from among a plurality of images for 3D content so as to reproduce the 3D content on a 2D content reproduction terminal, information used to combine 3D content segments with 2D content segments into a single file using the temporal syn- chronization information of a file format, information used to combine 3D content and 2D content using script data so that resulting content is reproduced over time, information used to combine with script data which provides reality by applying animation effect to 3D content, and the combinations thereof; and one or more user terminals each for requesting and receiving integrated multimedia content from the 3D content authoring server, detecting the authoring information of the integrated multimedia content, and then executing the integrated multimedia content.
  • a 3D content authoring server for authoring an integrated multimedia content file format by
  • the integrated multimedia content file format include compressed
  • 3D video/image content 2D video/image content, audio, text, script, metadata including content information, and 3D video/image authoring, reproduction and modification information.
  • the type of video of the integrated multimedia content file format include at least one of the type of video in which 3D video/image content is separately reproduced based on temporal synchronization, the type of video in which 3D video/ image content can be converted into 2D video/image content, that is, the 3D video/ image content is compatible with the 2D video/image content, and the type of video in which 3D video/image content is combined with 2D video/image content in a composite manner.
  • the 3D video/image authoring and transformation information include camera information, image information, and display information; and that the camera information includes camera setting information, camera position information, information about positions between a plurality of cameras, and correction information, the image information include a plurality of image composition methods, and image position information, pixel position information, image size information and encoding/decoding information, which are required based on the plurality of image composition methods, and the display information includes information about the user terminals and information about image size difference generated depending on distance between images.
  • the camera information, the image information, ant the display information be defined in parameter forms and then recorded in the specific boxes of the integrated multimedia content file format, or represented in metadata forms and then recorded in the meta box of the integrated multimedia content file format.
  • the integrated multimedia content file format be configured using at least one of an International Organization for Standardization (ISO) file format, an Moving Picture Expert Group-4 (MPEG-4) file format, an apple quick time file format, an Adobe file format, and a Skeletal Model (SKM) file format.
  • ISO International Organization for Standardization
  • MPEG-4 Moving Picture Expert Group-4
  • apple quick time file format an apple quick time file format
  • Adobe file format an Adobe file format
  • Skeletal Model (SKM) file format a Skeletal Model
  • script data be configured in conformity with any one standard of MPEG-4 Binary Format for Scene (BIFS), MPEG-4 Lightweight Application Scene Representation (LASeR), Synchronized Multimedia Integration Language (SMIL), Scalable Vector Graphics (SVC), SVC Tiny, and an Extensible Markup Language (XML).
  • BIFS MPEG-4 Binary Format for Scene
  • LAR MPEG-4 Lightweight Application Scene Representation
  • SMIL Synchronized Multimedia Integration Language
  • SVC Scalable Vector Graphics
  • SVC Tiny SVC Tiny
  • XML Extensible Markup Language
  • Metadata be written in any one form of MPEG-21, MPEG-7,
  • TV Television-Anytime, and combinations thereof.
  • the present invention provides a 3D content authoring server for generating an integrated multimedia content file format including the authoring information of 3D content, the 3D content authoring server including an image obtainment unit for taking and obtaining images selected by a user; a user command input unit for, when a command signal input based on the selection of the user is received, transmitting the command signal to a relevant configuration unit so that a relevant command is executed; an information recording unit for storing information related to authoring, editing, and configuration of 3D content; an editing image configuration unit for configuring an application section for 2D content and 3D content in a single piece of content; an image editing unit for editing image content received from the image obtainment unit based on content configuration information configured by the editing image configuration unit; a metadata generation unit for generating content authoring information, received by the user command input unit, the editing image configuration unit and the image editing unit, in a metadata form; and a file format generation unit for configuring content reproduction information and metadata, which are generated when content is authored, in a single integrated multimedia content file format.
  • the 3D content authoring server further include a file transmission unit for providing the corresponding integrated multimedia content file format, g enerated by the file format generation unit, in response to a content request received from a user terminal.
  • the integrated multimedia content file format include a file type box (ftyp box) containing brand-declared information indicative of a 3D video- included multimedia application format, a movie box containing content-related information, a meta box (meta box) containing information about configuration which does not require temporal synchronization within the file format and information about content representation, and a media data box (mdat box) containing actual multimedia content; that the moov box include at least one track box (trak box) containing a plurality of pieces of single-type multimedia content, multimedia content encoding information, pointing information, information about reproduction of multimedia content requiring the temporal synchronization; and that the meta box include information about the locations of 2D and 3D video which constitute a file and information about content itself, and includes an xml box (xml box) containing content author informa tion, camera information, reproduction information and content information, which are written in metadata.
  • a file type box containing brand-declared information indicative of a 3D video- included multimedia application format
  • the integrated multimedia content file format include a file type box (ftyp box) containing brand-declared information indicative of a 3D video- included multimedia application format, a movie box containing content-related information, a meta box (meta box) containing information about configuration which does not require temporal synchronization within the file format and information about content representation, and a media data box (mdat box) containing actual multimedia content; that the moov box include a plurality of trak boxes, and each of the trak boxes comprise metadata containing information about each single piece of content; and that the combinatory representation of 2D and 3D video be realized by using temporal synchronization included in a sample table box (stbl box) within the trak box.
  • ftyp box file type box
  • metadata box containing information about configuration which does not require temporal synchronization within the file format and information about content representation
  • midat box media data box
  • the present invention provides a user terminal for executing integrated multimedia content, the user terminal being connected to a 3D content authoring server for generating an integrated multimedia content file format including the authoring information of 3D content, the user terminal including an image reception unit for requesting and receiving integrated multimedia content from the 3D content authoring server; a file format parsing unit for parsing a file format in order to determine the configuration information of the integrated multimedia content received by the image reception unit; a metadata analysis unit for analyzing metadata in order to detect authoring information based on data included in the integrated multimedia content; a terminal performance determination unit for determining whether the user terminal supports 3D content; an image conversion determination unit for determining whether to execute 3D content without change, or to select a single piece of 2D video content from among a plurality of pieces of 2D video content which constitutes the 3D content and then reproduce the selected 2D video content, depending on terminal performance; and a rendering unit for reproducing image content, determined based on the terminal performance, on the user terminal as the integrated multimedia content using file format information, analyzed
  • the image conversion determination unit analyze images based on the file format information and the metadata of the 3D content of the corresponding integrated multimedia content, and then converts the images into 2D content.
  • the present invention provides a method of providing a 3D multimedia content service using an integrated multimedia file structure, the method being used to generate an integrated multimedia content file format using a 3D content authoring server for generating the integrated multimedia content file format including the authoring information of 3D content, the method including the steps of: a) the 3D content authoring server receiving 3D content or one or more pieces of 2D content which are encoded based on the selection of a user; b) configuring an application section for the 2D and 3D content segments in a single piece of content; c) editing image content received at step a) based on content configuration information configured at step b); d) generating content authoring information generated at steps b) and c) in a metadata form; and e) configuring content configuration information, content reproduction information and metadata, which are generated when content is authored, in a single integrated multimedia content file format from.
  • the present invention provides a method of providing a 3D multimedia content service using an integrated multimedia file structure, the method being used to execute integrated multimedia content using a user terminal, the user terminal being connected to a 3D content authoring server for generating an integrated multimedia content file format including the authoring information of 3D content, the method including the steps of: a) the user terminal requesting and receiving the integrated multimedia content from the 3D content authoring server; b) checking information about the 2D content and 3D content of the integrated multimedia content, and then parsing the file format; c) analyzing the metadata of the integrated multimedia content; and d) reproducing the integrated multimedia content based on information recorded in the file format which is detected at step b) and based on the metadata analyzed at step c).
  • the present invention is configured such that 2D video and 3D video can be implemented in a single file in a composite manner, and is configured to provide an integrated multimedia file structure, including information about combination with 2D video segments or about the modification of 3D video so that the 3D video can be utilized with the 2D video in a composite manner, various authoring methods can be provided, so that there is an advantage of satisfying the content consumption desires of users.
  • FIG. 1 is a diagram showing the relationship between a system for providing a 3D multimedia content service according to the present invention and a communication network;
  • FIG. 2 is a diagram showing the configuration of a 3D content authoring server according to the present invention in detail;
  • FIGS 3 to 5 are views showing the structure of an integrated multimedia content file format;
  • FIG. 6 is a diagram showing the configuration of a user terminal according to the present invention in detail;
  • FIG. 7 is a flowchart showing an example of a method of providing a 3D multimedia content service according to the present invention;
  • FIG. 8 is a flowchart showing another example of a method of providing a 3D multimedia content service according to the present invention.
  • FIG. 1 is a diagram showing the relationship between a system for providing a
  • the system for providing a 3D multimedia content service includes a 3D content authoring server 100 for authoring an integrated multimedia content file format by containing any one of information used to reproduce 3D content, image conversion information used to select a single view from among a plurality of images for 3D content so as to reproduce the 3D content on a 2D content reproduction terminal, information used to combine 3D content segments with 2D content segments into a single file using the temporal synchronization information of a file format, information used to combine 3D content and 2D content using script data so that the resulting content is reproduced over time, information used to combine with script data which provides reality by applying animation effect to 3D content, and the combinations thereof; and a user terminal 200 for requesting and receiving integrated multimedia content from the 3D content authoring server 100, detecting the authoring information of the integrated multimedia content, and then executing the integrated multimedia content.
  • the integrated multimedia content file format includes compressed 3D video/image content, 2D video/image content, audio, text, script, metadata including content information, and 3D video/image authoring and modification information.
  • the type of content of the integrated multimedia content file format includes the type of content in which 3D video/image content is separately reproduced based on temporal synchronization, the type of content in which 3D video/image content can be converted into 2D video/image content, that is, the 3D video/image content is compatible with the 2D video/image content, and the type of content in which 3D video/image content is combined with 2D video/image content in a composite manner.
  • script data is configured in conformity with any one standard of MPEG-4
  • BIFS Binary Format for Scene
  • LASeR Lightweight Application Scene Representation
  • SMIL Synchronized Multimedia Integration Language
  • SVC Scalable Vector Graphics
  • SVC Tiny SVC Tiny
  • XML Extensible Markup Language
  • the metadata is written in any one of MPEG-21, MPEG-7, Television
  • the 3D video/image authoring and modification information includes camera information, image information, and display information.
  • the camera information includes camera setting information, camera position information, information about the positions between a plurality of cameras, and correction information.
  • the image information includes a plurality of image composition methods, and image position information, pixel position information, image size information and encoding/decoding information, which are required based on the plurality of image composition methods.
  • the display information includes information about user terminals and information about image size difference generated depending on the distance between images.
  • the camera information, the image information, and the display information are defined in the form of parameters and then recorded in the specific boxes of the integrated multimedia content file format, or represented in the form of metadata and then recorded in the meta box of the integrated multimedia content file format.
  • the integrated multimedia content file format is configured using at least one of an
  • FIG. 2 is a diagram showing the configuration of the 3D content authoring server according to the present invention in detail.
  • the 3D content authoring server 100 includes an image obtainment unit 110, a user command input unit 120, an editing image configuration unit 130, an image editing unit 140, an information recording unit 150, a metadata generation unit 160, a file format generation unit 170, and a file transmission unit 180.
  • the image obtainment unit 110 takes and obtains images selected by a user.
  • the images correspond to 3D content or one or more pieces of 2D content taken from various angles.
  • the user command input unit 120 transmits it to a relevant configuration unit so that the command is executed.
  • the editing image configuration unit 130 configures an application section for 2D content and 3D content in a single piece of content.
  • the configuration of the 2D content and the 3D content can be made based on the user's command signal received from the user command input unit 120.
  • image content is configured entirely with only 2D content or only 3D content, or configured such that 3D content is reproduced during the reproduction of 2D content.
  • the image editing unit 140 edits the image content received from the image obtainment unit 110 based on content configuration information configured by the editing image configuration unit 130.
  • the image editing unit 140 stores content reproduction information (for example, image information and 3D content reproduction information), which is required to reproduce 3D content, in the information recording unit 150 such that the entirety or parts (segments) of the 2D content can be converted into 3D content on the user terminal 200.
  • content reproduction information for example, image information and 3D content reproduction information
  • the image editing unit 140 stores content reproduction information (for example, the type of image content, 2D/3D content section storage location, 2D/3D content segment frame location, segment length, and segment Identifications (IDs)), which is required to repeat the reproduction of the 2D content and the 3D content, in the information recording unit 150.
  • content reproduction information for example, the type of image content, 2D/3D content section storage location, 2D/3D content segment frame location, segment length, and segment Identifications (IDs)
  • the information recording unit 150 stores information related to the authoring, editing, and configuration of 3D content.
  • the metadata generation unit 160 generates content authoring information, received by the user command input unit 120, the editing image configuration unit 130 and the image editing unit 140, in the form of metadata.
  • the file format generation unit 170 configures the content reproduction information and metadata, which are generated when content is authored, in the form of a single integrated multimedia content file format.
  • the file transmission unit 180 provides the corresponding integrated multimedia content file format, generated by the file format generation unit 170, in response to a content request received from the user terminal 200.
  • information about the integrated multimedia content file format includes information about the compatibility of 3D video content with a 2D video reproducer, information causing 2D video segments to be mixed with 3D video segments, thereby enabling a single service to be provided, and script information in which animation effects are inserted into 3D video content or video content in which 2D video is mixed with 3D video.
  • FIG. 3 is a view showing an example of the structure of an integrated multimedia content file format according to the present invention.
  • FIG. 3 shows an example of the integrated multimedia content file format which includes one or more 2D video streams so that the entirety of or part of a video can be provided in the form of a single piece of 3D video content.
  • 2D content based on information within a file format, and time, location and composition information written in metadata or script information, the entirety of 2D content is converted into 3D content using two or more pieces of 2D content and then the resulting content is used, or one or more 2D segments can be represented by respective tracks such that part of content displayed in the form of 2D content is displayed in the form of 3D content.
  • the integrated multimedia content file format includes a file type box (ftyp box) containing brand-declared information indicative of a 3D video-included multimedia application format, a movie box containing content-related information, a meta box (meta box) containing information about configuration which does not require temporal synchronization within the file format, and information about content representation, and a media data box (mdat box) containing actual multimedia content.
  • the moov box includes at least one track box (trak box) containing a plurality of pieces of single-type multimedia content (one or more pieces of single video and single audio), multimedia content encoding information, pointing information, information about reproduction of multimedia content requiring temporal synchronization.
  • trak box a track box containing a plurality of pieces of single-type multimedia content (one or more pieces of single video and single audio), multimedia content encoding information, pointing information, information about reproduction of multimedia content requiring temporal synchronization.
  • the meta box includes information about the locations of 2D video and audio which constitute a file, and information about content itself, and includes an xml box (xml box) containing content author information, camera information, reproduction information and content information, which are written in metadata. Further, the meta box may be included in the trak box.
  • FIG. 4 is a view showing another example of the structure of an integrated multimedia content file format according to the present invention, and shows an example of an integrated multimedia content file format which includes one or more 2D video segments and one or more 3D video segments in the file format so that they can be provided in the form of a single piece of composite video content.
  • the integrated multimedia content file format includes a file type box (ftyp box) containing brand-declared information indicative of a 3D video- included multimedia application format, a movie box containing content-related information, a meta box (meta box) containing information about configuration which does not require temporal synchronization within the file format, and information about content representation, and a media data box (mdat box) containing actual multimedia content.
  • the moov box includes a plurality of trak boxes, and each of the trak boxes includes metadata containing information about each single piece of content.
  • the camera information and the reproduction information can be defined as the parameters of the trak box or can be described in the form of metadata in the meta box.
  • the combinatory representation of 2D and 3D video includes reproduction time indicative of the time that each of the video segments should be reproduced using temporal synchronization information, which uses decoding time information, chunk unit and chunk offset, and which is stored in a sample table box (stbl box) within the trak box.
  • temporal synchronization information which uses decoding time information, chunk unit and chunk offset, and which is stored in a sample table box (stbl box) within the trak box.
  • FIG. 5 is a view showing another example of the structure of an integrated multimedia content file format, including only 3D content, according to the present invention.
  • the integrated multimedia content file format includes a file type box (ftyp box) containing brand-declared information indicative of a 3D video- included multimedia application format, a movie box containing content-related information, an xml box (xml box), a meta box (meta box) containing information about configuration which does not require temporal synchronization within the file format, and information about content representation, and a media data box (mdat box) containing actual multimedia content.
  • a file type box containing brand-declared information indicative of a 3D video- included multimedia application format
  • xml box xml box
  • meta box meta box
  • mdat box media data box
  • FIG. 5 includes a single encoded video stream in which 2D video streams, taken from various angles, are encoded using a 3D video codec. If a reproduction terminal for receiving and reproducing content does not support 3D content reproduction, only 2D video streams, taken from one angle, are reproduced using reproduction and authoring information included in the file format and metadata.
  • FIG. 6 is a diagram showing the configuration of the user terminal according to the present invention in detail.
  • the user terminal 200 includes an image reception unit 210, a file format parsing unit 220, a metadata analysis unit 230, an image conversion determination unit 240, a terminal performance determination unit 250, and a rendering unit 260.
  • the image reception unit 210 requests and receives integrated multimedia content from the 3D content authoring server 100.
  • the file format parsing unit 220 parses a file format in order to determine the configuration information of the integrated multimedia content received by the image reception unit 210.
  • the metadata analysis unit 230 analyzes metadata in order to detect authoring information based on data included in the integrated multimedia content.
  • the image conversion determination unit 240 determines whether to execute 3D content without change or to select a single piece of 2D video content from among a plurality of pieces of 2D video content which constitute the 3D content and then reproduce it, depending on the performance of a terminal.
  • the image conversion determination unit 240 analyzes images and then converts them into 2D content based on the file format information and metadata of the 3D content of the corresponding integrated multimedia content.
  • the image conversion determination unit 240 selectively reproduces one of the images taken from two angles if two video streams separately exist, or performs decoding and then selectively reproduces one of the generated video images taken from two angles if a single piece of 3D video stream exists.
  • the terminal performance determination unit 250 checks whether the user terminal 200 supports 3D content.
  • the rendering unit 260 reproduces the image content, determined based on the performance of the terminal, as the integrated multimedia content on the user terminal using file format information, analyzed metadata information, script information, or a method in which the pieces of information are combined with each other.
  • FIG. 7 is a flowchart showing an example of a method of providing a 3D multimedia content service according to the present invention, and the case in which the 3D content authoring server of FIG. 2 authors integrated multimedia content will be described as an example.
  • the image obtainment unit 110 of the 3D content authoring server 100 receives 3D content or one or more pieces of 2D content which are encoded based on the selection of a user at step SlOl.
  • the editing image configuration unit 130 configures an application section for the 2D and 3D content segments in a single piece of content at step S 103.
  • the configuration of the 2D and 3D content can be made in response to a user command signal received from the user command input unit 120.
  • specific image content is configured entirely with only 2D content or only 3D content, or configured such that the reproduction of 3D content is repeated during the reproduction of 2D content.
  • the image editing unit 140 edits the image content received from the image obtainment unit 110 based on content configuration information configured by the editing image configuration unit 130 at step S 105.
  • the image editing unit 140 stores the content reproduction information (for example, image information and 3D content reproduction information), which is required to reproduce the 3D content, in the information recording unit 150 such that entirety or parts (segments) of the 2D content can be converted into 3D content on the user terminal 200.
  • content reproduction information for example, image information and 3D content reproduction information
  • the image editing unit 140 stores content reproduction information (for example, the type of image content, 2D/3D content section storage location, 2D/3D content segment frame number, segment length, and segment IDs), which is required to repeat the reproduction of the 2D content and the 3D content, in the information recording unit 150.
  • content reproduction information for example, the type of image content, 2D/3D content section storage location, 2D/3D content segment frame number, segment length, and segment IDs
  • the metadata generation unit 160 generates content authoring information, received by the user command input unit 120, the editing image configuration unit 130 and the image editing unit 140, in the form of metadata at step S 107.
  • the file format generation unit 170 configures content configuration in- formation, content reproduction information and metadata, which are generated when content is authored, in the form of a single integrated multimedia content file format at step S 109.
  • the file transmission unit 180 when the file transmission unit 180 receives an integrated multimedia content request received from the user terminal 200, the file transmission unit 180 provides relevant integrated multimedia content to the user ter minal 200.
  • FIG. 8 is a flowchart showing another example of a method of providing a 3D multimedia content service according to the present invention, and the case in which the user terminal 200 of FIG. 6 reproduces integrated multimedia content will be described as an example.
  • the image reception unit 210 of the user terminal 200 requests integrated multimedia content from the 3D content authoring server 100 and receives the integrated multimedia content at step S201.
  • the file format parsing unit 220 checks information about the 2D content and 3D content of the integrated multimedia content received by the image reception unit 210, and then parses a file format at step S203.
  • the metadata analysis unit 230 analyzes the metadata of the integrated multimedia content at step S205.
  • the image conversion determination unit 240 reproduces the integrated multimedia content using the rendering unit 260 based on information recorded in the file format which is detected by the file format parsing unit 220 and based on the metadata analyzed by the metadata analysis unit 230 at step S207.
  • the image conversion determination unit 240 analyzes the images based on the file format information, and then converts them into 2D content.
  • the image conversion determination unit 240 selectively reproduces one of the images taken from two angles if two video streams separately exist, and performs decoding and then selectively reproduces one of the generated video images taken from two angles if a single piece of 3D video stream exists.
  • the terminal performance determination unit 250 of the user terminal 200 checks whether the user terminal 200 supports 3D content. If, as the result of the check, the user terminal 200 does not support 3D content, the terminal performance determination unit 250 of the user terminal 200 notifies the image conversion determination unit 240 of the fact that the user terminal 200 does not support 3D content so that the corresponding integrated multimedia content is converted into 2D content and then reproduced. [120] As described above, those skilled in the art will appreciate that the present invention can be implemented in other detailed forms without changing the technical spirit or essential features of the present invention. Therefore, the above-described embodiments should be interpreted as being disclosed for illustrative rather than restrictive purposes. The scope of the present invention should be interpreted as disclosed in the accompanying claims rather than the embodiments, and all the modified or modified configurations deduced from the meaning, scope, and equivalent concepts of the claims should be interpreted such that they are included in the scope of the present invention.
  • the system and method for providing a 3D multimedia content service using an integrated multimedia file structure generates an integrated multimedia file structure, including information required to reproduce 3D content or to modify the 3D content into 2D content, the present invention is appropriate to provide various reproduction methods using information required to author, modify, or reproduce the 3D content.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Business, Economics & Management (AREA)
  • Computer Security & Cryptography (AREA)
  • Tourism & Hospitality (AREA)
  • General Health & Medical Sciences (AREA)
  • Physics & Mathematics (AREA)
  • Health & Medical Sciences (AREA)
  • Economics (AREA)
  • Databases & Information Systems (AREA)
  • Human Resources & Organizations (AREA)
  • Marketing (AREA)
  • Primary Health Care (AREA)
  • Strategic Management (AREA)
  • Computer Graphics (AREA)
  • General Business, Economics & Management (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Library & Information Science (AREA)
  • Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
  • Information Transfer Between Computers (AREA)

Abstract

Système de fourniture d'un service de contenu multimédia en trois dimensions (3D) reposant sur l'emploi d'une structure de fichier multimédia intégrée. Un serveur-auteur de contenus 3D crée un format de fichier pour contenu multimédia intégré qui contient l'un quelconque des groupes d'informations suivants: informations pour reproduction d'un contenu 3D; informations de conversion d'images utilisées pour sélection d'une vue unique parmi une pluralité d'images pour contenu 3D; informations pour combinaison des segments de contenu 3D et des segments de contenu 2D en un seul fichier au moyen d'informations de synchronisation temporelle d'un format fichier; informations pour combinaison d'un contenu 3D et d'un contenu 2D au moyen de données de script de sorte que le contenu ainsi obtenu est reproduit dans le temps; informations utilisées pour combinaison avec des données de script; et combinaisons de ces diverses groupes d'informations. Un terminal utilisateur demande au serveur auteur de contenus 3D un contenu multimédia intégré et le reçoit dudit serveur, détecte l'information de création du contenu multimédia intégré, puis exécute le contenu multimédia intégré.
PCT/KR2008/005791 2007-10-08 2008-10-01 Système et procédé pour service de contenus multimédia 3d au moyen d'un format de fichier pour application multimédia WO2009048235A2 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
KR1020070100756A KR101257386B1 (ko) 2007-10-08 2007-10-08 통합 멀티미디어 파일 구조를 이용한 3ｄ 멀티미디어콘텐츠 서비스 시스템 및 방법
KR10-2007-0100756 2007-10-08

Publications (2)

Publication Number Publication Date
WO2009048235A2 true WO2009048235A2 (fr) 2009-04-16
WO2009048235A3 WO2009048235A3 (fr) 2009-05-28

Family

ID=40549725

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/KR2008/005791 WO2009048235A2 (fr) 2007-10-08 2008-10-01 Système et procédé pour service de contenus multimédia 3d au moyen d'un format de fichier pour application multimédia

Country Status (2)

Country Link
KR (1) KR101257386B1 (fr)
WO (1) WO2009048235A2 (fr)

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2011005056A3 (fr) * 2009-07-09 2011-04-21 엘지전자 주식회사 Procédé de présentation d'image pour un dispositif d'affichage qui présente des contenus tridimensionnels et dispositif d'affichage faisant appel audit procédé
WO2011109814A1 (fr) * 2010-03-05 2011-09-09 General Instrument Corporation Procédé et appareil servant à convertir un contenu vidéo bidimensionnel pour insertion dans un contenu vidéo tridimensionnel
US20120288208A1 (en) * 2010-01-08 2012-11-15 Sony Corporation Image processing device, information recording medium, image processing method, and program
WO2013155708A1 (fr) * 2012-04-20 2013-10-24 Nokia Corporation Système assurant la fonction de zoomage sélectif et intelligent dans un flux media issu d'une collaboration
US11606576B2 (en) 2018-10-08 2023-03-14 Samsung Electronics Co., Ltd. Method and apparatus for generating media file comprising 3-dimensional video content, and method and apparatus for replaying 3-dimensional video content

Families Citing this family (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR20140104121A (ko) * 2013-02-20 2014-08-28 주식회사 카몬 복합 차원 콘텐츠 제공 시스템, 그 제공 방법 및 복합 차원 콘텐츠 파일
KR101430985B1 (ko) * 2013-02-20 2014-09-18 주식회사 카몬 2d-3d 복합 차원 콘텐츠 파일을 사용하는 복합 차원 콘텐츠 서비스 제공 시스템, 그 서비스 제공 방법
WO2019194571A1 (fr) * 2018-04-05 2019-10-10 Lg Electronics Inc. Procédé pour transmettre une vidéo à 360 degrés, procédé pour recevoir une vidéo à 360 degrés, appareil pour transmettre une vidéo à 360 degrés et appareil pour recevoir une vidéo à 360 degrés
CN113360682B (zh) * 2021-05-21 2023-03-21 青岛海尔空调器有限总公司 信息处理方法和装置

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20030103074A1 (en) * 2001-12-04 2003-06-05 Koninklijke Philips Electronics N.V. Methods for multimedia content repurposing
KR20060100347A (ko) * 2006-08-31 2006-09-20 한국정보통신대학교 산학협력단 멀티미디어 방송 콘텐츠용 통합 파일 포맷 구조와 이를이용한 멀티미디어 방송 콘텐츠 서비스 시스템 및 그 방법
US20060209061A1 (en) * 2005-03-18 2006-09-21 Microsoft Corporation Generating 2D transitions using a 3D model
KR20070025221A (ko) * 2005-09-01 2007-03-08 주식회사 피앤조이 3차원 영상을 2차원 영상으로 변환하여 표시하는 방법

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR100478934B1 (ko) * 2002-10-22 2005-03-28 한국전자통신연구원 객체기반 엠팩-4 컨텐츠 편집/저작과 검색 장치 및 방법
JP2005151128A (ja) 2003-11-14 2005-06-09 Canon Inc データ処理方法および装置
KR20050092688A (ko) * 2005-08-31 2005-09-22 한국정보통신대학교 산학협력단 통합 멀티미디어 파일 포맷 구조와 이를 기반으로 하는멀티미디어 서비스 제공 시스템 및 그 방법

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20030103074A1 (en) * 2001-12-04 2003-06-05 Koninklijke Philips Electronics N.V. Methods for multimedia content repurposing
US20060209061A1 (en) * 2005-03-18 2006-09-21 Microsoft Corporation Generating 2D transitions using a 3D model
KR20070025221A (ko) * 2005-09-01 2007-03-08 주식회사 피앤조이 3차원 영상을 2차원 영상으로 변환하여 표시하는 방법
KR20060100347A (ko) * 2006-08-31 2006-09-20 한국정보통신대학교 산학협력단 멀티미디어 방송 콘텐츠용 통합 파일 포맷 구조와 이를이용한 멀티미디어 방송 콘텐츠 서비스 시스템 및 그 방법

Cited By (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2011005056A3 (fr) * 2009-07-09 2011-04-21 엘지전자 주식회사 Procédé de présentation d'image pour un dispositif d'affichage qui présente des contenus tridimensionnels et dispositif d'affichage faisant appel audit procédé
US20120288208A1 (en) * 2010-01-08 2012-11-15 Sony Corporation Image processing device, information recording medium, image processing method, and program
WO2011109814A1 (fr) * 2010-03-05 2011-09-09 General Instrument Corporation Procédé et appareil servant à convertir un contenu vidéo bidimensionnel pour insertion dans un contenu vidéo tridimensionnel
CN102783160A (zh) * 2010-03-05 2012-11-14 通用仪表公司 将用于插入的二维视频内容转换为三维视频内容的方法和设备
KR20140031388A (ko) * 2010-03-05 2014-03-12 제너럴 인스트루먼트 코포레이션 3-차원 비디오 콘텐츠로의 삽입을 위해 2-차원 비디오 콘텐츠를 변환하는 방법 및 장치
US9154814B2 (en) 2010-03-05 2015-10-06 Google Technology Holdings LLC Method and apparatus for converting two-dimensional video content for insertion into three-dimensional video content
US20160029003A1 (en) * 2010-03-05 2016-01-28 Google Technology Holdings LLC Method and apparatus for converting two-dimensional video content for insertion into three-dimensional video content
KR101670560B1 (ko) 2010-03-05 2016-10-28 제너럴 인스트루먼트 코포레이션 3-차원 비디오 콘텐츠로의 삽입을 위해 2-차원 비디오 콘텐츠를 변환하는 방법 및 장치
WO2013155708A1 (fr) * 2012-04-20 2013-10-24 Nokia Corporation Système assurant la fonction de zoomage sélectif et intelligent dans un flux media issu d'une collaboration
US11606576B2 (en) 2018-10-08 2023-03-14 Samsung Electronics Co., Ltd. Method and apparatus for generating media file comprising 3-dimensional video content, and method and apparatus for replaying 3-dimensional video content

Also Published As

Publication number Publication date
KR20090035794A (ko) 2009-04-13
KR101257386B1 (ko) 2013-04-23
WO2009048235A3 (fr) 2009-05-28

Similar Documents

Publication Publication Date Title
KR101777348B1 (ko) 데이터 전송 방법 및 장치와 데이터 수신 방법 및 장치
WO2009048235A2 (fr) Système et procédé pour service de contenus multimédia 3d au moyen d'un format de fichier pour application multimédia
KR101750049B1 (ko) 적응적인 스트리밍 방법 및 장치
KR101786050B1 (ko) 데이터 전송 방법 및 장치
US9596430B2 (en) Data generation apparatus, data generating method, data reproduction apparatus, and data reproducing method
US9788020B2 (en) File generation apparatus, file generating method, file reproduction apparatus, and file reproducing method
Avaro et al. MPEG-4 systems: overview
US20130185760A1 (en) Streaming distributing device and method, streaming receiving device and method, streaming system, program, and recording medium
US20080252719A1 (en) Apparatus, method, and system for generating stereo-scopic image file based on media standards
US9918099B2 (en) File generation apparatus, file generating method, file reproduction apparatus, and file reproducing method
WO2009031872A2 (fr) Système et procédé de génération et de reproduction d'un fichier d'images stéréoscopiques 3d comprenant une image 2d
KR20110105710A (ko) 복수의 챕터를 포함하는 콘텐트를 적응적으로 스트리밍하는 방법 및 장치
TW201909007A (zh) 使用用於檔案格式邏輯框之一通用描述符處理媒體資料
RU2767300C2 (ru) Высокоуровневая передача служебных сигналов для видеоданных типа "рыбий глаз"
KR20240032178A (ko) 송신 장치, 송신 방법, 수신 장치 및 수신 방법
KR100837721B1 (ko) 개인화된 디지털 방송 콘텐츠의 소비를 위한 통합 형식인코딩 장치와, 이의 디코딩 장치 및 인코딩 방법
US7778526B2 (en) System and method for maintaining DVD-subpicture streams upon conversion to higher compressed data format
US11722751B2 (en) Method, an apparatus and a computer program product for video encoding and video decoding
KR101324427B1 (ko) 장면 기술자를 이용하여 ｍｐｅｇ-2 전송스트림을 포함하는 콘텐츠 저작/재생 장치 및 방법
KR20110117568A (ko) 멀티미디어 전송 시스템에서 서비스 발견 정보를 송수신하는 방법 및 장치와 이를 위한 파일 구조
TW201909647A (zh) 增強區域取向包封及視埠獨立高效視頻寫碼媒體資料檔
KR20100109333A (ko) 디지털 데이터 인터페이스를 통한 압축 데이터 전송 방법 및 장치, 수신 방법 및 장치
Cardoso et al. Hyperlinked video with moving objects in digital television
Chan et al. Data hiding in MP4 video container based on subtitle track
WO2009045051A2 (fr) Procédé de présentation d'un comportement initial d'un contenu de format d'application multimédia et système associé

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 08837556

Country of ref document: EP

Kind code of ref document: A2

NENP Non-entry into the national phase

Ref country code: DE

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 23/06/2010)

122 Ep: pct application non-entry in european phase

Ref document number: 08837556

Country of ref document: EP

Kind code of ref document: A2