JP2011511593A - Apparatus and method for generating and displaying media files - Google Patents

Abstract

A computer-readable recording medium having stored data is provided. The data structure includes a media data box including two or more media data and a movie data ('moov') box including information on view sequence data of the media data, and the 'moov' box includes one Track reference information indicating that a track box for a view sequence refers to a track box of another view sequence is included.
[Selection] Figure 2

Description

  The present invention relates to an apparatus and method for generating and displaying stereoscopic media files.

  MPEG (Moving Picture Experts Group), an international standardization organization related to multimedia, has started standardization work for MPEG-2, MPEG-4, MPEG-7, and MPEG-21 as well as its first standard, MPEG-1. We are promoting. The development of such various standards has led to the need to combine different standard technologies to generate a single profile. As one such movement, various multimedia application formats (MAF) are MPEG-A ( MPEG Multimedia Application Format: ISO / IEC (International Organization for Standardization / International Electrotechnical Commission) 23000 (multi-media application standardization). MAF aims to increase the utility value of a standard by combining not only the existing MPEG standard but also a non-MPEG standard. In this way, the utility value can be maximized by creating a MAF by combining standard techniques that have already been verified without an effort to create a new standard.

  Recently, in order to represent more realistic video information, intensive research has been conducted in connection with methods for realizing three-dimensional (3D) video. Among the methods, using human visual characteristics, a method considered to be effective in various aspects is that the left view and the right view are imaged on the left eye and right eye of the user, respectively. Thus, the left view image and the right view image are scanned on the existing display device at the corresponding position so that the user can feel the 3D effect. As an example, a mobile terminal equipped with a barrier LCD (Barrier Liquid Crystal Display) can reproduce stereoscopic content and provide a realistic video to the user.

  However, in the case of stereoscopic content composed of two or more view sequences, the syntax is not defined on the file format. Based on this syntax, it can be determined whether or not the tracks of the stereoscopic content view sequence are related to each other. A view sequence is a video bit stream made up of one or more video frames, and can also be called an elementary stream. It also includes stereoscopic video and monoscopic video, where the stereoscopic video is spatially combined with two-dimensional (2D) video, or content that appears together in one scene. For example, in the case of a service in which there is a video subtitle at the bottom of the monoscopic music video and the video subtitle is displayed as a 2D image, the syntax for determining whether or not the music video and the video subtitle are mutually related is a file It is not defined on the format. Therefore, it is necessary to additionally provide information indicating whether or not the music video and the video subtitle are related to each other.

  The present invention has been made in view of the above-described conventional problems, and an object of the present invention is to provide a stereoscopic content composed of two or more view sequences or a stereoscopic display that is simultaneously displayed in one scene. It is an object of the present invention to provide a media file generation and display apparatus and method for clearly determining whether or not a track of a view sequence is related to content having video and monoscopic video.

  In order to achieve the above object, according to one aspect of the present invention, a computer-readable recording medium having stored data is provided. The data structure includes a media data box including two or more media data items, and a movie data ('moov') box including information on view sequence data of the media data, and the 'moov' box Includes track reference information indicating that a track box for one view sequence refers to a track box of another view sequence.

  According to another aspect of the invention, a computer-implemented method is provided. The method includes receiving a media file, a media data box of the received media file including two or more view sequence data, and a movie data ('moov') box including information about the view sequence data. A view sequence to be referred to and a view sequence to be referred to by track reference information indicating that a track box for one view sequence refers to a track box for another view sequence included in the 'moov' box. Generating a video based on.

  According to still another aspect of the present invention, a media data box of a media file including two or more view sequence data and a movie data ('moov') box including information on the view sequence data are analyzed, and the ' A file for extracting video based on a reference view sequence and a reference view sequence by track reference information included in a moov 'box and indicating that a track box for one view sequence refers to a track box of another view sequence A terminal device is provided that includes an analysis unit and a display unit that displays the extracted video.

  According to the present invention, a stereoscopic content composed of two or more view sequences, or a track included in a content having a stereoscopic video and a monoscopic video that are simultaneously displayed in one scene are mutually connected. It is possible to clearly determine the track related to the, and to avoid redundancy for additional metadata.

  The above and other aspects, features and advantages of the present invention will become more apparent from the following detailed description taken in conjunction with the accompanying drawings in which:

FIG. 3 is a diagram illustrating an ISO-based media file format. It is a figure which shows the file structure by the 1st Embodiment of this invention. FIG. 3 shows a file structure designed to interconnect related tracks according to a first embodiment of the present invention. FIG. 3 shows a file structure designed to interconnect related tracks according to a first embodiment of the present invention. It is a figure which shows operation | movement of the terminal by the 1st Embodiment of this invention. It is a figure which shows the file structure by the 2nd Embodiment of this invention. FIG. 6 is a diagram illustrating a method of expressing a first view sequence according to a second embodiment of the present invention. It is a figure which shows operation | movement of the terminal by the 2nd Embodiment of this invention. It is a figure which shows the file structure by the 3rd Embodiment of this invention. FIG. 7 is a diagram illustrating a method for expressing a first view sequence according to a third embodiment of the present invention. It is a figure which shows the file structure by the 4th Embodiment of this invention. It is a figure which shows the file structure of the stereoscopic video by the 5th Embodiment of this invention. It is a figure which shows the file structure of the stereoscopic video by the 5th Embodiment of this invention. It is a figure which shows the file structure of the multi view content by the 5th Embodiment of this invention. It is a figure which shows the file structure of the multi view content by the 5th Embodiment of this invention. 1 is a diagram illustrating a media file generation apparatus according to an embodiment of the present invention. 1 is a diagram illustrating a media file playback apparatus according to an embodiment of the present invention.

  Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings.

  In the following description, specific descriptions of well-known functions or configurations related to the present invention are omitted for clarity and brevity. The terminology used herein is defined in consideration of the function of the present invention, and can be changed according to the intention or practice of the user or operator. Therefore, the above terms must be defined based on the entire contents of this specification.

  In the present invention, the ISO-based media file format will be described first. In addition, the present invention provides a method for indicating a relationship between pairs of tracks in a stereoscopic content composed of two or more view sequences, and a stereoscopic video and a monoscopic video displayed simultaneously in one scene. A method is provided for showing the relationship between a stereoscopic video track and a monoscopic video track with content having.

  FIG. 1 shows an ISO-based media file format.

  Referring to FIG. 1, an ISO-based media file 100 includes a file type box (“ftyp” box; not shown), a movie data box (“moov” box) 110, and a media data box (“mdat” box) 120. including. This file type box contains details of the file type and compatible type. Normal reproduction is possible with the corresponding decoder according to the compatible type. The 'moov' box 110 corresponds to a file format header box, and each data is formed in a structure based on an object called 'atom'. The 'moov' box 110 is a synchronization used to support content information such as frame rate, bit rate, image size and playback functions such as FF (Fast-Forward) / REW (Rewind). Contains all the information necessary for a playback file containing information. The media data box 120, which is a data box, includes actual media data, and video data and audio data are stored in units of frames in each track.

  The stereoscopic video includes stereoscopic video related information. The stereoscopic video related information is necessary information such as the configuration type of the stereoscopic video, and may be additional data such as camera parameters and display information. If the stereoscopic video is composed of two or more view sequences, each view sequence may have the same stereoscopic video related information. For example, for a stereoscopic video composed of two view sequences, each of the left and right views of the video can contain additional information about the same camera and display. Thus, when each view sequence has related information of the same stereoscopic video, in order to prevent the same information from being redundantly stored in each view sequence, the corresponding information is only one view sequence. The remaining view sequence can use the corresponding stereoscopic video related information included in the corresponding view sequence with reference to the corresponding view sequence. However, in order to do so, it is necessary to notify other elementary streams which elementary stream contains stereoscopic video related information and to determine a view sequence containing stereoscopic video related information. In the case of a stereoscopic video composed of two or more view sequences, the two view sequences are divided into a first (primary) view sequence and a second (secondary) view sequence. As described above, when the stereoscopic video-related information is included in only one basic stream, the corresponding information can be confirmed by distinguishing the first view sequence from the second view sequence. The first view sequence and the second view sequence described in the present invention display a high priority display when only one of two or more view sequences must be selected and displayed on the screen. Distinguish between view sequences.

  The method for distinguishing the first view sequence from the second view sequence includes a first method for confirming the track ID (track_ID) of each view sequence. The track header 'tkhd' box of each view sequence has a track ID (track_ID) which is an identifier that can identify each track. Since the track ID is an integer value sequentially assigned to the tracks of each view sequence, the view sequence of the track having the minimum value of the track ID is determined as the first view sequence.

  The second method indicates whether one of the left view sequence and the right view sequence (or two or more view sequences) of the configuration type information of the stereoscopic video is first encoded. Check the is_left_first 'parameter, and determine whether the left view sequence or the right view sequence (or two or more view sequences) is the first view sequence or the second view sequence according to the value of the corresponding parameter. . In the third method, a track that refers to another track is determined as the first view sequence or the second view sequence.

  Determining the first view sequence based on the information about the track criteria is when the referenced track is determined as the first view sequence (reference is performed by another track) (other The reference track is referred to as the second view sequence. Since tracks that reference other tracks have a track reference box ('tref' box), the stereo video of the other side or the remaining view is determined as the first view sequence in the above example. Is done. In this example, the location of the 'tref' box with information about track references can be a way to distinguish the first view sequence from the second view sequence. With the use of track references, it is possible to connect mutually related view sequences within a media file composed of two or more video tracks, thereby determining which tracks are related to each other. It becomes possible. It can also be used as a way of interconnecting to create a single video from multiview video. By using the track reference method, several tracks are inserted by inserting overlapping stereoscopic video related information into a specific track, ie only one of the first view sequence and the second view sequence. It is possible to prevent the stereoscopic video related information from being inserted in duplicate.

  According to another method for distinguishing the first view sequence from the second view sequence, the first view sequence and the second view sequence are not only one information or one parameter as in the above method. 2 such as the stereoscopic video information required to represent the structure of the stereoscopic video including the track ID and the 'is_left_first' parameter, the parameter for identifying the 'tref' box information, and the handler type. It is determined by combining the above stereoscopic video related information, fields, parameters, and boxes. The following method can determine the first view sequence and the second view sequence by combining two or more stereoscopic video related information, fields, parameters, and boxes. First, in the case of a stereoscopic video composed of two view sequences of a left view and a right view, the value of the 'is_left_first' field and the information of the 'tref' box that refers to the stereoscopic video of another track are used. The corresponding track can be determined as the first view sequence or the second view sequence according to a criterion for distinguishing the first view sequence from the second view sequence. Alternatively, the first view sequence may be based on a criterion for distinguishing the first view sequence from the second view sequence using information in a 'tref' box that refers to the stereoscopic video of another track and the track ID. Alternatively, the corresponding track can be determined as the second view sequence.

  Furthermore, for a stereoscopic video composed of two or more view sequences (ie, multiple or multi-view sequences) by a combination of two or more stereoscopic video related information, fields, parameters, and boxes. There are other ways of determining the first view sequence and the second view sequence. According to this other method, the first view sequence and the second view sequence are determined using the value of the 'is_left_first' field, the ID of the track, and the 'tref' box that refers to the stereoscopic video track. Can do.

  Parameters or information other than those mentioned above are used as parameters or information for determining the first view sequence and the second view sequence as described above, and two or more stereoscopic video related information fields , Parameters and boxes can be expanded or added in various ways.

  Hereinafter, a method for indicating a relationship between tracks of a view sequence paired with a stereoscopic content composed of two or more view sequences according to the present invention will be described. The following description further includes a method of showing a relationship between a stereoscopic view sequence and a monoscopic view sequence in content having stereoscopic video and monoscopic video according to the present invention.

<First Embodiment>
In order to decode a stereoscopic content composed of two or more view sequences and display it on the screen, it is necessary to indicate that the tracks of the left view sequence and the tracks of the right view sequence are related to each other. However, since the box and information indicating the relationship between tracks do not currently exist in the stereoscopic file format, the first embodiment of the present invention provides the following method to solve this problem.

  Among the boxes defined in the 'ISO / IEC 14496-12 ISO-based media file format' document, there are a handler reference box ('hdlr' box) and a track reference box ('tref' box). The handler reference box ('hdlr' box) indicates the type of media data in the current track using the handler type ('handler_type'), and is defined as shown in Table 1.

  In order to connect two related tracks to a stereoscopic content composed of two or more video tracks, in the first embodiment of the present invention, as shown in Table 2, a handler is used. A 'svid' value indicating that the media data type of the corresponding track is a stereoscopic video is added to the handler type ('handler_type') of the reference box ('hdlr' box).

  The track reference box ('tref' box) is used to further connect other tracks referenced by the current track using the reference type ('reference_type') and the track ID (track_ID). Table 3 shows the 'reference_type' currently defined in the 'ISO / IEC 14496-12 ISO-based media file format' document.

  In the first embodiment of the present invention, in order to connect two related tracks, ‘avmi’ is added to ‘reference_type’ in the track reference box (‘tref’ box) as shown in Table 4.

  FIG. 2 illustrates a view sequence associated with a stereoscopic content composed of two view sequences using newly defined 'handler_type' and 'reference_type' according to the first embodiment of the present invention. A file structure for connecting tracks to each other is shown.

  Referring to FIG. 2, the track of the stereoscopic left view sequence includes a “tref” box, and the track of the stereoscopic right view sequence to be referred to by the corresponding track is used using a track reference box (“tref” box) 210. Connect. Here, by setting reference_type = 'avmi' in the track reference box ('tref' box), the corresponding reference track is a track including stereoscopic video related information, and is referred to as a track to be referred to, that is, referred to. Note that it is associated with a referenced track. The stereoscopic video related information included in the corresponding reference track is stereoscopic video information that must basically be included in each track of the view sequence constituting the stereoscopic content. It can be stored in only one of the tracks. If this track has a relationship with a referenced track, it means that the two tracks make a pair, which means that there is a dependency between the two tracks. In other words, when the view sequence of the referenced track is the first view sequence, the view sequence of the referenced track is the second view sequence, so that the referenced track is dependent on the referenced track. Have Further, by setting handler_type = 'svid' in the handler reference box ('hdlr' box) 220 of the referenced track, it can be seen that the referenced track is a stereoscopic video track.

  Since the first view sequence and the second view sequence can be determined based on the presence or absence of the track reference box ('tref' box), the track having the track reference box ('tref' box) is determined as the second view sequence. In FIG. 2, the stereoscopic left sequence track is the second view sequence track. The left view sequence may be the first view sequence by the first view sequence determination method. When the first view sequence and the second view sequence are determined using the track reference box ('tref' box) 210, the track of the stereoscopic right view sequence is determined as the first view sequence. The stereoscopic right-view sequence track is set to be referred to by the stereoscopic left-view sequence track. In this case, the reference track having the track reference box ('tref' box) is considered to be set as the second view sequence.

  FIG. 3 shows a file structure for interconnecting related tracks for multi-view content having multiple view sequences according to the first embodiment of the present invention.

  Referring to FIG. 3, assuming that a track of the first (or main) view sequence has a track reference box ('tref' box), the first view is shown using a track reference box ('tref' box) 310. A sequence track and a plurality of tracks related to the track can be connected. In this case, in the track including the first view sequence, the reference type of the track reference box ('tref' box) 310 is set to reference_type = 'avmi', and the handler reference box ('hdlr') of the track referred to by this track is set. The handler type ('handler_type') of 'box' 320, 330 is set to handler_type = 'svid'.

  As described above, the first view sequence can be distinguished from the second view sequence using information in the track reference box ('tref' box). FIG. 4 is related to the case where the sequence of tracks that do not have a track reference box ('tref' box) according to the first embodiment of the present invention, ie, the referenced track, is the first view sequence. This shows the file structure that connects tracks together.

  FIG. 5 illustrates an operation of a terminal for identifying a related video track and displaying it on a screen when a stereoscopic video is composed of two or more view sequences according to the first embodiment of the present invention. Show.

  Referring to FIG. 5, the terminal analyzes a file type box (“ftyp” box) of the media file in step 401. In steps 402 and 403, the terminal analyzes the 'moov' box and the track box ('trak' box) of the media file. In step 404, the terminal determines whether a track reference box ('tref' box) exists in the track box. If the track has a track reference box ('tref' box), the terminal checks a reference type ('reference_type') of the track reference box ('tref' box) in step 405. When it is determined that the reference type ('reference_type') is 'avmi', the terminal checks the track ID ('track_ID') referred to by the track reference box ('tref' box) and determines which stereoscopic It is determined whether a track in the view sequence is paired with the corresponding track. The terminal checks the media information box ('mdia' box) in step 406, and in step 407, based on whether the terminal can determine the media data type of the corresponding track, the handler type (handler box ('hdlr' box)) Check 'handler_type'). In step 408, the terminal checks the information in the remaining boxes including the stereoscopic information, analyzes the stereoscopic video related information of the tracks of the stereoscopic view sequence, and displays the related view sequence on the screen. A series of processes for analyzing a track box ('trak' box) is performed in the same way from the first track to the last track of a media file when the corresponding track is a track of a stereoscopic view sequence.

  On the other hand, if it is determined in step 404 that the track does not have a track reference box ('tref' box), the terminal proceeds to step 406, and the media information box ('mdia' box) of the corresponding track. Confirm. Thereafter, in step 407, the terminal checks the handler type ('handler_type'). In step 408, the terminal checks the remaining boxes including the stereoscopic information and displays the stereoscopic content on the screen.

  The terminal identifies the first view sequence and the second view sequence in step 408 of FIG. 5, and the order of the steps of identifying the first view sequence and the second view sequence is the first view sequence. The sequence can be modified by the method described above to distinguish it from the second view sequence.

  For example, if the track reference box ('tref' box) is used to identify the first view sequence and the second view sequence, the terminal performs the track reference box ('tref' box) in step 405 of FIG. The first view sequence and the second view sequence are identified by confirming the reference type ('reference_type') and the track ID ('track_ID'). If it is determined that the track of the view sequence having the track reference box ('tref' box) is the second view sequence, the reference type ('reference_type') of the track reference box ('tref' box) is 'avmi'. The track ID ('track_ID') to be referred to is the track ID ('track_ID') of the first view sequence. For example, a track with track ID = 1 (track_ID = 1) has a track reference box ('tref' box), and the reference type of the corresponding track reference box is 'avmi' (reference_type = 'avmi'), see When the track ID to be tracked is 2 (track_ID = 2), the track with track ID = 1 (track_ID = 1) is paired with the track with track ID = 2 (track_ID = 2). The view sequence of the track with track ID = 2 (track_ID = 2) is the first view sequence.

  As another method for distinguishing the first view sequence from the second view sequence according to an embodiment of the present invention, the terminal may include a left view sequence and a right view sequence (or 2) of the configuration type information of the stereoscopic video. The 'is_left_first' field indicating whether one of the at least one view sequence is to be encoded first is checked, and one of the left view sequence and the right view sequence (or two or more view sequences) is checked. When determining whether one of the first view sequence or the second view sequence is based on the value of the corresponding field, the UE performs a stereoscopic related operation including an 'is_left_first' parameter in step 408 according to the operation of FIG. By checking the information box Identify the primary view sequence and the secondary view sequence, and displays the related view sequences.

  In this way, the operational order of the process of identifying the first view sequence and the second view sequence can be changed according to each method for distinguishing the first view sequence from the second view sequence according to the present invention. .

  In one embodiment of the present invention, the handler type of the referenced track, ie, the remaining track that does not have a 'tref' box, is displayed as a stereoscopic video type ('svid'), but the referenced track Becomes a video type ('video'), and the track to be referenced can be a stereoscopic video type ('svid'). In addition, the track type to be referred to and the handler type ('handler_type') of the referenced track can be displayed as a video type ('vide') without distinction.

  On the other hand, the process of checking the track of the media file described in FIG. 5 and displaying it on the screen may not be sequentially provided by the terminal or the system. Here, the file format analysis process and the operation of the corresponding terminal not specifically described are in accordance with ISO / IEC 14496-12 and ISO / IEC 23000-11.

<Second Embodiment>
In the second embodiment of the present invention, a track reference box ('tref' box) is used in stereoscopic content to refer to a track including camera parameters and display safety information as additional information. The track reference method will be described. Camera parameters included as additional information in the stereoscopic content include baseline, focal_length, convergence_distance, translation, rotation, etc. Display safety information includes display size related information, viewing distance, and disparity ) Contains information. Here, the camera parameters and display safety information are shown as additional information, but these parameters and safety information are optional. Therefore, the box containing the relevant information can be expressed as a selective box.

  In the second embodiment of the present invention, a 'tref' box as shown in Table 5 for track reference of a track including camera parameters and display safety information used to acquire stereoscopic content. Add 'cdsi' to the reference_type.

  FIG. 6 illustrates a method for referring to a track including camera parameters and display safety information, which are additional information for stereoscopic content, according to the second embodiment of the present invention.

  Referring to FIG. 6, the tracks of the stereoscopic left view sequence and the tracks of the stereoscopic right view sequence use the track reference boxes ('tref' boxes) 510 and 520 to refer to the tracks including additional information. it can. In such a case, it is not necessary to store additional information in all the tracks, and other tracks refer to a track including additional information, thereby preventing the same information from being stored repeatedly in multiple tracks. can do.

  FIG. 7 illustrates a method for applying the second embodiment of the present invention to multi-view content having multiple view sequences.

  Even in this case, as shown in FIG. 6, the track including each view sequence refers to the track including the additional information using the ‘tref’ boxes 610, 620, and 630.

  FIG. 8 is a flowchart illustrating an operation of the terminal according to the second embodiment of the present invention.

  Referring to FIG. 8, the terminal analyzes a file type box (“ftyp” box) of the media file in step 701. In step 702 and 703, the terminal analyzes the 'moov' box and the track box ('trak' box) of the media file. In step 704, the terminal determines whether a track reference box ('tref' box) exists in the track box. If the track has a track reference box ('tref' box), the terminal confirms the reference type ('reference_type') of the track reference box ('tref' box) in step 705. When it is determined that the reference type ('reference_type') is 'cdsi', the terminal checks the track ID ('track_ID') referred to by the track reference box ('tref' box), and the corresponding track is referred to. It is determined which track contains additional information including camera parameters and display safety information, which are additional information regarding the stereoscopic video to be attempted. The terminal checks the media information box ('mdia' box) in step 706, and in step 707, based on which media data type of the corresponding track the terminal can determine, the handler type of the handler box ('hdlr' box) Confirm ('handler_type'). Finally, in step 708, the terminal checks the information in the remaining boxes containing the stereoscopic information, analyzes the stereoscopic video related information of the tracks in the stereoscopic view sequence, and displays the related tracks on the screen. Display. A series of processes for analyzing a track box ('trak' box) is performed in the same way from the first track to the last track of a media file when the corresponding track is a track of a stereoscopic view sequence.

  On the other hand, if it is determined in step 704 that the track does not have a track reference box ('tref' box), the terminal proceeds to step 706, and the media information box ('mdia' box) of the corresponding track. ) Thereafter, the terminal confirms the handler type ('handler_type') in step 707, and confirms the remaining boxes including the stereoscopic information in step 708, and displays the stereoscopic content on the screen.

  The terminal identifies the first view sequence and the second view sequence in step 708 of FIG. 8, and the process step of identifying the first view sequence and the second view sequence is the first step of the present invention. As described in FIG. 5 according to one embodiment, the order can be changed by the above-described method of distinguishing the first view sequence from the second view sequence.

  If the handler type of the track analyzed in step 707 is a stereoscopic video type ('svid'), the corresponding track includes selective information including camera parameters and display safety information as additional information for the stereoscopic video. It is a track.

  On the other hand, the process of confirming the track of the media file described in FIG. 8 and displaying it on the screen may not be sequentially executed by the terminal or the system. Here, the file format analysis process and the operation of the corresponding terminal not specifically described are in accordance with ISO / IEC14496-12 and ISO / IEC23000-11.

<Third Embodiment>
In the case of a service in which stereoscopic content and monoscopic content are simultaneously displayed in one scene as elements constituting one scene, the stereoscopic view sequence and monoscopic view sequence that must be expressed in one scene In order to decode and display the images, it is necessary to connect the tracks of the two view sequences so that the user knows that they are related to each other. However, since there is no method for indicating such a relationship in the current stereoscopic file format, the third embodiment of the present invention proposes a method for solving this problem.

  Among the boxes defined in the 'ISO / IEC 14496-12 ISO-based media file format' document, there are a handler reference box ('hdlr' box) and a track reference box ('tref' box). As described above, the handler reference box ('hdlr' box) represents the media data type of the current track using the handler type ('handler_type'). In the third embodiment of the present invention, in order to connect a stereoscopic view sequence track and a monoscopic view sequence track that must be expressed in one scene, as shown in Table 6, Add the handler type ('handler_type') 'mvid' of the handler criteria box ('hdlr' box).

  A track reference box ('tref' box) is a box used to connect another track referenced by the current track using a reference type ('reference_type') and a track ID ('track_ID'). . In the third embodiment of the present invention, in order to connect two related tracks, the reference type ('reference_type') of the track reference box ('tref' box) is set to 'scmi' as shown in Table 7. Add '.

  FIG. 9 illustrates content having a stereoscopic view sequence and a monoscopic view sequence that are simultaneously displayed in one scene using newly defined 'handler_type' and 'reference_type' according to the third embodiment of the present invention. A file structure for connecting a stereoscopic view sequence track and a monoscopic view sequence track formed in one scene is shown.

  Referring to FIG. 9, the current track is a stereoscopic view sequence track, and the monoscopic view sequence track that must be displayed in one scene together with the stereoscopic view sequence is a track reference box ('tref'). Box) 810 to connect to the current track. When the reference type is set as reference_type = 'scmi', the referenced track is a track that contains monoscopic content (spatially) that must be displayed in one scene with the stereoscopic video track being the referenced track. Media track). In addition, when the handler type of the referenced track's handler criteria box ('hdlr' box) 820 is set as handler_type = 'mvid', the referenced track is displayed in one scene with a stereoscopic view sequence. This is a monoscopic view sequence track (spatial media track).

  FIG. 10 illustrates a stereoscopic view including two or more view sequences, which includes a stereoscopic view sequence and a monoscopic view sequence that are simultaneously displayed in one scene according to the third embodiment of the present invention. The file structure which connects a sequence and a monoscopic view sequence mutually is shown.

  Referring to FIG. 10, the stereoscopic left-view sequence track and the stereoscopic right-view sequence track that constitute the stereoscopic video have one track reference box ('tref' box) 910 and 920, respectively. It can be connected to reference a track of a monoscopic view sequence that must be displayed together in the scene. Also in this case, the reference type of the track reference box ('tref' box) 910, 920 is set to reference_type = 'scmi' for each of the track of the stereoscopic left view sequence and the track of the stereoscopic right view sequence. If set and the handler type of this referenced track's handler criteria box ('hdlr' box) 930 is set as handler_type = 'mvid', the track of the referenced view sequence will be with a stereoscopic view sequence A monoscopic view sequence track (a spatially combined media track) that must be displayed in one scene.

<Fourth Embodiment>
FIG. 11 illustrates a stereo that is simultaneously displayed in one scene using a newly defined handler type (handler_type) 'svid' and a reference type ('reference_type') 'avmi' according to the fourth embodiment of the present invention. FIG. 5 further illustrates another file structure that interconnects a stereoscopic view sequence track and a monoscopic view sequence track with content having a stereoscopic view sequence and a monoscopic view sequence.

  In the fourth embodiment of the present invention, the track is referred to by the same method as the track reference method used in the first embodiment of the present invention. However, in the fourth embodiment of the present invention, the track reference box ('tref' box) 1010 is displayed simultaneously in one scene, as well as a connection with the stereoscopic view sequence paired with that view sequence. Also used to connect to monoscopic view sequences. Reference_type = 'avmi' in the track reference box ('tref' box) 1010, handler reference box ('hdlr' box) 1020 in the referenced stereoscopic video track, handler_type = 'svid', referenced monoscopic video track By setting handler_type = 'vide' in the handler reference box ('hdlr' box) 1030, the remaining one of the stereoscopic views paired with the first view sequence is combined with the stereoscopic content. It can be distinguished from a monoscopic view sequence that must be displayed simultaneously in one scene.

  In the example of FIG. 11, since the track of the stereoscopic right-side view sequence is determined as the first view sequence track by the above-described method for determining the first view sequence, the track reference box ('tref' box) is included. The track is set as the second view sequence.

  Furthermore, in the present embodiment, when the connection relationship between stereoscopic contents composed of two or more tracks is indicated using reference_type = 'avmi', the handler_type of the first view sequence track is set to 'vide'. And the handler_type of the second view sequence track is 'svid'. Of course, in such a distinction, the referenced view sequence can be a video type ('video'). Also, all view sequences can be shown using only the video type ('video') without separate division.

<Fifth Embodiment>
In the fifth embodiment of the present invention, a view sequence that shows and refers to a relationship between stereoscopics composed of two or more view sequences by using the track reference method as in the first embodiment of the present invention. A structure of a stereoscopic media file generated by storing display and camera information relative to a reference view sequence with respect to the remaining view sequences other than.

  In the stereoscopic video related information, the additional information is included in the stereoscopic content as described in the first and second embodiments of the present invention. Additional information that can be included in the stereoscopic content includes display and camera information for the stereoscopic video, including information related to the stereoscopic video obtained in the process of acquiring the stereoscopic video. Display and camera information for such stereoscopic video may include baseline, focal_length, convergence_distance, transformation, rotation, etc., and display safety information may include display size related information, viewing distance, mismatch information, etc. . Although referred to as additional information in the present invention, this information is selective information. Therefore, the box containing the relevant information can be expressed as a selective box.

  One of the methods for storing stereoscopic video display and camera information is to assign the relative values for the reference view sequence display and camera information to the remaining view sequences based on the reference view sequence. Store as the parameter value of the field. For example, assuming that the referenced view sequence is the first view sequence, the display and camera information of the first view sequence are all stored as 0, relative to the referenced view sequence for each field. The parameter values of the display and camera information are stored in the remaining view sequence excluding the first view sequence, that is, the second view sequence. Since the display and camera information for the stereoscopic video of the view sequence to be referred to are all set to 0, the corresponding information can be omitted. Accordingly, display and camera information relative to the display and camera information of the referenced view sequence can be stored only in the remaining view sequences. For example, assuming that the distance between cameras with respect to two view sequences, which is one of stereoscopic video display and camera information, is 5, the value of the field for the corresponding information of the referenced view sequence is 0. Therefore, the value “0” is omitted, and the distance 5 from the camera of the view sequence to be referred to is stored as a field value for the corresponding information of the remaining view sequences.

  FIG. 12 shows a file structure according to the fifth embodiment of the present invention.

  In FIG. 12, it is assumed that the view sequence to be referred to is a first view sequence, and in a stereoscopic media file structure composed of two view sequences, a display and a camera relative to the first view sequence. A 'scdi' box 1140 for storing information is included in the track of the second view sequence, and the track of the first view sequence is the same method as the track reference method used in the first embodiment of the present invention. , The handler type 'svid' 1110 and the reference type 'avmi' 1120 are used to connect to the track of the second view sequence having the 'scdi' information. Even in this case, the video type “video” can be used as the handler type 1110 of the stereoscopic view sequence. In FIG. 12, the first view sequence is shown as a left-side view sequence without a track reference box ('tref' box) and is a stereoscopic view sequence with a track reference box ('tref' box), i.e., a reference type. The view sequence track where ('reference_type') = 'avmi' includes a 'scdi' box with relative display and camera information from the referenced view sequence.

  FIG. 13 illustrates a case where a track having a track reference box ('tref' box) 1150 that refers to a track is provided independently of a track having a 'scdi' box 1160. Here, the view sequence referred to by the ‘scdi’ information is the left view sequence, and the relative ‘scdi’ information of the left view sequence is included in the track of the right view sequence.

  FIG. 14 shows a file structure generated by extending the method of the fifth embodiment of the present invention for multi-view content having two or more view sequences.

  Referring to FIG. 14, according to the fifth embodiment of the present invention, 'scdi' boxes 1224 and 1234 storing relative display and camera information from the first view sequence are displayed in the remaining views other than the first view sequence. The tracks of the first view sequence are included in a plurality of view sequences, and the handler type 'svid' 1210 and the reference type 'avmi' 1220 are the same as the track reference method used in the first embodiment of the present invention. 1230 is used to connect to the remaining view sequence tracks having 'scdi' information. Even in this case, the video type “video” can be used as a handler type (1222 and 1232) of the stereoscopic video.

  FIG. 15 illustrates a case where a track having a 'tref' box 1250 that references a track is provided separately from a track having 'scdi' boxes 1260 and 1270.

  In the first to fifth embodiments, the title, name, and semantic of the reference type and the handler type can be expressed by different titles, names, and meanings as long as they correspond to the same object and method. .

  Next, a system for generating and playing back a media file using the media file format according to an embodiment of the present invention will be described. A system according to an embodiment of the present invention can be mainly composed of a media file generation device and a media file playback device.

  FIG. 16 shows a media file generation device according to an embodiment of the present invention.

  Referring to FIG. 16, the media file generation apparatus according to the present embodiment includes two or more cameras 1301 to 1304, an input unit 1310, a video signal processor 1320, a storage unit 1330, an encoder 1340, and a file generation unit 1350.

  The cameras 1301 to 1304 each capture a specific subject in the left view or the right view, and output different view sequences. When monographic video is served, the monoscopic video data is input to the input unit 1310 together with the stereoscopic video data. At this time, information such as camera parameters can also be transmitted to the input unit 1310.

  The video signal processor 1320 preprocesses all video data received through the input unit 1310. Here, the pre-processing operation converts an external video value, that is, an analog value generated by recognizing a light and color component by a CCD (Charge Coupled Device) or CMOS (Complementary Metal-Oxide Semiconductor) type sensor into a digital value. Means an action to convert.

  The storage unit 1330 stores the video data preprocessed by the video signal processor 1320 and provides the video data to the encoder 1340. FIG. 16 shows the storage unit 1330, but this storage unit 1330 does not separately show a storage configuration for buffering between the components shown in FIG. The encoder 1340 encodes each video data provided from the storage unit 1330. The encoding operation performed by the encoder 1340 is encoding of data that can be omitted as necessary.

  The file generation unit 1350 generates a media file 1300 using each video data encoded by the encoder 1340. This video data is stored in the data area, particularly the media data area, and track reference information indicating the relationship between the video data, handler information indicating the media type of each video data, the configuration type of the stereoscopic video, and the camera and display The information is stored in a box for the corresponding information of each video data track. The generated media file 1300 is input or transferred to a stereoscopic media file playback device, and the media file playback device plays back and displays the stereoscopic service video from the media file 1300.

  Next, a stereoscopic media file playback device according to an embodiment of the present invention will be described.

  FIG. 17 is a block diagram showing a media file playback apparatus according to an embodiment of the present invention. As shown in FIG. 17, the media file playback apparatus includes a file analyzer 1410, a decoder 1420, a storage unit 1430, a playback unit 1440, and a display unit 1450.

  The file analyzer 1410 receives and analyzes the media file 1400 generated by the file generation unit 1350 of the media file generation device, for example. At this time, the file analyzer 1410 analyzes the information stored in each of the file, moov, track, and metadata areas, and then extracts the video data 1401 to 1404 stored in the media data area. Through the file analysis operation shown in FIGS. 5 and 8, the file analyzer 1410 can also extract information indicating relevance including reference information between tracks and identify related tracks.

  The decoder 1420 decodes the extracted video data. In the present embodiment, the decoder 1420 is used when the media file generation apparatus encodes data using the encoder 1340. The decrypted data is stored in the storage unit 1430. The playback unit 1440 synthesizes and plays back related stereoscopic view sequences using the video data stored in the storage unit 1430 based on the identification information, and / or the related stereoscopic view sequences and monoscopic views. Play the sequence together. The display unit 1450 displays the reproduced view sequence. The display unit 1450 can employ a barrier LCD. In this case, the barrier LCD can display each video on the screen by turning off the monoscopic video of the media file and turning on the stereoscopic video.

  While the invention has been illustrated and described with reference to specific embodiments, various changes in form and detail can be made without departing from the spirit and scope of the invention as defined by the appended claims. Certainly it will be apparent to those with ordinary knowledge in the art.

100, 1300, 1400 Media file 110 Movie data box ('moov' box)
120 Media data box ('mdat' box)
210, 310, 510, 520, 610, 620, 630, 810, 910, 920, 1010, 1150, 1250 Track reference box ('tref' box)
220, 320, 330, 820, 930, 1020, 1030 Handler reference box ('hdlr' box)
1110, 1210 Handler type 'svid'
1120, 1220, 1230 Reference type 'avmi'
1140, 1160, 1224, 1234 'scdi' box 1301 Information related to each video 1302 Camera (left)
1303 Camera (right)
1304 Multi-view or mono-view camera 1310 Input unit 1320 Video signal processor 1330, 1430 Storage unit 1340 Encoder 1350 File generation unit 1410 File analyzer 1420 Decoder 1440 Playback unit 1450 Display unit

Claims (15)

A computer-readable recording medium having stored data,
A media data box containing two or more media data items;
A movie data ('moov') box containing information about the view sequence data of the media data;
The computer-readable recording medium, wherein the 'moov' box includes track reference information indicating that a track box for one view sequence refers to a track box of another view sequence.   The computer-readable recording medium according to claim 1, wherein the track reference information is included in a track reference box of the track box. The view sequence data is divided into first view sequence data and second view sequence data,
The computer-readable recording medium according to claim 2, wherein the second view sequence data includes the track reference box.   The computer-readable medium of claim 3, wherein the 'moov' box of the second view sequence data includes a box in which display and camera information relative to the first view sequence data is stored. Recording medium. The 'moov' box includes a track header in which header information for each view sequence data is stored,
The computer-readable recording medium according to claim 1, wherein the view sequence data to be referenced is distinguished from the view sequence data to be referenced by a track identifier (ID) stored in the track header. Receiving a media file;
Analyzing a media data box of the received media file containing two or more view sequence data and a movie data ('moov') box containing information about the view sequence data;
The video is generated based on the reference view sequence and the reference view sequence according to the track reference information included in the 'moov' box and indicating that the track box for one view sequence refers to the track box for another view sequence. A computer-implemented method comprising:   The computer-implemented method of claim 6, wherein the track reference information is included in a track reference box of the track box. The view sequence data is divided into first view sequence data and second view sequence data,
The computer-implemented method of claim 7, wherein the second view sequence data includes the track reference box.   The computer-implemented method of claim 8, wherein the 'moov' box of the second view sequence data includes a box for storing display and camera information relative to the first view sequence data. . The 'moov' box includes a track header in which header information for each view sequence data is stored,
7. The computer-implemented method according to claim 6, wherein the view sequence data to be referenced is distinguished from the view sequence data to be referenced by a track identifier (ID) stored in the track header. A media data box of a media file including two or more view sequence data and a movie data ('moov') box including information on the view sequence data are analyzed, and one view sequence included in the 'moov' box is analyzed. A file analysis unit that extracts a video based on a reference view sequence and a reference view sequence according to track reference information indicating that the reference track box refers to a track box of another view sequence;
And a display unit for displaying the extracted video.   The terminal apparatus according to claim 11, wherein the track reference information is included in a track reference box of the track box. The view sequence data is divided into first view sequence data and second view sequence data,
The terminal apparatus according to claim 12, wherein the second view sequence data includes the track reference box.   The terminal apparatus according to claim 13, wherein the 'moov' box of the second view sequence data includes a box in which display and camera information relative to the first view sequence data are stored. The 'moov' box includes a track header in which header information for each view sequence data is stored,
The terminal device according to claim 11, wherein the file analysis unit distinguishes the view sequence data to be referenced from the view sequence data to be referred to based on a track identifier (ID) stored in the track header.

Images