JP5171270B2 - Multiplexing method and apparatus for moving picture, and program and computer-readable recording medium - Google Patents

Description

  The present invention is a technique related to multiplexing of encoded image data using an inter-frame predictive encoding method.

  Video coding includes coding methods that apply layer coding, such as scalable coding and multi-view video coding. In scalable coding, functions such as SNR scalability and spatial scalability can be realized. The SNR scalability is a function that improves the image quality of an image by adding the encoded data to the base layer encoded data. The spatial scalability is the encoded data for the base layer encoded data. This is a function that improves the spatial resolution of an image by adding.

  FIG. 11 shows a conceptual diagram of scalable coding. In scalable encoding, as shown in FIG. 11, the amount of code is reduced by reusing a decoded image or encoded information between layers.

  In recent years, the video encoding system As a H.264 extension method, a method called SVC is standardized as Annex G (see Non-Patent Document 1).

  Layer coding is also applied in multi-view video coding. In recent years, the video encoding system H.264 A method called MVC is being standardized as an H.264 extension method. In MVC, a mechanism for collectively encoding a plurality of viewpoint videos is employed. In motion picture encoding, motion compensation encoding is used. In the case of multi-view video, in addition to motion compensation, disparity compensation encoding that applies inter-frame prediction encoding between viewpoint videos is used. In MVC, by using parallax compensation, the compression efficiency can be improved as compared to encoding the viewpoint video individually. In addition, MVC assumes a video expression that allows a viewer to freely select a viewpoint position, such as a free viewpoint video as an application (see Non-Patent Document 2 and Non-Patent Document 3).

  For this reason, it is assumed that the video scene is photographed by a large number of cameras and the video at the viewpoint position that the viewer wants to view is selected. That is, it is assumed that some of the captured viewpoint video data are selected, decoded, and reproduced. Such a function of selecting and decoding several viewpoint videos from a very large number of viewpoint videos is called viewpoint video scalability. MVC is an encoding method that assumes multi-view video, but the target to be encoded together is not limited to camera video. That is, it can be considered as a coding method that generally compresses a plurality of moving images together.

  As described above, this MVC has a layer structure for one base layer as in scalable coding in order to select and decode several moving images from a large number of moving images. Instead, the configuration is such that a plurality of base layers have a layer structure.

  FIG. 12 is a diagram illustrating an example of a reference relationship of MVC, and FIG. 13 is a diagram illustrating a configuration example of encoded data of MVC. For example, if there is a reference relationship as shown in FIG. 12, the encoded data of MVC has a configuration as shown in FIG. 13 for frames at the same time. This group of frames at the same time is called an access unit. In MVC, there are a plurality of layers corresponding to the base layer. In MVC, a reference relationship between moving images is described by an MVC extension in a subset SPS (subset_seq_parameter_set). The subset SPS is included in the encoded data, and a plurality of subsets can be set. In each frame, a subset SPS to be used can be designated.

  On the other hand, a file format technique is used to store such encoded video data as a file. The file format is also a technique for multiplexing encoded data, and has a function of extracting encoded data related to specific information with low delay by structuring the encoded data stream. Generally, in a file format, various data streams are stored in a track, and track information is divided and multiplexed in units of specific times. Each track stores metadata such as CG data and copyright information in addition to video or audio encoded data.

  The simplest file format is the AVI format. In the AVI format, video data and audio data are multiplexed in units of one frame. Another ISO standard file format is an international standardized format. In the ISO format, a sample is defined as being equivalent to a frame, and a track is divided and multiplexed in units of samples.

  In the ISO media file format, in addition to encoded data, time information such as a frame number and position information in the file at the head of the frame can be configured as separate tracks as management information. When playing back a file, by checking the track in which this management information is recorded, it is possible to know the start position information of the frame, and to access the start of the desired frame with low delay.

An example of the relationship between tracks and management information in a file is shown in FIG. The encoded data track stores encoded data of each moving image, and the management information track stores frame position information at each time. A file format has also been proposed for the SVC described above. The SVC also forms a file in units of tracks (see Non-Patent Document 4).
Heiko Schwarz, Detlev Marpe, and Thomas Wiegand, “Overview of the Scalable Video Coding Extension of the H.264 / AVC Standard”, IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY, VOL.17, NO.9, SEPTEMBER 2007, pp .1103-1120. Hideaki Kizen, “Trends in MPEG 3DAV international standardization,” Journal of the Institute of Image Information and Television Engineers 60, 2, pp.143-149, (2006). Hideaki Kizen, “International Standardization Trend of Multi-View Video Coding MVC,” Journal of the Institute of Image Information and Television Engineers 61, 4, pp.426-430, (April 2007). Peter Amon, Thomas Rathgen, and David Singer, “File Format for Scalable Video Coding”, IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY, VOL.17, NO.9, SEPTEMBER 2007, pp.1174-1185.

  For example, when decoding encoded data encoded using conventional layer encoding, such as SVC, the lower layer is always decoded and the upper layer is decoded. On the other hand, when there are a plurality of base layers as in MVC, it is not always necessary to decode lower layers. Therefore, when decoding a desired moving image, only a moving image necessary for decoding may be selected and decoded. Since such a moving image reference relationship in MVC is described in SPS, the decoder has derived the reference relationship by decoding SPS.

  However, in order to decode the SPS, it is necessary to find the SPS from the encoded data. For this reason, in particular, when it is stored in a file, it is necessary to sequentially search and decode the SPS from the beginning of the file, which is a heavy load. It is an object of the present invention to reduce this load.

  Based on the technical background of the present invention, the problem of the present invention will be described in more detail. When encoding a plurality of moving images collectively, inter-frame prediction encoding is applied between the moving images. It is not always necessary for all moving images to refer to other moving images, and there exists a moving image that refers only to its own image. Conversely, such a moving image has an advantage that it can be decoded independently. In multi-view video encoding, a video that can be decoded independently is used as a base layer, and other video is encoded as an enhancement layer. In the case of multi-view video encoding, since many video are encoded together, there is a feature that a plurality of base layers can be included in encoded data.

  When there are a plurality of base layers in the encoded data, if only the base layer can be extracted, a partial moving image can be decoded, which is very effective. Information such as which moving image is the base layer can be included in the encoded data.

  However, in the configuration of the conventional encoded data, it is necessary to decode the encoded data in order to know the position of each base layer in the file. For this reason, there is a problem that a large load is applied to extract only the encoded data of the base layer.

  It is an object of the present invention to provide a new technical means for reducing the load when taking out encoded data of such a base layer (moving image that can be decoded independently).

  The present invention solves the above problems as follows. There is a file format technique for multiplexing encoded data. In this file format technology, each frame head position information is stored in management information separately into encoded data and management information. In the present invention, such management information includes, in addition to the frame head position information, information indicating whether or not independent decoding is possible regardless of other moving images. Thereby, it is possible to know the head position of a frame that can be decoded independently only by reading the management information, and the encoded data of such a frame can be taken out with a low load.

Specifically, the present invention is a multiplexing method in which encoded data created by predictive encoding among n (n ≧ 2) moving images is multiplexed to form a file . For each frame time , the position information in the file at the head of the moving image, the moving image number for distinguishing the moving image, and whether the decoded image information of another moving image frame is necessary when decoding the frame The management information for n moving images including the independent decoding identification information indicating the above is multiplexed and set at a specific position of the encoded data.

  According to this multiplexing method, by decoding the management information at a specific position in the encoded data, the start position of the frame of the moving image frame that becomes the base layer from the file without decoding all the encoded data And the encoded data of the frame can be extracted. At the same time, the moving image number of the moving image can be obtained without decoding the encoded data. As a result, when the moving image number of the moving image serving as the base layer is known in advance, the information of the reference relationship between the moving images included in the encoded data is not searched and decoded, and the calculation load is low. The encoded data of the desired moving image can be decoded and reproduced with a delay.

  In this invention, with respect to the position where management information such as independent decoding identification information is multiplexed in the encoded data, the position in the predetermined file such as the end of all encoded data, before all encoded data, etc. Or before or after the track including the encoded data.

Further, the present invention is a multiplexing method for forming a file by multiplexing encoded data created by predictive encoding between n (n ≧ 2) moving images, and comprising a file at the head of a frame of the moving image. N pieces of information including position information in the file, a moving image number for distinguishing moving images, and independent decoding identification information indicating whether or not decoded image information of a frame of another moving image is necessary when decoding the frame The management information for the moving image is the first track, the reference relationship information in the predictive coding between the encoded data and the moving image is the second track, and the first track and the second track are multiplexed independently. It is characterized by becoming.

  According to this multiplexing method, by decoding the independent decoding identification information included in the first track, it is possible to decode the frame of the moving image frame serving as the base layer without decoding the encoded data of the second track. The head position can be obtained and the encoded data of the frame can be extracted. At the same time, the moving image number of the moving image can be obtained without decoding the encoded data. Thereby, when the moving image number of the moving image serving as the base layer is known in advance, the information on the reference relationship between the moving images included in the encoded data is not decoded, and with low calculation load and low delay, It is possible to decode and reproduce encoded data of a desired moving image.

  Furthermore, the position information in the information file indicating the reference relationship between the moving images may be multiplexed together with the position information of the encoded data. In this case, by referring to this position information, information indicating the reference relationship between moving images can be read from the encoded data. From the reference relationship obtained by reading, a moving image number referring to the moving image can be obtained. As a result, when the moving image number of the moving image serving as the base layer is known in advance, the encoded data of the base layer and the encoded data of the moving image that refers to the extracted data are extracted, and the calculation load is low and the delay is low. , The encoded data of the desired moving image can be decoded and reproduced.

Further, a related invention of the present invention is a multiplexing method for forming a file by multiplexing encoded data created by predictive encoding between a plurality of moving images, in a file at the head of a frame of a moving image. Multiplexing position information, a moving picture number for distinguishing moving pictures, and a reference moving picture number that is a moving picture number of a moving picture necessary for decoding the frame, and setting it to a specific position of encoded data It is characterized by.

  According to this multiplexing method, by decoding management information at a specific position in the encoded data, the moving image necessary for decoding the frame to be decoded without decoding all the encoded data. A reference moving image number which is a moving image number of an image is obtained, and all encoded data of the moving image can be extracted. As a result, it is possible to decode and reproduce the encoded data of a desired moving image with low computational load and low delay without searching for and decoding information on the reference relationship between the moving images included in the encoded data. Can do.

  In addition to the reference moving image number, frame head position information of a frame having a moving image number corresponding to the reference moving image number may be multiplexed as management information.

  In the present invention, with respect to the position where the management information such as the reference video number is multiplexed with the encoded data, the position in the predetermined file such as the end of all the encoded data, before all the encoded data, etc. Or before or after the track including the encoded data.

Another related invention of the present invention is a multiplexing method in which encoded data created by predictive encoding between a plurality of moving images is multiplexed to form a file. Position information, a moving image number for distinguishing moving images, and a reference moving image number which is a moving image number of a moving image necessary for decoding the frame as a first track, and between encoded data and moving images The reference relationship information in the predictive encoding is used as the second track, and the first track and the second track are multiplexed independently.

  According to this multiplexing method, by decoding the reference video number included in the first track, the frame to be decoded from the file can be decoded without decoding the encoded data of the second track. Thus, it is possible to obtain the moving image number of the moving image necessary for obtaining all the encoded data of the moving image. As a result, the encoded data of the desired moving image can be decoded and reproduced with low computational load and low delay without decoding the reference relationship information between the moving images included in the encoded data.

Still another related invention of the present invention is a multiplexing method for forming a file by multiplexing encoded data created by predictive encoding between a plurality of moving images, and in a file at the head of a frame of a moving image. If the decoded image or encoded data of the frame is used for decoding other moving images, the position information of the moving image, the moving image number of the moving image of the use destination, It is characterized in that a certain use destination moving picture number is multiplexed and set at a specific position of encoded data.

  According to this multiplexing method, by decoding the information at a specific position in the encoded data, the subsequent frame (frame to be processed) can be determined for a specific frame (processing target frame) without decoding all the encoded data. When decoding the encoded data of the use destination frame), it is possible to determine whether to use the decoded image or encoded data of the processing target frame. Thereby, the processing load of the distribution server when distributing the video data can be reduced. For example, in a distribution system in which a video distribution server reads a file and a client requests encoded data of a desired moving image, when the encoded data of a moving image usage frame requested by the client is distributed, It is possible to select and distribute encoded data of a processing target frame necessary for frame decoding.

  In the present invention, the position where the management information such as the use destination video number is multiplexed with the encoded data, the end of all the encoded data, before all the encoded data, etc. It may be before or after the position or the track containing the encoded data.

Still another related invention of the present invention is a multiplexing method for forming a file by multiplexing encoded data created by predictive encoding between a plurality of moving images, and in a file at the head of a frame of a moving image. If the decoded image or encoded data of the frame is used for decoding other moving images, the position information of the moving image, the moving image number of the moving image of the use destination, A certain destination video number is set as the first track, reference relationship information in predictive encoding between the encoded data and the video is set as the second track, and the first track and the second track are multiplexed independently. It is characterized by doing.

  According to this multiplexing method, the management information contained in the first track is decoded, so that the subsequent frames of the specific frame (processing target frame) are decoded without decoding the encoded data of the second track. When decoding the encoded data of (use destination frame), it is possible to determine whether to use the decoded image or encoded data of the processing target frame. Thereby, the processing load of the distribution server when distributing the video data can be reduced. For example, in a distribution system in which a video distribution server reads a file and a client requests encoded data of a desired moving image, when the encoded data of a moving image usage frame requested by the client is distributed, It is possible to select and distribute encoded data of a processing target frame necessary for frame decoding.

  Although an example in which management information is set for each frame has been described above, in the present invention, management information may be set for each slice obtained by dividing a frame. In this case, the management information includes slice head position information instead of the frame head position. In addition, multiplexing and reading are performed not on a frame basis but on a slice basis.

  According to the present invention, a moving image code that can be decoded independently from a file obtained by multiplexing encoded data obtained by collectively encoding a plurality of moving images with low computational load without decoding the encoded data. Encoded data and encoded data of a desired moving image can be extracted.

Embodiments of a multiplexing method according to the present invention and related invention and a method for reading and reproducing a multiplexed file will be described below with reference to the drawings. The first embodiment described below is mainly an embodiment of the present invention, and the second and third embodiments are embodiments of the related invention of the present invention.

  In the first embodiment, an example is shown in which the frame head position information, moving picture number, and independent decoding identification information are multiplexed as management information, and all encoded data are set at the end of multiplexing. It is assumed that there are four moving images A, B, C, and D, which are encoded in advance, and the position information of the frame head in each moving image is known.

  FIG. 1 shows the configuration of the multiplexing apparatus. The multiplexing apparatus 10 includes a moving image A encoded data input unit 101, a moving image B encoded data input unit 102, and a moving image C encoded data to which encoded data of moving images A, B, C, and D are input. Input unit 103, moving image D encoded data input unit 104, moving image A frame position information input unit 106 for inputting position information in the moving image at the head of each moving image, moving image B frame position information input unit 107, a moving image C frame position information input unit 108, a moving image D frame position information input unit 109, a reference relationship data input unit 105 for inputting information indicating a reference relationship between moving images, encoded data of a moving image, Data switching unit 110 that switches input of frame position information, encoded data multiplexing unit 111 that multiplexes encoded data, and moving image number setting unit that sets a moving image number 12, an independent decoding identification information setting unit 113 for setting independent decoding identification information, a frame head position information setting unit 114 for setting head information of a frame position, a management information multiplexing unit 116 for multiplexing management information, It comprises a file multiplexing unit 115 that multiplexes encoded data and management information.

  Among the above, the moving image number setting unit 112 sets moving image numbers 0, 1, 2, and 3 for moving images A, B, C, and D, respectively. Frame position information input units 101 to 104 for moving images A to D are assumed to input position information of the next frame. The frame head position information setting unit 114 counts the frame position information of each moving image, and sets the position information of each frame after the encoded data multiplexing unit 111 multiplexes the encoded data. The independent decoding identification information setting unit 113 sets 1 as independent decoding identification information when independent decoding is possible, and sets 0 as independent decoding identification information when independent decoding is impossible. The file multiplexing unit 115 sets the management information created by the management information multiplexing unit 116 after the coded data created by the coded data multiplexing unit 111. When there is reference relation data, the encoded data multiplexing unit 111 multiplexes the encoded data before the encoded data of each moving image.

  The data switching unit 110 switches between encoded data and file head position information as follows according to the reference relation data. In order from the moving images A to D, it is first determined whether the moving images can be decoded independently. If it can be decoded independently, select it as it is. If it cannot be decoded independently, it is determined whether the encoded data of the moving image to be referred to has already been selected. If it has already been selected, it is selected as it is. If it has not been selected yet, the moving image is not selected until all the moving images to be referenced are selected.

  Assume that the reference relationship between moving images is as follows. First, in the first two frames, moving images A and C can be decoded independently, moving image B refers to moving image A, and moving image D refers to moving image C. Thereafter, in two frames, moving images A and D can be decoded independently, and moving images B and C refer to moving images A and D.

  When four frames of the moving images A, B, C, and D are multiplexed on the above assumption, each frame of these moving images is multiplexed as follows.

  First, the first frame is multiplexed as follows. The reference relationship data input unit 105 inputs reference relationship data between moving images. The data switching unit 110 determines the order in which the encoded data is multiplexed according to the reference relation data. Here, the order is determined as follows. Since the moving image A can be decoded independently, the moving image A is first selected. Since the moving image B refers only to the moving image A, the moving image B is selected next. Since the moving image C can be decoded independently, the moving image C is selected next. Then, the moving image D is selected. The encoded data multiplexing unit 111 multiplexes the reference relation data. The frame head position information setting unit 114 adds the file size by the data size of the reference relation data.

  In accordance with the order of multiplexing, the data switching unit 110 switches between encoded data and frame head position information. First, the moving image A is selected. The moving image A encoded data input unit 101 inputs encoded data for one frame, and the moving image A frame position information input unit 106 inputs frame position information of the next frame. The encoded data multiplexing unit 111 multiplexes encoded data. The frame head position information setting unit 114 adds the input frame position information to the file size and sets it as the frame position information of the file. The moving image number setting unit 112 sets moving image number 0. The independent decoding identification information setting unit 113 sets 1 as the independent decoding identification information. The management information multiplexing unit 116 multiplexes the frame position information, the moving image number, and the independent decoding identification information.

  Subsequently, the data switching unit 110 selects the moving image B. The moving image B encoded data input unit 102 inputs encoded data for one frame, and the moving image B frame position information input unit 107 inputs frame position information of the next frame. The encoded data multiplexing unit 111 multiplexes encoded data. The frame head position information setting unit 114 adds the input frame position information to the file size and sets it as the frame position information of the file. The moving image number setting unit 112 sets moving image number 1. The independent decoding identification information setting unit 113 sets 0 as the independent decoding identification information. The management information multiplexing unit 116 multiplexes the frame position information, the moving image number, and the independent decoding identification information.

  Subsequently, the data switching unit 110 selects the moving image C. The moving image C encoded data input unit 103 inputs encoded data for one frame, and the moving image C frame position information input unit 108 inputs the frame position information of the next frame. The encoded data multiplexing unit 111 multiplexes encoded data. The frame head position information setting unit 114 adds the input frame position information to the file size and sets it as the frame position information of the file. The moving image number setting unit 112 sets moving image number 2. The independent decoding identification information setting unit 113 sets 1 as the independent decoding identification information. The management information multiplexing unit 116 multiplexes the frame position information, the moving image number, and the independent decoding identification information.

  Subsequently, the data switching unit 110 selects the moving image D. The moving image D encoded data input unit 104 inputs encoded data for one frame, and the moving image D frame position information input unit 109 inputs frame position information of the next frame. The encoded data multiplexing unit 111 multiplexes encoded data. The frame head position information setting unit 114 adds the input frame position information to the file size and sets it as the frame position information of the file. The moving image number setting unit 112 sets moving image number 3. The independent decoding identification information setting unit 113 sets 0 as the independent decoding identification information. The management information multiplexing unit 116 multiplexes the frame position information, the moving image number, and the independent decoding identification information.

  The above processing of the moving images A, B, C, and D is further performed for one frame.

  Next, the reference relationship data input unit 105 inputs reference relationship data between moving images. The data switching unit 110 determines the order in which the encoded data is multiplexed according to the reference relation data. Here, the order is determined as follows. Since the moving image A can be decoded independently, the moving image A is first selected. Since the moving image B refers to the moving image A and the moving image D, the moving image B is not selected. Since the moving image C refers to the moving image A and the moving image D, the moving image C is not selected. Since the moving image D can be decoded independently, the moving image D is selected next. Then, the moving image B is selected, and the moving image C is selected. The encoded data multiplexing unit 111 multiplexes the reference relation data. The frame head position information setting unit 114 adds the file size by the data size of the reference relation data.

  In accordance with the order of multiplexing, the data switching unit 110 switches between encoded data and frame head position information. First, the moving image A is selected. The moving image A encoded data input unit 101 inputs encoded data for one frame, and the moving image A frame position information input unit 106 inputs frame position information of the next frame. The encoded data multiplexing unit 111 multiplexes encoded data. The frame head position information setting unit 114 adds the input frame position information to the file size and sets it as the frame position information of the file. The moving image number setting unit 112 sets moving image number 0. The independent decoding identification information setting unit 113 sets 1 as the independent decoding identification information. The management information multiplexing unit 116 multiplexes the frame position information, the moving image number, and the independent decoding identification information.

  Subsequently, the data switching unit 110 selects the moving image D. The moving image D encoded data input unit 104 inputs encoded data for one frame, and the moving image D frame position information input unit 109 inputs frame position information of the next frame. The encoded data multiplexing unit 111 multiplexes encoded data. The frame head position information setting unit 114 adds the input frame position information to the file size and sets it as the frame position information of the file. The moving image number setting unit 112 sets moving image number 3. The independent decoding identification information setting unit 113 sets 1 as the independent decoding identification information. The management information multiplexing unit 116 multiplexes the frame position information, the moving image number, and the independent decoding identification information.

  Subsequently, the data switching unit 110 selects the moving image B. The moving image B encoded data input unit 102 inputs encoded data for one frame, and the moving image B frame position information input unit 107 inputs frame position information of the next frame. The encoded data multiplexing unit 111 multiplexes encoded data. The frame head position information setting unit 114 adds the input frame position information to the file size and sets it as the frame position information of the file. The moving image number setting unit 112 sets moving image number 1. The independent decoding identification information setting unit 113 sets 0 as the independent decoding identification information. The management information multiplexing unit 116 multiplexes the frame position information, the moving image number, and the independent decoding identification information.

  Subsequently, the data switching unit 110 selects the moving image C. The moving image C encoded data input unit 103 inputs encoded data for one frame, and the moving image C frame position information input unit 108 inputs the frame position information of the next frame. The encoded data multiplexing unit 111 multiplexes encoded data. The frame head position information setting unit 114 adds the input frame position information to the file size and sets it as the frame position information of the file. The moving image number setting unit 112 sets moving image number 2. The independent decoding identification information setting unit 113 sets 0 as the independent decoding identification information. The management information multiplexing unit 116 multiplexes the frame position information, the moving image number, and the independent decoding identification information.

  The above processing of the moving images A, D, B, and C is further executed for one frame. Finally, the file multiplexing unit 115 multiplexes the encoded data obtained by the encoded data multiplexing unit 111 and the management information obtained by the management information multiplexing unit 116 to create a file. By the above processing, it can be multiplexed into a file.

  In the present embodiment, an example is shown in which a file is created by multiplexing from encoded data obtained by encoding in advance and file head position information, but a file may be configured while creating encoded data.

  Next, the operation of a file reading method for extracting encoded data of a specific moving image from the multiplexed file will be described. FIG. 2 shows the configuration of the file reading apparatus. 2 includes a file input unit 201 that reads a file, a management information reading unit 202 that reads management information, a moving image number searching unit 203 that searches for a desired moving image number from the management information, An independent decoding identification information determination unit 204 that determines independent decoding identification information from management information, a frame head position information setting unit 205 that reads frame head position information from management information, and a code that reads encoded data from a file according to the frame head position information And a digitized data reading unit 206.

  An operation when reading encoded data of a frame that can be independently decoded with a moving image D from a file created by the multiplexing apparatus 10 of FIG. 1 will be described.

  The file input unit 201 inputs a file. The management information reading unit 202 reads management information in the second half of the file. The moving image number search unit 203 searches for management information of the first frame of the moving image D. The independent decoding identification information determination unit 204 determines whether or not the decoding can be performed independently from the independent decoding identification information of the management information obtained by the search. Since the independent decoding identification information is 0 in the first frame, it cannot be decoded independently.

  Next, the moving image number search unit 203 searches for management information of the second frame of the moving image D. The independent decoding identification information determination unit 204 determines whether or not the decoding can be performed independently from the independent decoding identification information of the management information obtained by the search. In the second frame, the independent decoding identification information is 0 and cannot be decoded independently.

  Next, the moving image number search unit 203 searches the management information of the third frame of the moving image D. The independent decoding identification information determination unit 204 determines whether or not the decoding can be performed independently from the independent decoding identification information of the management information obtained by the search. Since the independent decoding identification information is 1 in the third frame, it can be decoded independently. The frame head position information setting unit 205 sets the frame head position information of this frame. The encoded data reading unit 206 reads encoded data from the set file position. At this time, the frame head position information setting unit 205 reads up to the head position of the next frame.

  Next, the moving image number search unit 203 searches for management information of the fourth frame of the moving image D. The independent decoding identification information determination unit 204 determines whether or not the decoding can be performed independently from the independent decoding identification information of the management information obtained by the search. Since the independent decoding identification information is 1 in the third frame, it can be decoded independently. The frame head position information setting unit 205 sets the frame head position information of this frame. The encoded data reading unit 206 reads encoded data from the set file position. At this time, the frame head position information setting unit 205 reads up to the head position of the next frame.

  As described above, the encoded data of the frame that can be decoded independently can be extracted from the moving image D.

  Next, the operation of a file reading method for extracting encoded data of moving images that can be decoded independently from the multiplexed file will be described. FIG. 3 shows the configuration of the file reading apparatus. The file reading apparatus 21 shown in FIG. 3 includes a file input unit 201 that reads a file, a management information reading unit 202 that reads management information, and an independent decoding identification information searching unit 207 that searches for desired independent decoding identification information from the management information. A frame head position information setting unit 205 that reads frame head position information from management information, and an encoded data reading unit 206 that reads encoded data from a file according to the frame head position information.

  An operation when reading encoded data of a moving image that can be decoded independently from a file created by the multiplexing apparatus 10 of FIG. 1 will be described.

  The file input unit 201 inputs a file. The management information reading unit 202 reads management information in the second half of the file. The independent decoding identification information search unit 207 searches the management information of the first frame whose independent decoding identification information is 1. First, the moving image A is searched with the independent decoding identification information set to 1. The frame head position information setting unit 205 sets the frame head position information of this frame. The encoded data reading unit 206 reads encoded data from the set file position. At this time, the frame head position information setting unit 205 reads up to the head position of the next frame. Subsequently, the independent decoding identification information search unit 207 searches for the moving image C. The frame head position information setting unit 205 sets the frame head position information of this frame. The encoded data reading unit 206 reads encoded data from the set file position. At this time, the frame head position information setting unit 205 reads up to the head position of the next frame. From the above, the encoded data of the moving image A and the moving image C are read.

  Similarly, the encoded data of the moving image A and the moving image C is read for the second frame.

  Next, the independent decoding identification information search unit 207 searches for the management information of the third frame whose independent decoding identification information is 1. First, the moving image A is searched with the independent decoding identification information set to 1. The frame head position information setting unit 205 sets the frame head position information of this frame. The encoded data reading unit 206 reads encoded data from the set file position. At this time, the frame head position information setting unit 205 reads up to the head position of the next frame. Subsequently, the independent decoding identification information search unit 207 searches for the moving image D. The frame head position information setting unit 205 sets the frame head position information of this frame. The encoded data reading unit 206 reads encoded data from the set file position. At this time, the frame head position information setting unit 205 reads up to the head position of the next frame. From the above, the encoded data of the moving image A and the moving image D are read.

  Similarly, the encoded data of the moving image A and the moving image C are read for the fourth frame.

  As described above, encoded video data that can be decoded independently can be extracted.

  As described above, according to this embodiment, it is possible to extract encoded data of a frame that can be decoded independently without decoding the reference relation data included in the encoded data.

  In the multiplexing apparatus 10 of this embodiment, the management information is set after the encoded data. However, when the track structure is set in the file, the management information is set in the first track and the encoded data is set. May be set as the second track.

  FIG. 4 shows a configuration example of tracks and management information in a file. The created file includes an encoded data track 30 and a management information track 40. In the encoded data track 30, moving image A encoded data 31, moving image B encoded data 32,... Are stored. . Further, management information 41, 42, 43,... For each time Tn is stored in the management information track 40, and information such as frame head position information, moving picture number, independent decoding identification information is stored in each of these. Is done.

  Next, as a second embodiment, an example is shown in which the frame head position information, moving picture number, and reference moving picture number are multiplexed as management information, and all encoded data are set at the end of multiplexing. As in the first embodiment, it is assumed that there are four moving images A, B, C, and D, which are encoded in advance, and the position information of the frame head in each moving image is known.

  FIG. 5 shows the configuration of the multiplexing apparatus. The multiplexing apparatus 11 shown in FIG. 5 has a moving image A encoded data input unit 101, a moving image B encoded data input unit 102, and a moving image C that input encoded data of moving images A, B, C, and D, respectively. Encoded data input unit 103, moving image D encoded data input unit 104, moving image A frame position information input unit 106 for inputting position information in the moving image at the head of each moving image, moving image B frame position An information input unit 107, a moving image C frame position information input unit 108, a moving image D frame position information input unit 109, a reference relationship data input unit 105 for inputting information indicating a reference relationship between moving images, and a moving image code Data switching unit 110 that switches input of encoded data and frame position information, encoded data multiplexing unit 111 that multiplexes encoded data, and moving image number that sets a moving image number A fixed part 112, a reference moving picture number setting part 117 for setting a reference moving picture number, a frame head position information setting part 114 for setting head information of a frame position, and a management information multiplexing part 116 for multiplexing management information. And a file multiplexing unit 115 for multiplexing the encoded data and the management information.

  Among the above, the moving image number setting unit 112 sets moving image numbers 0, 1, 2, and 3 for moving images A, B, C, and D, respectively. Frame position information input units 106 to 109 for moving images A to D are assumed to input position information of the next frame. The frame head position information setting unit 114 counts the frame position information of each moving image, and sets the position information of each frame after the encoded data multiplexing unit 111 multiplexes the encoded data. The reference moving image number setting unit 117 sets the moving image number of the moving image to be referred to. The file multiplexing unit 115 sets the management information created by the management information multiplexing unit 116 after the coded data created by the coded data multiplexing unit 111. When there is reference relation data, the encoded data multiplexing unit 111 multiplexes the encoded data before the encoded data of each moving image.

  The data switching unit 110 switches between encoded data and file head position information as follows according to the reference relation data. In order from the moving images A to D, it is first determined whether the moving images can be decoded independently. If it can be decoded independently, select it as it is. If it cannot be decoded independently, it is determined whether the encoded data of the moving image to be referred to has already been selected. If it has already been selected, it is selected as it is. If it has not been selected yet, the moving image is not selected until all the moving images to be referenced are selected.

  Assume that the reference relationship between moving images is as follows. First, in the first two frames, moving images A and C can be decoded independently, moving image B refers to moving image A, and moving image D refers to moving image C. Thereafter, in two frames, moving images A and D can be decoded independently, and moving images B and C refer to moving images A and D.

  When four frames of the moving images A, B, C, and D are multiplexed on the above assumption, each frame of these moving images is multiplexed as follows.

  First, the first frame is multiplexed as follows. The reference relationship data input unit 105 inputs reference relationship data between moving images. The data switching unit 110 determines the order in which the encoded data is multiplexed according to the reference relation data. Here, the order is determined as follows. Since the moving image A can be decoded independently, the moving image A is first selected. Since the moving image B refers only to the moving image A, the moving image B is selected next. Since the moving image C can be decoded independently, the moving image C is selected next. Then, the moving image D is selected. The encoded data multiplexing unit 111 multiplexes the reference relation data. The frame head position information setting unit 114 adds the file size by the data size of the reference relation data.

  In accordance with the order of multiplexing, the data switching unit 110 switches between encoded data and frame head position information. First, the moving image A is selected. The moving image A encoded data input unit 101 inputs encoded data for one frame, and the moving image A frame position information input unit 106 inputs frame position information of the next frame. The encoded data multiplexing unit 111 multiplexes encoded data. The frame head position information setting unit 114 adds the input frame position information to the file size and sets it as the frame position information of the file. The moving image number setting unit 112 sets moving image number 0. The reference moving image number setting unit 117 sets 0 as the reference moving image number. The management information multiplexing unit 116 multiplexes the frame position information, the moving image number, and the reference moving image number.

  Subsequently, the data switching unit 110 selects the moving image B. The moving image B encoded data input unit 102 inputs encoded data for one frame, and the moving image B frame position information input unit 107 inputs frame position information of the next frame. The encoded data multiplexing unit 111 multiplexes encoded data. The frame head position information setting unit 114 adds the input frame position information to the file size and sets it as the frame position information of the file. The moving image number setting unit 112 sets moving image number 1. The reference moving image number setting unit 117 sets 0 and 1 as reference moving image numbers. The management information multiplexing unit 116 multiplexes the frame position information, the moving image number, and the reference moving image number.

  Subsequently, the data switching unit 110 selects the moving image C. The moving image C encoded data input unit 103 inputs encoded data for one frame, and the moving image C frame position information input unit 108 inputs the frame position information of the next frame. The encoded data multiplexing unit 111 multiplexes encoded data. The frame head position information setting unit 114 adds the input frame position information to the file size and sets it as the frame position information of the file. The moving image number setting unit 112 sets moving image number 2. The reference moving image number setting unit 117 sets 2 as the reference moving image number. The management information multiplexing unit 116 multiplexes the frame position information, the moving image number, and the reference moving image number.

  Subsequently, the data switching unit 110 selects the moving image D. The moving image D encoded data input unit 104 inputs encoded data for one frame, and the moving image D frame position information input unit 109 inputs frame position information of the next frame. The encoded data multiplexing unit 111 multiplexes encoded data. The frame head position information setting unit 114 adds the input frame position information to the file size and sets it as the frame position information of the file. The moving image number setting unit 112 sets moving image number 3. The reference moving image number setting unit 117 sets 2 and 3 as reference moving image numbers. The management information multiplexing unit 116 multiplexes the frame position information, the moving image number, and the reference moving image number.

  The above processing of the moving images A, B, C, and D is further performed for one frame.

  Next, the reference relationship data input unit 105 inputs reference relationship data between moving images. The data switching unit 110 determines the order in which the encoded data is multiplexed according to the reference relation data. Here, the order is determined as follows. Since the moving image A can be decoded independently, the moving image A is first selected. Since the moving image B refers to the moving image A and the moving image D, the moving image B is not selected. Since the moving image C refers to the moving image A and the moving image D, the moving image C is not selected. Since the moving image D can be decoded independently, the moving image D is selected next. Then, the moving image B is selected, and the moving image C is selected. The encoded data multiplexing unit 111 multiplexes the reference relation data. The frame head position information setting unit 114 adds the file size by the data size of the reference relation data.

  In accordance with the order of multiplexing, the data switching unit 110 switches between encoded data and frame head position information. First, the moving image A is selected. The moving image A encoded data input unit 101 inputs encoded data for one frame, and the moving image A frame position information input unit 106 inputs frame position information of the next frame. The encoded data multiplexing unit 111 multiplexes encoded data. The frame head position information setting unit 114 adds the input frame position information to the file size and sets it as the frame position information of the file. The moving image number setting unit 112 sets moving image number 0. The reference moving image number setting unit 117 sets 0 as the reference moving image number. The management information multiplexing unit 116 multiplexes the frame position information, the moving image number, and the reference moving image number.

  Subsequently, the data switching unit 110 selects the moving image D. The moving image D encoded data input unit 104 inputs encoded data for one frame, and the moving image D frame position information input unit 109 inputs frame position information of the next frame. The encoded data multiplexing unit 111 multiplexes encoded data. The frame head position information setting unit 114 adds the input frame position information to the file size and sets it as the frame position information of the file. The moving image number setting unit 112 sets moving image number 3. The reference moving image number setting unit 117 sets 3 as the reference moving image number. The management information multiplexing unit 116 multiplexes the frame position information, the moving image number, and the reference moving image number.

  Subsequently, the data switching unit 110 selects the moving image B. The moving image B encoded data input unit 102 inputs encoded data for one frame, and the moving image B frame position information input unit 107 inputs frame position information of the next frame. The encoded data multiplexing unit 111 multiplexes encoded data. The frame head position information setting unit 114 adds the input frame position information to the file size and sets it as the frame position information of the file. The moving image number setting unit 112 sets moving image number 1. The reference moving image number setting unit 117 sets 0, 3, and 1 as reference moving image numbers. The management information multiplexing unit 116 multiplexes the frame position information, the moving image number, and the reference moving image number.

  Subsequently, the data switching unit 110 selects the moving image C. The moving image C encoded data input unit 103 inputs encoded data for one frame, and the moving image C frame position information input unit 108 inputs the frame position information of the next frame. The encoded data multiplexing unit 111 multiplexes encoded data. The frame head position information setting unit 114 adds the input frame position information to the file size and sets it as the frame position information of the file. The moving image number setting unit 112 sets moving image number 2. The reference moving image number setting unit 117 sets 0, 3, and 2 as reference moving image numbers. The management information multiplexing unit 116 multiplexes the frame position information, the moving image number, and the reference moving image number.

  The above processing of the moving images A, D, B, and C is further executed for one frame. Finally, the file multiplexing unit 115 multiplexes the encoded data obtained by the encoded data multiplexing unit 111 and the management information obtained by the management information multiplexing unit 116 to create a file. By the above processing, it can be multiplexed into a file.

  In the present embodiment, an example is shown in which a file is created by multiplexing from encoded data obtained by encoding in advance and file head position information, but a file may be configured while generating encoded data.

  Next, the operation of a file reading method for extracting encoded data of a specific moving image from the multiplexed file will be described. FIG. 6 shows the configuration of the file reading apparatus. 6 includes a file input unit 201 that reads a file, a management information reading unit 202 that reads management information, a moving image number search unit 203 that searches for a desired moving image number from the management information, and a management The reference moving image number search unit 208 that searches for management information of the moving image specified by the reference moving image number from the information, the frame head position information setting unit 205 that reads the frame head position information from the management information, and the frame head position information An encoded data reading unit 206 that reads encoded data from a file.

  The operation when reading the encoded data necessary for decoding the moving image D from the file created by the multiplexing device 11 of FIG. 5 will be described.

  The file input unit 201 inputs a file. The management information reading unit 202 reads management information in the second half of the file. The moving image number search unit 203 searches for management information of the first frame of the moving image D. The reference moving image number search unit 208 searches for management information regarding the moving image number specified by the reference moving image number. In this case, management information relating to the moving image numbers 2 and 3 is searched. The frame head position information setting unit 205 sets the frame head position information of the moving image number 2. The encoded data reading unit 206 reads encoded data from the set file position. At this time, the frame head position information setting unit 205 reads up to the head position of the next frame. Next, the frame head position information setting unit 205 sets the frame head position information of the moving image number 3. The encoded data reading unit 206 reads encoded data from the set file position. At this time, the frame head position information setting unit 205 reads up to the head position of the next frame.

  The encoded data is read in the same manner for the second frame.

  Next, for the third frame, the moving image number search unit 203 searches the management information of the third frame of the moving image D. The reference moving image number search unit 208 searches for management information regarding the moving image number specified by the reference moving image number. In this case, the management information relating to the moving image number 3 is searched. The frame head position information setting unit 205 sets the frame head position information of the moving image number 3. The encoded data reading unit 206 reads encoded data from the set file position. At this time, the frame head position information setting unit 205 reads up to the head position of the next frame.

  The encoded data is read in the same manner for the fourth frame. Thus, encoded data that can decode the moving image D can be extracted.

  As described above, according to this embodiment, it is possible to extract encoded data of a desired moving image without decoding the reference relation data included in the encoded data.

  In the multiplexing apparatus 11 of this embodiment, the management information is set after the encoded data. However, when the track structure is set in the file, the management information is set in the first track, and the encoded data is set. May be set as the second track.

  FIG. 7 shows a configuration example of tracks and management information in a file. The created file includes an encoded data track 30 and a management information track 40. In the encoded data track 30, moving image A encoded data 31, moving image B encoded data 32,... Are stored. . Further, management information 41, 42, 43,... For each time Tn is stored in the management information track 40, and information such as frame head position information, moving image number, reference moving image number, etc. is stored in each of them. Is done.

  In this embodiment, even in the case of moving images that can be decoded independently, the own moving image number is set as the reference moving image number. However, in this case, a method that does not set the reference moving image number or the reference moving image A method of setting a predetermined special value as the number is also suitable.

  In this embodiment, for the frame of the moving image designated by the reference moving image number, the frame start position information of the reference moving image is read by further searching the management information for the frame. It may be multiplexed including the frame head position information of the moving image specified by the image number.

  Next, as a third embodiment, an example in which the position information at the beginning of the frame, the moving image number, and the usage destination moving image number are multiplexed as management information and all the encoded data is set at the end of multiplexing is shown. As in the first embodiment, it is assumed that there are four moving images A, B, C, and D, which are encoded in advance, and the position information of the frame head in each moving image is known.

  FIG. 8 shows the configuration of the multiplexing apparatus. The multiplexing apparatus 12 shown in FIG. 8 includes a moving image A encoded data input unit 101, a moving image B encoded data input unit 102, and a moving image C that input encoded data of moving images A, B, C, and D, respectively. Encoded data input unit 103, moving image D encoded data input unit 104, moving image A frame position information input unit 106 for inputting position information in the moving image at the head of each moving image, moving image B frame position An information input unit 107, a moving image C frame position information input unit 108, a moving image D frame position information input unit 109, a reference relationship data input unit 105 for inputting information indicating a reference relationship between moving images, and a moving image code Data switching unit 110 that switches input of encoded data and frame position information, encoded data multiplexing unit 111 that multiplexes encoded data, and moving image number that sets a moving image number A determining unit 112, a used moving image number determining unit 118 for determining a used moving image number, a used moving image number setting unit 119 for setting a used moving image number, and a frame for setting head information of a frame position A head position information setting unit 114, a management information multiplexing unit 116 that multiplexes management information, and a file multiplexing unit 115 that multiplexes encoded data and management information.

  Among the above, the moving image number setting unit 112 sets moving image numbers 0, 1, 2, and 3 for moving images A, B, C, and D, respectively. Frame position information input units 106 to 109 for moving images A to D are assumed to input position information of the next frame. The frame head position information setting unit 114 counts the frame position information of each moving image, and sets the position information of each frame after the encoded data multiplexing unit 111 multiplexes the encoded data. The file multiplexing unit 115 sets the management information created by the management information multiplexing unit 116 after the coded data created by the coded data multiplexing unit 111. When there is reference relation data, the encoded data multiplexing unit 111 multiplexes the encoded data before the encoded data of each moving image.

  The data switching unit 110 switches between encoded data and file head position information as follows according to the reference relation data. In order from the moving images A to D, it is first determined whether the moving images can be decoded independently. If it can be decoded independently, select it as it is. If it cannot be decoded independently, it is determined whether the encoded data of the moving image to be referred to has already been selected. If it has already been selected, it is selected as it is. If it has not been selected yet, the moving image is not selected until all the moving images to be referenced are selected.

  Assume that the reference relationship between moving images is as follows. First, in the first two frames, moving images A and C can be decoded independently, moving image B refers to moving image A, and moving image D refers to moving image C. Thereafter, in two frames, moving images A and D can be decoded independently, and moving images B and C refer to moving images A and D.

  The use destination moving image number determination unit 118 determines, for each moving image, a moving image number for which the decoded image or encoded data is used from the reference relation data. From the reference relationship between the moving images, for the first two frames, the use destination moving image of the moving image A is the moving image A and the moving image B, the use destination moving image of the moving image B is the moving image B, and the moving image The usage destination moving image of the image C is the moving image C and the moving image D, and the usage destination moving image of the moving image D is determined to be the moving image D. After that, for two frames, the use destination moving image of the moving image A is the moving image A, the moving image B, and the moving image C, the use destination moving image of the moving image B is the moving image B, and the use destination of the moving image C is It is determined that the moving image is the moving image C, and the usage destination moving images of the moving image D are the moving image D, the moving image B, and the moving image C.

  When the moving images A, B, C, and D are multiplexed in four frames based on the above premise, they are multiplexed as follows.

  First, the first frame is multiplexed as follows. The reference relationship data input unit 105 inputs reference relationship data between moving images. The data switching unit 110 determines the order in which the encoded data is multiplexed according to the reference relation data. Here, the order is determined as follows. Since the moving image A can be decoded independently, the moving image A is first selected. Since the moving image B refers only to the moving image A, the moving image B is selected next. Since the moving image C can be decoded independently, the moving image C is selected next. Then, the moving image D is selected. The encoded data multiplexing unit 111 multiplexes the reference relation data. The frame head position information setting unit 114 adds the file size by the data size of the reference relation data. The use destination moving image number determination unit 118 determines the use destination moving image number corresponding to the reference data for each moving image.

  In accordance with the order of multiplexing, the data switching unit 110 switches between encoded data and frame head position information. First, the moving image A is selected. The moving image A encoded data input unit 101 inputs encoded data for one frame, and the moving image A frame position information input unit 106 inputs frame position information of the next frame. The encoded data multiplexing unit 111 multiplexes encoded data. The frame head position information setting unit 114 adds the input frame position information to the file size and sets it as the frame position information of the file. The moving image number setting unit 112 sets moving image number 0. The usage destination moving image number setting unit 119 sets 0 and 1 as the usage destination moving image numbers. The management information multiplexing unit 116 multiplexes the frame position information, the moving image number, and the usage destination moving image number.

  Subsequently, the data switching unit 110 selects the moving image B. The moving image B encoded data input unit 102 inputs encoded data for one frame, and the moving image B frame position information input unit 107 inputs frame position information of the next frame. The encoded data multiplexing unit 111 multiplexes encoded data. The frame head position information setting unit 114 adds the input frame position information to the file size and sets it as the frame position information of the file. The moving image number setting unit 112 sets moving image number 1. The usage destination moving image number setting unit 119 sets 1 as the usage destination moving image number. The management information multiplexing unit 116 multiplexes the frame position information, the moving image number, and the usage destination moving image number.

  Subsequently, the data switching unit 110 selects the moving image C. The moving image C encoded data input unit 103 inputs encoded data for one frame, and the moving image C frame position information input unit 108 inputs the frame position information of the next frame. The encoded data multiplexing unit 111 multiplexes encoded data. The frame head position information setting unit 114 adds the input frame position information to the file size and sets it as the frame position information of the file. The moving image number setting unit 112 sets moving image number 2. The usage destination moving image number setting unit 119 sets 2 and 3 as the usage destination moving image numbers. The management information multiplexing unit 116 multiplexes the frame position information, the moving image number, and the usage destination moving image number.

  Subsequently, the data switching unit 110 selects the moving image D. The moving image D encoded data input unit 104 inputs encoded data for one frame, and the moving image D frame position information input unit 109 inputs frame position information of the next frame. The encoded data multiplexing unit 111 multiplexes encoded data. The frame head position information setting unit 114 adds the input frame position information to the file size and sets it as the frame position information of the file. The moving image number setting unit 112 sets moving image number 3. The use destination moving image number setting unit 119 sets 3 as the use destination moving image number. The management information multiplexing unit 116 multiplexes the frame position information, the moving image number, and the usage destination moving image number.

  The above processing of the moving images A, B, C, and D is further performed for one frame.

  Next, the reference relationship data input unit 105 inputs reference relationship data between moving images. The data switching unit 110 determines the order in which the encoded data is multiplexed according to the reference relation data. Here, the order is determined as follows. Since the moving image A can be decoded independently, the moving image A is first selected. Since the moving image B refers to the moving image A and the moving image D, the moving image B is not selected. Since the moving image C refers to the moving image A and the moving image D, the moving image C is not selected. Since the moving image D can be decoded independently, the moving image D is selected next. Then, the moving image B is selected, and the moving image C is selected. The encoded data multiplexing unit 111 multiplexes the reference relation data. The frame head position information setting unit 114 adds the file size by the data size of the reference relation data. The use destination moving image number determination unit 118 determines the use destination moving image number corresponding to the reference data for each moving image.

  In accordance with the order of multiplexing, the data switching unit 110 switches between encoded data and frame head position information. First, the moving image A is selected. The moving image A encoded data input unit 101 inputs encoded data for one frame, and the moving image A frame position information input unit 106 inputs frame position information of the next frame. The encoded data multiplexing unit 111 multiplexes encoded data. The frame head position information setting unit 114 adds the input frame position information to the file size and sets it as the frame position information of the file. The moving image number setting unit 112 sets moving image number 0. The use destination moving image number setting unit 119 sets 0, 1 and 2 as reference moving image numbers. The management information multiplexing unit 116 multiplexes the frame position information, the moving image number, and the usage destination moving image number.

  Subsequently, the data switching unit 110 selects the moving image D. The moving image D encoded data input unit 104 inputs encoded data for one frame, and the moving image D frame position information input unit 109 inputs frame position information of the next frame. The encoded data multiplexing unit 111 multiplexes encoded data. The frame head position information setting unit 114 adds the input frame position information to the file size and sets it as the frame position information of the file. The moving image number setting unit 112 sets moving image number 3. The use destination moving image number setting unit 119 sets 3, 1 and 2 as the use destination moving image numbers. The management information multiplexing unit 116 multiplexes the frame position information, the moving image number, and the usage destination moving image number.

  Subsequently, the data switching unit 110 selects the moving image B. The moving image B encoded data input unit 102 inputs encoded data for one frame, and the moving image B frame position information input unit 107 inputs frame position information of the next frame. The encoded data multiplexing unit 111 multiplexes encoded data. The frame head position information setting unit 114 adds the input frame position information to the file size and sets it as the frame position information of the file. The moving image number setting unit 112 sets moving image number 1. The usage destination moving image number setting unit 119 sets 1 as the usage destination moving image number. The management information multiplexing unit 116 multiplexes the frame position information, the moving image number, and the usage destination moving image number.

  Subsequently, the data switching unit 110 selects the moving image C. The moving image C encoded data input unit 103 inputs encoded data for one frame, and the moving image C frame position information input unit 108 inputs the frame position information of the next frame. The encoded data multiplexing unit 111 multiplexes encoded data. The frame head position information setting unit 114 adds the input frame position information to the file size and sets it as the frame position information of the file. The moving image number setting unit 112 sets moving image number 2. The use destination moving image number setting unit 119 sets 2 as the use destination moving image number. The management information multiplexing unit 116 multiplexes the frame position information, the moving image number, and the usage destination moving image number.

  The above processing of the moving images A, D, B, and C is further executed for one frame. Finally, the file multiplexing unit 115 multiplexes the encoded data obtained by the encoded data multiplexing unit 111 and the management information obtained by the management information multiplexing unit 116 to create a file. By the above processing, it can be multiplexed into a file.

  In the present embodiment, an example is shown in which a file is created by multiplexing from encoded data obtained by encoding in advance and file head position information, but a file may be configured while creating encoded data.

  Next, the operation of a file reading method for extracting encoded data of a specific moving image from the multiplexed file will be described. FIG. 9 shows the configuration of the file reading apparatus. The file reading device 23 shown in FIG. 9 includes a file input unit 201 that reads a file, a management information reading unit 202 that reads management information, and a usage destination that searches a usage destination moving image number corresponding to a desired moving image from the management information. A moving image number search unit 209, a frame head position information setting unit 205 that reads frame head position information from management information, and an encoded data reading unit 206 that reads encoded data from a file according to the frame head position information.

  The operation when reading the encoded data necessary for decoding the moving picture D from the file created by the multiplexing device 12 of FIG. 8 will be described.

  The file input unit 201 inputs a file. The management information reading unit 202 reads management information in the second half of the file. The use destination moving image number search unit 209 searches for the moving image having the use destination moving image number 3 from the management information of the first frame in order from the management information of the moving image A. The management information of the moving image C (moving image number 2) is searched for that the usage destination moving image number is 3. The frame head position information setting unit 205 sets the frame head position information of the moving image number 2. The encoded data reading unit 206 reads encoded data from the set file position. At this time, the frame head position information setting unit 205 reads up to the head position of the next frame. Subsequently, the use destination moving image number search unit 209 searches the management information of the moving image D (moving image number 3) that the use destination moving image number is 3. The frame head position information setting unit 205 sets the frame head position information of the moving image number 3. The encoded data reading unit 206 reads encoded data from the set file position. At this time, the frame head position information setting unit 205 reads up to the head position of the next frame.

  The encoded data is read in the same manner for the second frame.

  Next, for the third frame, the use destination moving image number search unit 209 searches the management information of the moving image A for moving images having 3 as the use destination moving image number from the management information of the third frame. The management information of the moving image D (moving image number 3) is searched for that the usage destination moving image number is 3. The frame head position information setting unit 205 sets the frame head position information of the moving image number 3. The encoded data reading unit 206 reads encoded data from the set file position. At this time, the frame head position information setting unit 205 reads up to the head position of the next frame.

  The encoded data is read in the same manner for the fourth frame. Thus, encoded data that can decode the moving image D can be extracted.

  As described above, according to this embodiment, it is possible to extract encoded data of a desired moving image without decoding the reference relation data included in the encoded data.

  In the multiplexing device 12 of this embodiment, management information is set after the encoded data. However, when the track structure is set in the file, the management information is set in the first track and the encoding is performed. Data may be set in the second track.

  FIG. 10 shows a configuration example of tracks and management information in a file. The created file includes an encoded data track 30 and a management information track 40. In the encoded data track 30, moving image A encoded data 31, moving image B encoded data 32,... Are stored. . Further, management information 41, 42, 43,... For each time Tn is stored in the management information track 40, and information such as frame head position information, moving picture number, use destination moving picture number, etc. is stored in each of them. Stored.

  In this embodiment, the user's moving image number is also set as the use destination moving image number. However, a method in which the user's moving image number is not set as the use destination moving image number, or a special method predetermined as the use destination moving image number. A method of setting the value is also suitable.

  In the above description of the embodiment, an example in which management information is set in units of frames has been described. However, management information may be set in units of slices obtained by dividing a frame. In this case, the management information includes slice head position information instead of the frame head position. In addition, multiplexing and reading are performed not on a frame basis but on a slice basis. That is, in the description of the above embodiment, the present invention can be implemented by replacing a frame with a slice.

  The above processing by the multiplexing device and the file reading device can be realized by a computer and a software program, and the program can be provided by being recorded on a computer-readable recording medium or provided through a network. It is.

It is a figure which shows the structure of the multiplexing apparatus in 1st Example. It is a figure which shows the structure 1 of the file reading apparatus in a 1st Example. It is a figure which shows the structure 2 of the file reading apparatus in a 1st Example. It is a figure which shows the structural example of the track | truck in the file of a 1st Example, and management information. It is a figure which shows the structure of the multiplexing apparatus in a 2nd Example. It is a figure which shows the structure of the file reading apparatus in a 2nd Example. It is a figure which shows the structural example of the track | truck in the file of 2nd Example, and management information. It is a figure which shows the structure of the multiplexing apparatus in a 3rd Example. It is a figure which shows the structure of the file reading apparatus in a 3rd Example. It is a figure which shows the structural example of the track | truck and management information in the file of 3rd Example. It is a figure which shows the concept of scalable encoding. It is a figure which shows the example of the reference relationship of MVC. It is a figure which shows the structural example of the encoding data of MVC. It is a figure which shows the structural example of the track | truck and management information in a file.

Explanation of symbols

10, 11, 12 Multiplexer 101 Moving image A encoded data input unit 102 Moving image B encoded data input unit 103 Moving image C encoded data input unit 104 Moving image D encoded data input unit 105 Reference relation data input unit 106 moving image A frame position information input unit 107 moving image B frame position information input unit 108 moving image C frame position information input unit 109 moving image D frame position information input unit 110 data switching unit 111 encoded data multiplexing unit 112 moving image Number setting unit 113 Independent decoding identification information setting unit 114 Frame head position information setting unit 115 File multiplexing unit 116 Management information multiplexing unit 117 Reference moving image number setting unit 118 Usage destination moving image number determination unit 119 Usage destination moving image number setting Section 20, 21, 22, 23 File reading device 201 File input Power unit 202 Management information reading unit 203 Moving image number searching unit 204 Independent decoding identification information determining unit 205 Frame head position information setting unit 206 Encoded data reading unit 207 Independent decoding identification information searching unit 208 Reference moving image number searching unit 209 Usage destination Video number search section

Claims (10)

A multiplexing method for multiplexing encoded data created by predictive encoding between n (n ≧ 2) moving images to form a file,
For each moving image frame or slice time , position information in the moving image frame or slice head file, a moving image number for distinguishing moving images, and other moving images when decoding the frame or slice. A process of multiplexing management information for n moving images including independent decoding identification information indicating whether decoded image information of a frame or a slice is necessary;
A process of configuring the multiplexed information at a specific position different from the encoded data to be decoded of each moving image frame or slice, and configuring a file;
A method for multiplexing moving images, comprising: A multiplexing method for multiplexing encoded data created by predictive encoding between n (n ≧ 2) moving images to form a file,
Position information in the file at the beginning of a moving image frame or slice, a moving image number for distinguishing moving images, and whether decoding image information of other moving image frames or slices is necessary when decoding the frame or slice Management information for n moving images including independent decoding identification information indicating
The reference relationship information in the predictive encoding between the encoded data of each frame or slice and the moving image is used as the second track.
A method for multiplexing moving images, wherein the first track and the second track are multiplexed independently to form a file. A multiplexing device that multiplexes encoded data created by predictive encoding between n (n ≧ 2) moving images to form a file,
For each moving image frame or slice time , position information in the moving image frame or slice head file, a moving image number for distinguishing moving images, and other moving images when decoding the frame or slice. A management information multiplexing unit that multiplexes management information for n moving images including independent decoding identification information indicating whether decoded image information of a frame or a slice is necessary;
A file multiplexing unit that configures a file by setting the management information created by the management information multiplexing unit at a specific position different from the encoded data to be decoded for each moving image frame or slice;
An apparatus for multiplexing moving images, comprising: A multiplexing device that multiplexes encoded data created by predictive encoding between n (n ≧ 2) moving images to form a file,
For each moving image frame or slice time , position information in the moving image frame or slice head file, a moving image number for distinguishing moving images, and other moving images when decoding the frame or slice. A management information multiplexing unit that multiplexes management information for n moving images including independent decoding identification information indicating whether decoded image information of a frame or a slice is necessary, as a first track;
An encoded data multiplexing unit that multiplexes the reference relationship information in the predictive encoding between the encoded data of each frame or slice and the moving image as a second track;
A file multiplexing unit configured to multiplex the first track and the second track to form a file;
An apparatus for multiplexing moving images, comprising: A multiplexing method for multiplexing encoded data created by predictive encoding between n (n ≧ 2) moving images to form a file,
For each moving image frame or slice time , position information in the moving image frame or slice head file, a moving image number for distinguishing the moving image, and a moving image necessary for decoding the frame or slice. A process of multiplexing management information for n moving images including a reference moving image number that is a moving image number;
A process of configuring the multiplexed information at a specific position different from the encoded data to be decoded of each moving image frame or slice, and configuring a file;
A method for multiplexing moving images, comprising: A multiplexing method for multiplexing encoded data created by predictive encoding between n (n ≧ 2) moving images to form a file,
For each moving image frame or slice time , position information in the moving image frame or slice head file, a moving image number for distinguishing the moving image, and a moving image necessary for decoding the frame or slice. Management information for n moving images including a reference moving image number that is a moving image number is set as a first track,
The reference relationship information in the predictive encoding between the encoded data of each frame or slice and the moving image is used as the second track.
A method for multiplexing moving images, wherein the first track and the second track are multiplexed independently to form a file. A multiplexing device that multiplexes encoded data created by predictive encoding between n (n ≧ 2) moving images to form a file,
For each moving image frame or slice time , position information in the moving image frame or slice head file, a moving image number for distinguishing the moving image, and a moving image necessary for decoding the frame or slice. A management information multiplexing unit that multiplexes management information for n moving images including a reference moving image number that is a moving image number;
A file multiplexing unit that configures a file by setting the management information created by the management information multiplexing unit at a specific position different from the encoded data to be decoded for each moving image frame or slice;
An apparatus for multiplexing moving images, comprising: A multiplexing device that multiplexes encoded data created by predictive encoding between n (n ≧ 2) moving images to form a file,
For each moving image frame or slice time , position information in the moving image frame or slice head file, a moving image number for distinguishing the moving image, and a moving image necessary for decoding the frame or slice. A management information multiplexing unit that multiplexes management information for n moving images including a reference moving image number that is a moving image number as a first track;
An encoded data multiplexing unit that multiplexes the reference relationship information in the predictive encoding between the encoded data of each frame or slice and the moving image as a second track;
A file multiplexing unit configured to multiplex the first track and the second track to form a file;
An apparatus for multiplexing moving images, comprising: A moving picture multiplexing program for causing a computer to execute the moving picture multiplexing method according to claim 1, claim 2, claim 5 or claim 6 . A computer-readable recording medium in which the moving picture multiplexing program according to claim 9 is recorded.

Images