JP2012100341A - Reproducing device, and reproducing method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To easily realize a 3D display of a sub-image based on offset information, when the offset information of the sub-image is disposed in a stream of a main image.SOLUTION: A video stream for a right eye comprises offset metadata for generating offset information. Only either coded video data of a main image for the right eye or the offset metadata are disposed in a data portion of a TS packet of the video stream for the right eye. In a header of the TS packet of the video stream for the right eye, a transport priority is described, and the transport priority indicates that data disposed in the data portion of the TS packet are the coded video data of the main image for the right eye or the offset metadata. The present invention is applicable, for example, to reproducing devices that display sub-images, in a three-dimensional way.

Description

  The present invention relates to a playback apparatus and a playback method, and in particular, when sub-image offset information is arranged in a main image stream, 3D display of the sub-image based on the offset information can be easily realized. The present invention relates to a playback apparatus and a playback method.

  2D images are the mainstream content for movies and the like, but recently, 3D images have attracted attention.

  As a method for showing a 3D image to the user, there is a method in which one of two images separated by a predetermined distance in a predetermined direction is displayed on the user's left eye and the other image is displayed on the right eye. is there.

  In this method, in order to display a 3D image, both a left-eye image and a right-eye image are prepared, or a predetermined image and the shift direction of the left-eye and right-eye images with respect to the predetermined image are displayed. It is necessary to prepare direction information representing a combination and an offset value representing a deviation amount. Hereinafter, the direction information and the offset value are collectively referred to as offset information.

  Further, as an image encoding method used when both a left-eye image and a right-eye image are prepared, an MVC (Multi-view Video coding) method (for example, see Patent Document 1) can be considered.

  Therefore, in order to display a sub-image (eg, subtitles, menu buttons, etc.) displayed together with a main image such as a movie in 3D, offset information of the sub-image is arranged in the main image stream encoded by the MVC method or the like. It is possible.

JP 2009-100070 A

  However, in this case, if the main image stream is converted to a TS (Transport Stream) packet without any contrivance, the main image stream cannot be decoded by a general-purpose decoder. That is, since it is necessary to extract and decode the video data and offset information of the main image from the TS packet of the main image, it is necessary to prepare a dedicated decoder. Therefore, the development cost is high, and 3D display of the sub-image cannot be easily realized.

  The present invention has been made in view of such a situation, and when the offset information of the sub-image is arranged in the stream of the main image, 3D display of the sub-image based on the offset information is easily realized. Is to be able to.

  A playback device according to an aspect of the present invention includes a sub-image stream packet that is a stream of a predetermined sub-image, and a main image stream that is a sub-image for the left eye that is used for 3D display of the predetermined sub-image. And a video stream packet that is a stream including offset information including the shift direction and shift amount of the right-eye sub-image, and the data portion of the video stream packet includes the video data of the main image or the offset information Only one of these is arranged, and the header of the packet of the video stream indicates whether the data arranged in the data part of the packet is the video data of the main image or the offset information. When reproducing data having a data structure in which flag information, which is information to be displayed, is reproduced, A read unit for reading a packet of a stream and the video stream, and the data unit of the packet indicating that the flag information described in the header of the packet read by the read unit is the offset information A generating unit that generates video data of the sub-image for the left eye and video data of the sub-image for the right eye from the packet of the sub-image stream based on the offset information arranged in .

  The reproduction method according to one aspect of the present invention corresponds to the reproduction apparatus according to one aspect of the present invention.

  In one aspect of the present invention, a sub-image stream packet, which is a predetermined sub-image stream, and a main image stream, the left-eye sub-image and the right-eye used for 3D display of the predetermined sub-image. A video stream packet that is a stream including offset information including a shift direction and a shift amount of a sub-image for the image, and the data portion of the video stream packet includes either the video data of the main image or the offset information. Only one of them is arranged, and in the header of the packet of the video stream, information indicating whether the data arranged in the data portion of the packet is the video data of the main image or the offset information When reproducing data having a data structure in which flag information is described, the sub-image The offset information arranged in the data portion of the packet indicating that the packet of the video stream and the video stream is read and the flag information described in the header of the packet is the offset information The sub-image video data for the left eye and the sub-image video data for the right eye are generated from the packet of the sub-image stream.

  As described above, according to the present invention, when sub-image offset information is arranged in the main image stream, 3D display of the sub-image based on the offset information can be easily realized.

It is a block diagram which shows the structural example of one Embodiment of the reproducing | regenerating apparatus to which this invention is applied. It is a figure which shows the structural example of the first dependent unit in the display order in each GOP of the video stream for right eyes. It is a figure explaining the production | generation method of TS packet of the video stream for right eyes. It is a figure which shows the detailed structural example of TS packet. It is a block diagram which shows the detailed structural example of the video production | generation part for right eyes, and an offset production | generation part. It is a flowchart explaining the offset production | generation process of a reproducing | regenerating apparatus. It is a flowchart of 3D display processing of the playback device when executing a movie object. It is a block diagram which shows the structural example of a recording device. It is a flowchart explaining the recording process of a recording device. It is a block diagram which shows the structural example of the hardware of a computer. It is a figure which shows the example of a description of offset metadata. It is a figure which shows the example of a description of offset metadata. It is a block diagram which shows the detailed structural example of the 3D display data generation part of FIG. It is a figure explaining the superimposition order in 3D display data.

<First embodiment>
[Configuration example of playback device]
FIG. 1 is a block diagram showing a configuration example of an embodiment of a playback apparatus to which the present invention is applied.

  The playback device 20 of FIG. 1 includes an input unit 21, a control unit 22, and a playback unit 23.

  The input unit 21 includes a keyboard, a mouse, a microphone, and the like. The input unit 21 receives a command from the user and supplies it to the control unit 22. The control unit 22 controls the reproduction unit 23 according to a command from the input unit 21.

  The playback unit 23 includes a drive 31, a read buffer 32, a PID filter 33, a transport priority filter 34, an offset generation unit 35, a right-eye video generation unit 36, a left-eye video generation unit 37, a subtitle / menu generation unit 38, and a 3D display. The data generation unit 39, the BD-J graphics generation unit 40, and the audio generation unit 41 are configured.

  The drive 31 drives the disc 11 made of a mounted BD (Blu-Ray (registered trademark)) or the like according to the control of the control unit 22. As a result, the drive 31 reads the index file, movie object file, BD-J object file, playlist file, clip information file, stream file, and the like recorded on the disc 11.

  The index file is a file in which a list of title numbers recorded on the disc 11 and the types and numbers of objects to be executed corresponding to the title numbers are described. There are two types of objects: movie objects and BD-J objects.

  A movie object file is a file in which a plurality of movie objects are described. A program called a navigation command is described in the movie object. In the following description, navigation commands are simply referred to as commands unless it is necessary to distinguish them.

  Furthermore, the BD-J object file is a file in which a plurality of BD-J applications are described. A playlist file is a file that is played back only by a movie object or a BD-J object, and describes information about an AV stream (details will be described later) played back by one command described in these objects.

  Specifically, the playlist file is composed of one or a plurality of play items. Each play item describes information specifying a clip information file corresponding to an AV stream to be played back and time information indicating a playback section of the AV stream.

  The AV stream is recorded on the disk 11 as a stream file. The AV stream is composed of a TS (Transport Stream) packet in which a left-eye video stream and a right-eye video stream, an audio stream corresponding to them, a subtitle stream, and a menu stream are multiplexed according to ISO13818-2.

  Note that the left-eye video stream is a stream of the main image for the left eye encoded by the MVC method. The right-eye video stream is a stream of the main image for the right eye encoded by the MVC method, and includes offset metadata (details will be described later). The subtitle stream is a stream including data in a bitmap format or text format for displaying 2D subtitles, and the menu stream is a stream including data for displaying 2D menu buttons.

  The offset metadata is data for generating offset information for each picture. Specifically, the offset metadata is composed of offset information, a PTS (Presentation Time Stamp) of the first picture in which the offset information is set, and a frame rate representing the interval between pictures in which the offset information is set. The The offset information is described for each subtitle, menu button, and BD-J graphics screen (plane) corresponding to the playback section for 1 GOP. In the BD-ROM (Blu-Ray Disc-Read Only Memory) standard, the menu button and BD-J graphics are in an exclusive relationship, so the offset information is applied to the subtitle screen and the menu button screen. Or, it is applied to the subtitle screen and BD-J graphics screen. Details of the description example of the offset metadata will be described with reference to FIGS. 11 and 12 described later.

  The clip information file is a file in which a map for associating time information described in the playlist file with the packet number of the AV stream is described. Therefore, the control unit 22 can recognize the packet number of the AV stream to be reproduced corresponding to each play item by referring to the clip information file. The stream file is an AV stream file.

  The drive 31 supplies the read index file, movie object file, BD-J object file, playlist file, clip information file, and the like to the control unit 22. The drive 31 supplies the read stream file AV stream to the read buffer 32.

  The read buffer 32 holds the AV stream supplied from the drive 31 or reads the held AV stream and supplies it to the PID filter 33 under the control of the control unit 22.

  Based on the packet ID (PID) of each TS packet of the AV stream from the read buffer 32, the PID filter 33 converts the left-eye video stream, right-eye video stream, subtitle stream, menu stream, and audio stream from the AV stream. Each TS packet is extracted. The PID is a unique ID for each type of data constituting the TS packet, and is described in the packet header.

  The PID filter 33 supplies the extracted TS packet of the right-eye video stream to the transport priority filter 34 and supplies the TS packet of the left-eye video stream to the left-eye video generation unit 37. Also, the PID filter 33 supplies the subtitle stream and menu stream TS packets to the subtitle / menu generation unit 38, and supplies the audio stream TS packets to the audio generation unit 41.

  Based on the transport priority described in the header of the TS packet of the right-eye video stream supplied from the PID filter 33, the transport priority filter 34 offsets a predetermined TS packet of the TS packets to the offset generation unit 35. To supply. Further, the transport priority filter 34 supplies the TS packet of the right-eye video stream supplied from the PID filter 33 to the right-eye video generation unit 36.

  The offset generation unit 35 generates offset information based on the offset metadata arranged in the data part of the TS packet supplied from the transport priority filter 34 and supplies the offset information to the 3D display data generation unit 39.

  The right-eye video generation unit 36 is based on the transport priority described in the header of the TS packet of the right-eye video stream supplied from the transport priority filter 34, and is used for the right-eye arranged in the data portion of the TS packet. The video data of the main image is decoded. The right-eye video generation unit 36 supplies the video data obtained as a result of decoding to the 3D display data generation unit 39 as right-eye video data.

  The left-eye video generation unit 37 decodes the video data of the left-eye main image included in the TS packet of the left-eye video stream supplied from the PID filter 33. The left-eye video generation unit 37 supplies the video data obtained as a result of decoding to the 3D display data generation unit 39 as left-eye video data.

  The caption / menu generation unit 38 includes a caption generation unit 51 and a menu generation unit 52. The caption generation unit 51 supplies caption data included in the TS packet of the caption stream supplied from the PID filter 33 to the 3D display data generation unit 39 when the movie object is executed.

  The menu generation unit 52 supplies menu data included in the TS packet of the menu stream supplied from the PID filter 33 to the 3D display data generation unit 39 when the movie object is executed.

  The 3D display data generation unit 39 sets the right-eye video data supplied from the right-eye video generation unit 36 and the left-eye video data supplied from the left-eye video generation unit 37 as 3D video data. Further, the 3D display data generation unit 39 sets the caption corresponding to the caption data supplied from the caption generation unit 51 in a predetermined offset direction by a predetermined offset value based on the offset information supplied from the offset generation unit 35. Subtitle data of subtitles that are shifted from each other is generated as subtitle data for the right eye and subtitle data for the left eye. Then, the 3D display data generation unit 39 sets the caption data for the right eye and the caption data for the left eye as 3D caption data for displaying the caption in 3D.

  In addition, the 3D display data generation unit 39 displays 3D menu buttons from the menu data supplied from the menu generation unit 52 based on the offset information supplied from the offset generation unit 35, similarly to the 3D subtitle data. 3D menu data is generated.

  Further, the 3D display data generation unit 39 synthesizes 3D video data, 3D caption data, and 3D menu data for each of the left and right eye data. Specifically, the 3D display data generation unit 39 generates display data for the left eye by combining the video data for the left eye, the caption data for the left eye, and the menu data for the left eye. The 3D display data generation unit 39 combines the right-eye video data, the right-eye caption data, and the right-eye menu data to generate right-eye display data.

  In addition, the 3D display data generation unit 39, based on the offset information from the offset generation unit 35, provides a BD-J including menu buttons and the like supplied from the BD-J graphics generation unit 40 in the same manner as the 3D subtitle data. 3D graphics data for 3D display of BD-J graphics is generated from graphics data for 2D display of graphics.

  The 3D display data generation unit 39 synthesizes 3D video data and 3D graphics data for each of the left and right eye data, uses the left eye synthesis result as the left eye display data, and the right eye synthesis result as the right eye. Display data. Then, the 3D display data generation unit 39 supplies the display data for the left eye and the display data for the right eye to the display unit 61 as 3D display data, and causes the display unit 61 to display the image for the left eye and the image for the right eye.

  The BD-J graphics generation unit 40 generates graphics data according to the control from the control unit 22 when the BD-J object is executed, and supplies the graphics data to the 3D display data generation unit 39.

  The audio generation unit 41 decodes the audio data included in the TS packet of the audio stream supplied from the PID filter 33 and supplies the audio data obtained as a result to the speaker 62.

  The display unit 61 is configured by a 3D display or the like. The display unit 61 displays an image for the left eye and an image for the right eye based on the 3D display data supplied from the 3D display data generation unit 39. As a result, the user can see the 3D image.

  The speaker 62 outputs sound corresponding to the audio data supplied from the audio generation unit 41.

[Example structure of video stream for right eye]
FIG. 2 is a diagram illustrating a configuration example of the first dependent unit (picture) in the display order in each GOP (Group of Picture) of the video stream for the right eye.

  As shown in FIG. 2, the first dependent unit in the display order within each GOP of the video stream for the right eye includes the dependent delimiter (Dependent delimiter), SPS (Sequence Parameter Set), and Subset PPS. (Picture Parameter Set), SEI (Supplemental Enhancement Information), and one or more slices are arranged.

  The dependent delimiter is a start code indicating the head of the dependent unit. The dependent delimiter includes, for example, information indicating the type of slice included in the dependent unit.

  The SPS is a header that includes information related to the entire sequence. The SPS includes, for example, information indicating the profile of the right-eye video stream and information indicating the level of the right-eye video stream. For example, the SPS includes information necessary for calculating a POC (Picture Order Count). POC is information indicating the display order of pictures.

  PPS is a header including information related to a picture. For example, the PPS includes information necessary for calculating the POC.

  SEI is information indicating additional information that is not essential for decoding of VCL (Video Coding Layer). SEI includes user SEI information (BD User data SEI messages in MVC scalable nesting SEI) and information other than that (Other SEI messages in MVC scalable nesting SEI). Can be classified. In the disk 11, offset metadata is described as at least user SEI information. A slice is video data of a main image for the right eye encoded by the MVC method, and is actual data of a picture.

  In addition, after one or more slices, filler data (Filler Data), an end of sequence (End of Sequence), and an end of stream (End of Stream) are arranged as necessary.

  Filler data is data added to adjust the data size. The end of sequence is information indicating the end of the sequence. The end of stream is information indicating the end of the right-eye video stream.

  FIG. 3 is a diagram for explaining a method of generating a TS packet of the right-eye video stream.

  As shown in FIG. 3, the right-eye video stream is TS-packed so that the offset metadata is not arranged in the TS packet together with other data.

Specifically, as shown in FIG. 3, if the last data size of the data of the TS packet TS _n of PPS is smaller than the data size of the data portion of the TS packet, the data size of the TS packet TS _n is TS packet Arbitrary stuffing bytes are inserted into the header using the adaptation field of the header of the TS packet TS _n so as to be the same as the data size. Thus, the top of the offset metadata, the TS packet TS _n where PPS immediately before are disposed are packetized into different TS packets TS n _{+ 1.} TS _i represents the i-th TS packet.

Also, as shown in FIG. 3, when the data size of the data for the last TS packet TS _m of the offset metadata is smaller than the data size of the data portion of the TS packet, the data size of the TS packet TS _{m is} the data of the TS packet. Arbitrary stuffing bytes are inserted into the header using the adaptation field of the header of the TS packet TS _m so as to be the same as the size. Thus, the head of the slice, the TS packet TS _m immediately before the offset metadata is arranged is packetized into another TS packet TS m _{+ 1.}

  Further, as shown in FIG. 3, 0 is described as the transport priority in the header of the TS packet of data other than the offset metadata. Also, 1 is described as the transport priority in the header of the TS packet of the offset metadata.

  Specifically, as shown in FIG. 4, the TS packet is composed of a head header and subsequent data portions.

  As shown in FIG. 4, an 8-bit synchronization byte (sync_byte), a 1-bit transport error indicator (transport_error_indicator), and a 1-bit payload unit start indicator (payload_unit_start_indicator) are described in the header. The header describes 1-bit transport priority (transport_priority), 13-bit PID, and 2-bit transport scramble control (transport_scrambling_control). Further, 2-bit adaptation field control (adaptation_field_control) and 2-bit cyclic counter (continuity_counter) are described in the header. The cyclic counter is data that is incremented by 1 for each frame.

  When the adaptation field control is a predetermined value (“10” and “11” in the example of FIG. 4), an adaptation field as an extension header is further arranged in the header.

  When the adaptation field control is a predetermined value (“01”, “11” in the example of FIG. 4), the payload is stored in the data portion. For example, a right-eye video stream is stored as the payload.

  In the TS packet of the right-eye video stream, when offset metadata is described as the payload, 1 is described as the transport priority of the header of the TS packet. On the other hand, when data other than offset metadata is described as the payload, 0 is described as the transport priority of the header of the TS packet including the payload.

  In other words, the transport priority is used as flag information indicating whether the data arranged in the data part is offset metadata or data other than offset metadata.

  Therefore, the playback device 20 can extract TS packets including offset metadata from the TS packets of the right-eye video stream based on the transport priority.

[Detailed configuration example of the right eye video generator and offset generator]
FIG. 5 is a block diagram illustrating a detailed configuration example of the right-eye video generation unit 36 and the offset generation unit 35 in FIG. 1.

  As shown in FIG. 5, the PID filter 33 receives a TS packet including a header in which 0 or 1 is described as a transport priority. In the example of FIG. 5, the nth to n + 3th TS packets from the top are input to the PID filter 33. The transport priority of the nth and n + 3 TS packets is 0, and the transport priority of the (n + 1) th and n + 2th TS packets is 1.

  The PID filter 33 extracts the TS packet of the video stream for the right eye from the TS packet based on the PID described in the header of the input TS packet, and supplies it to the transport priority filter 34.

  The transport priority filter 34 supplies all TS packets supplied from the PID filter 33 to the right-eye video generation unit 36 or supplies only TS packets having a transport priority of 0 to the right-eye video generation unit 36. .

  In addition, when the transport priority described in the header of the TS packet supplied from the PID filter 33 is 1, the transport priority filter 34 supplies the TS packet to the offset generation unit 35. Thereby, only the TS packet of the video stream for the right eye whose transport priority is 1, that is, the TS packet in which the offset metadata is arranged in the data part is supplied to the offset generation unit 35.

  The right-eye video generation unit 36 includes a transport buffer 71, an elementary buffer 72, and a video decoder 73.

  The transport buffer 71 stores TS packets supplied from the transport priority filter 34. The transport buffer 71 reads the stored TS packets at a predetermined bit rate and supplies the read TS packets to the elementary buffer 72. The elementary buffer 72 accumulates TS packets supplied from the transport buffer 71.

  The video decoder 73 is configured by a general-purpose MVC decoder. Based on the transport priority of the TS packet stored in the elementary buffer 72, the video decoder 73 is for the right eye arranged in the data part of the TS packet having a transport priority of 0 among the TS packets. The video data of the main image is extracted at a predetermined timing. Then, the video decoder 73 decodes the video data of the main image for the right eye using a method corresponding to the MVC method, and the resulting video data is converted into the 3D display data generation unit 39 (FIG. 1) as the right eye video data. Supply.

  The offset generation unit 35 includes an extra buffer 81 and an offset decoder 82.

  The extra buffer 81 stores TS packets having a transport priority of 1 supplied from the transport priority filter 34.

  The offset decoder 82 extracts the offset metadata arranged in the data part of the TS packet stored in the extra buffer 81 at a predetermined timing. The offset decoder 82 generates offset information based on the offset metadata and supplies the offset information to the 3D display data generation unit 39 (FIG. 1).

[Processing of playback device]
FIG. 6 is a flowchart for explaining the offset generation processing of the playback device 20. This offset generation process is started, for example, when a new TS packet is input to the PID filter.

  In step S11, the PID filter 33 determines whether the TS packet is a TS packet of the right-eye video stream based on the PID of the input TS packet. If it is determined in step S11 that the input TS packet is not a TS packet of the right-eye video stream, the process ends.

  On the other hand, when it is determined that the TS packet input in step S11 is the TS packet of the right-eye video stream, the PID filter 33 supplies the TS packet to the transport priority filter 34.

  In step S 12, the transport priority filter 34 determines whether the transport priority described in the header of the TS packet supplied from the PID filter 33 is 1. If it is determined in step S12 that the transport priority is not 1, that is, if the transport priority is 0, the process ends.

  On the other hand, when it is determined in step S12 that the transport priority is 1, the transport priority filter 34 supplies the TS packet supplied from the PID filter 33 to the transport buffer 71 (FIG. 5) and the extra buffer 81. .

  In step S <b> 13, the extra buffer 81 accumulates TS packets supplied from the transport priority filter 34.

  In step S <b> 14, the offset decoder 82 extracts the offset metadata arranged in the data portion of the TS packet stored in the extra buffer 81.

  In step S15, the offset decoder 82 generates offset information based on the offset metadata extracted in step S14, and supplies the offset information to the 3D display data generation unit 39. Then, the process ends.

  FIG. 7 is a flowchart of 3D display processing of the playback device 20 when a movie object is executed. This 3D display process is started when, for example, offset metadata, left-eye video data, right-eye video data, caption data, and menu data are input to the 3D display data generation unit 39.

  In step S <b> 31, the 3D display data generation unit 39 performs offset information input from the offset generation unit 35, right-eye video data input from the right-eye video generation unit 36, and left-eye video input from the left-eye video generation unit 37. Video data, caption data input from the caption generation unit 51, and menu data input from the menu generation unit 52 are acquired.

  In step S32, the 3D display data generation unit 39 generates 3D display data from the right-eye video data, left-eye video data, caption data, and menu data based on the offset information.

  In step S33, the 3D display data generation unit 39 displays a 3D image on the display unit 61 based on the 3D display data generated in step S32. Then, the process ends.

  The 3D display process at the time of executing the BD-J object is the same as the 3D display process of FIG. 7 except that the caption data and the menu data are replaced with the graphics data, and thus the description thereof is omitted.

[Configuration example of recording device]
FIG. 8 is a block diagram illustrating a configuration example of the recording apparatus 100 that records the right-eye video stream on the disk 11.

  8 includes a video acquisition unit 101, a video encoder 102, an offset acquisition unit 103, a buffer 104, a PES packetization unit 105, a TS packetization unit 106, a drive 107, and a control unit 108.

  The video acquisition unit 101 acquires video data of a main image for the right eye input from the outside and supplies the video data to the video encoder 102. The video encoder 102 encodes the video data supplied from the video acquisition unit 101 using the MVC method, and supplies an ES (Elementary Stream) obtained as a result to the buffer 104. The SEI of this ES is provided with an empty area for arranging offset metadata.

  The offset acquisition unit 103 acquires offset metadata input from the outside, and supplies the offset metadata to the buffer 104.

  The buffer 104 includes a video buffer 111 and an offset buffer 112. The video buffer 111 stores an ES of video data supplied from the video encoder 102. Based on the control from the control unit 108, the video buffer 111 reads the ES stored in itself, and arranges the offset metadata supplied from the offset buffer 112 in a free area provided in the SEI of the ES. To the PES packetization unit 105.

  The offset buffer 112 stores the offset metadata supplied from the offset acquisition unit 103. Based on the control from the control unit 108, the offset buffer 112 reads the offset metadata stored in itself and supplies it to the video buffer 111.

  The PES packetizing unit 105 packetizes the ES supplied from the video buffer 111 into a PES packet based on the control from the control unit 108.

  Based on the transport priority supplied from the control unit 108, the TS packetizing unit 106 packetizes the PES packet supplied from the PES packetizing unit 105 into a TS packet. Specifically, the TS packetizing unit 106 generates a TS packet that includes a header in which the transport priority supplied from the control unit 108 is described, and in which the PES packet is arranged in the data portion. In the TS packet header, a PID corresponding to the right-eye video stream is also described. The TS packetizing unit 106 supplies the generated TS packet to the drive 107.

  The drive 107 records the TS packet supplied from the TS packetizing unit 106 on the disk 11 based on the control from the control unit 108.

  The control unit 108 monitors the ES and offset metadata of the video data stored in the buffer 104. Further, the control unit 108 manages the PTS. Further, the control unit 108 controls each unit of the PES packetizing unit 105, the TS packetizing unit 106, and the drive 107.

  Specifically, for example, when the PES packetized video data is input to the TS packetizing unit 106, the control unit 108 supplies 0 as the transport priority to the TS packetizing unit 106. On the other hand, when the offset metadata converted into the PES packet is input to the TS packetizing unit 106, 1 is supplied to the TS packetizing unit 106 as the transport priority.

[Description of recording device processing]
FIG. 9 is a flowchart for explaining the recording process of the recording apparatus 100 of FIG. This recording process is started, for example, when a PES packet is input from the PES packetizer 105 to the TS packetizer 106.

  In step S51, the TS packetizing unit 106 determines whether a PES packet corresponding to the data size of the data portion of the TS packet has been supplied. If the PES packet corresponding to the data size of the data portion of the TS packet has not been supplied in step S51, the process proceeds to step S52.

  In step S52, the TS packetizing unit 106 determines whether or not the transport priority supplied from the control unit 108 has been changed. If it is determined in step S52 that the transport priority has not been changed, the process returns to step S51. Then, the processes of steps S51 and S52 are repeated until a PES packet corresponding to the data size of the data portion of the TS packet is supplied or the transport priority is changed.

  On the other hand, if it is determined in step S52 that the transport priority has been changed, the process proceeds to step S53. In step S53, the TS packetizing unit 106 adjusts the adaptation field so that the data size of the TS packet to be generated in which the PES packet from the PES packetizing unit 105 is arranged in the data unit is the same as the data size of the TS packet. Is used to insert stuffing data into the header of the TS packet to be generated. Specifically, the TS packetizing unit 106 inserts stuffing data corresponding to a value obtained by subtracting the data size of the PES packet from the data size of the data portion of the TS packet into the adaptation field of the header of the TS packet. Then, the process proceeds to step S54.

  If it is determined in step S51 that a PES packet corresponding to the data size of the data portion of the TS packet has been supplied, the process proceeds to step S54.

  In step S54, the TS packetizer 106 arranges the PES packet supplied from the PES packetizer 105 in the data portion of the TS packet.

  In step S55, the TS packetizing unit 106 determines whether or not the transport priority for the PES packet arranged in the data unit supplied from the control unit 108 is zero. If it is determined in step S55 that the transport priority is 0, the process proceeds to step S56.

  In step S56, the TS packetizer 106 adds a header in which 0 is described as the transport priority to the data part, and generates a TS packet. As a result, the video data of the main image for the right eye that is converted into a PES packet is arranged in the data portion, and a TS packet in which 0 is described as the transport priority of the header is generated. Then, the TS packetizing unit 106 supplies the generated TS packet to the drive 107, and the process proceeds to step S58.

  On the other hand, if it is determined in step S55 that the transport priority is 0, the process proceeds to step S57. In step S57, the TS packetizing unit 106 adds a header in which 1 is described as the transport priority to the data part to generate a TS packet. As a result, offset metadata converted into PES packets is arranged in the data part, and a TS packet in which 1 is described as the transport priority of the header is generated. Then, the TS packetizing unit 106 supplies the generated TS packet to the drive 107, and the process proceeds to step S58.

  In step S <b> 58, the drive 107 records the TS packet supplied from the TS packetizing unit 106 on the disk 11. Then, the process ends.

  In the above description, offset metadata is included in the right-eye video stream, but may be included in the left-eye video stream. Further, it may be included in both the right-eye video stream and the left-eye video stream.

  In the above description, encoding is performed such that the right-eye video stream is an MVC dependent stream and the left-eye video stream is an MVC base stream. However, the left-eye video stream is The encoding may be performed so that the video stream for the right eye becomes the base stream.

  The present invention can also be applied to a case where a video stream used for 2D display of a main image and offset information of the main image are recorded on a disc. In this case, for example, the video stream used for 2D display of the main image includes offset metadata of the sub image and offset information of the main image.

  At least a part of the series of processes described above can be executed by hardware or can be executed by software. When at least a part of the series of processes is executed by software, a program constituting the software is installed from a program recording medium into a computer incorporated in dedicated hardware or a general-purpose personal computer. .

  FIG. 10 is a block diagram illustrating a hardware configuration example of a computer that executes at least a part of the above-described series of processes using a program.

  A CPU (Central Processing Unit) 201, a ROM (Read Only Memory) 202, and a RAM (Random Access Memory) 203 are connected to each other via a bus 204.

  An input / output interface 205 is further connected to the bus 204. To the input / output interface 205, an input unit 206 such as a keyboard and a mouse and an output unit 207 such as a display and a speaker are connected. The bus 204 is connected to a storage unit 208 made up of a hard disk, a non-volatile memory, etc., a communication unit 209 made up of a network interface, etc., and a drive 210 that drives the removable media 211.

  In the computer configured as described above, for example, the CPU 201 loads the program stored in the storage unit 208 to the RAM 203 via the input / output interface 205 and the bus 204 and executes it, thereby executing the above-described series of processing. Is done.

  The program executed by the CPU 201 is recorded in the removable medium 211 or provided via a wired or wireless transmission medium such as a local area network, the Internet, or digital broadcasting, and is installed in the storage unit 208.

<Example of offset metadata description>
FIG. 11 and FIG. 12 are diagrams showing examples of description of offset metadata.

  As shown in FIG. 11, in the offset metadata, the frame rate (frame_rate) of the right-eye video stream including the offset metadata is described in 4 bits, and the first picture in the display order of the right-eye video stream is described. PTS (offset_start_PTS) is described by 33 (= 3 + 15 + 15) bits. The playback time of the screen to which the offset information is applied can be specified by this frame rate and PTS. Also, the offset metadata describes the number of GOP frames (number of frames) in which the offset metadata is included in the SEI in 8 bits. Furthermore, the offset metadata includes the number of types of offset information (number_of_PG_offset_sequences) of subtitles set corresponding to the GOP, and the number of types of offset information of menu buttons set corresponding to the GOP ( number_of_IG_offset_sequences) is described in 6 bits. Note that the type of offset information applied to the reproduction target is specified by a playlist or the like.

  Also, as shown in FIG. 11, the offset metadata describes subtitle offset information (PG_offset_sequence) for each type of subtitle offset information, and the menu button offset information for each type of menu button offset information. (IG_offset_sequence) is described.

  As offset information (PG_offset_sequence, IG_offset_sequence), as shown in FIGS. 12A and 12B, offset direction information (offset_direction_flag) is described in 1 bit, and offset value information (offset_value) is described in 7 bits. Is done.

  As information indicating the offset direction, for example, “0” is used when the offset direction is a direction in which the 3D image is projected to the near side (user side) with respect to the display surface, and the 3D image is retracted to the back side. If the direction is “1”, “1” is used. Further, the offset value is expressed by the number of pixels, for example.

  Further, as shown in FIG. 11, in the offset metadata, 1-bit marker bit (marker_bit) is arranged everywhere to prevent “0” from continuing eight times. Specifically, for example, in the MVC decoding, when “0” continues for eight times in the data to be decoded, the data is determined to be a start code. Therefore, the offset metadata includes “1” as a marker bit so that a part of the offset metadata does not represent a start code. In addition, an empty area (reserved_for_future_use) is also arranged in the offset metadata.

  In the example of FIG. 11, marker bits are arranged so that “0” does not continue eight times. However, marker bits are arranged so that part or all of offset metadata is not recognized as a predetermined code at the time of decoding. The marker bit arrangement method is not limited to the method shown in FIG.

<Detailed Configuration Example of 3D Display Data Generation Unit>
FIG. 13 is a block diagram illustrating a detailed configuration example of the 3D display data generation unit 39 in FIG. 1.

  The 3D display data generation unit 39 in FIG. 13 includes a subtitle plane 531, a menu plane 532, a left-eye display data generation unit 533, and a right-eye display data generation unit 534.

  The subtitle plane 531 holds subtitle data supplied from the subtitle generation unit 51 (FIG. 1).

  The menu plane 532 holds menu data supplied from the menu generation unit 52 (FIG. 1).

  The left-eye display data generation unit 533 includes a left-eye video plane 541, a transmission unit 542, an offset addition unit 543, a transmission unit 544, a synthesis unit 545, a transmission unit 546, an offset addition unit 547, a transmission unit 548, and a synthesis unit 549. Composed.

  The left-eye video plane 541 holds left-eye video data supplied from the left-eye video generation unit 37 (FIG. 1).

The transmission unit 542 reads the left-eye video data held in the left-eye video plane 541. The transmission unit 542 converts the read video data for the left eye so that the main image for the left eye is transmitted with a preset transmittance (1-α _1L ). The transmission unit 542 supplies the converted left-eye video data to the synthesis unit 545.

  The offset adding unit 543 reads caption data from the caption plane 531. The offset adding unit 543 generates subtitle data for the left eye from the read subtitle data based on the offset information of the subtitle data supplied from the offset generation unit 35 of FIG. The offset adding unit 543 supplies the caption data for the left eye to the transmission unit 544.

The transmission unit 544 converts the left-eye caption data supplied from the offset addition unit 547 so that the left-eye caption data is transmitted at a preset transmittance α _1L . The transmission unit 544 supplies the converted subtitle data for the left eye to the synthesis unit 545.

  The combining unit 545 combines the left-eye video data supplied from the transmission unit 542 and the left-eye caption data supplied from the transmission unit 544, and supplies the resultant data to the transmission unit 546.

The transmitting unit 546 converts the data so that an image corresponding to the data supplied from the combining unit 545 is transmitted with the transmittance (1-α _2L ), and supplies the converted data to the combining unit 549.

  The offset adding unit 547 reads menu data from the menu plane 532. The offset adding unit 547 generates menu data for the left eye from the read menu data, based on the menu button offset information supplied from the offset generation unit 35 of FIG. The offset addition unit 547 supplies the menu data for the left eye to the transmission unit 548.

The transmission unit 548 converts the menu data for the left eye supplied from the offset addition unit 547 so that the menu button for the left eye transmits at a preset transmittance α _2L . The transmission unit 548 supplies the converted menu data for the left eye to the synthesis unit 549.

  The combining unit 549 combines the data supplied from the transmission unit 546 and the menu data supplied from the transmission unit 548, and outputs the resulting data as display data for the left eye.

  The right-eye display data generation unit 534 includes a right-eye video plane 551, a transmission unit 552, an offset addition unit 553, a transmission unit 554, a synthesis unit 555, a transmission unit 556, an offset addition unit 557, a transmission unit 558, and a synthesis unit 559. Composed. The processing of each unit of the right-eye display data generation unit 534 is the same as the processing of each unit of the left-eye display data generation unit 533 except that the processing target is data for the right eye, and a description thereof will be omitted.

<Description of superposition order in 3D display data>
FIG. 14 is a diagram for explaining the superposition order in the 3D display data generated by the 3D display data generation unit 39.

  As shown in FIG. 14A, the superposition order of the left-eye display data in the 3D display data is, in order from the bottom, the left-eye video data, the left-eye caption data, and the left-eye menu data. As shown in FIG. 14B, the display order of the display data for the right eye is, in order from the bottom, the video data for the right eye, the caption data for the right eye, and the menu data for the right eye.

  In this specification, the step of describing the program stored in the program recording medium is not limited to the processing performed in time series according to the described order, but is not necessarily performed in time series, either in parallel or individually. This includes the processing to be executed.

  The embodiments of the present invention are not limited to the above-described embodiments, and various modifications can be made without departing from the scope of the present invention.

  11 disc, 20 playback device, 31 drive, 34 transport priority filter, 36 right-eye video generator, 39 3D display data generator, 100 recording device, 102 video encoder, 106 TS packetizer, 107 drive

Claims (2)

A sub-image stream packet that is a predetermined sub-image stream;
A packet of a video stream that is a stream of a main image and includes offset information including a shift direction and a shift amount of a left-eye sub-image and a right-eye sub-image used for 3D display with respect to the predetermined sub-image. Including and
In the data portion of the packet of the video stream, only one of the video data of the main image or the offset information is arranged,
In the header of the packet of the video stream, flag information that is information indicating whether the data arranged in the data portion of the packet is video data of the main image or the offset information is described. When playing back data with a data structure,
A reading unit for reading packets of the sub-image stream and the video stream;
The flag information described in the header of the packet read by the reading unit is based on the offset information arranged in the data portion of the packet representing that the offset information, A playback device comprising: a generating unit configured to generate video data of the left-eye sub-image and right-eye sub-image from the sub-image stream packet. A sub-image stream packet that is a predetermined sub-image stream;
A packet of a video stream that is a stream of a main image and includes offset information including a shift direction and a shift amount of a left-eye sub-image and a right-eye sub-image used for 3D display with respect to the predetermined sub-image. Including and
In the data portion of the packet of the video stream, only one of the video data of the main image or the offset information is arranged,
In the header of the packet of the video stream, flag information that is information indicating whether the data arranged in the data portion of the packet is video data of the main image or the offset information is described. A playback device that plays back data with a data structure
A reading step of reading packets of the sub-image stream and the video stream;
Based on the offset information arranged in the data portion of the packet indicating that the flag information described in the header of the packet read by the process of the reading step is the offset information And a generating step of generating video data of the sub-image for the left eye and video data of the sub-image for the right eye from the packet of the sub-image stream.

Images