JP4510854B2 - Playback system - Google Patents

Description

It relates to a recording medium such as a BD-ROM, in particular, a video stream, and reproduces the digital stream obtained by multiplexing a graphics stream to a technique for performing a caption display.

  Subtitle display realized by drawing a graphics stream is an indispensable means for allowing people in other languages to appreciate movie works produced in one language. A video stream representing a moving image is multiplexed with a graphics stream and recorded on a recording medium. This graphics stream includes a plurality of sets of display control information and graphics data called display sets. Each of these display sets realizes individual caption display when playing a movie work. As the video playback progresses, these display sets are read from the recording medium one by one and processed to display subtitles along with the video.

Here, when processing one display set, waiting for completion of processing of the previous display set causes a delay in processing. This delay is particularly noticeable when each of the multiple display sets to be displayed has a high resolution, such as 1920 × 1080. Therefore, when there are a plurality of display sets in the graphics stream, there is a desire to parallelize the processing for these display sets.
Japanese Patent No. 3128220

  However, in order to realize complete parallelization of graphics decoding, it is necessary to duplicate a combination of a processor that decodes graphics data and a controller that performs decoding control. If all the playback devices that are scheduled to play back BD-ROMs have a plurality of controller-processor pairs to construct an architecture based on parallel processing, the internal configuration becomes complicated. The more complicated the internal configuration is, the higher the manufacturing cost becomes, so that the spread of the playback device is hindered. In this way, since it can be a distraction factor, even if there is a request for parallelization, a configuration in which a plurality of controller-processor pairs are provided is not desirable from the viewpoint of standardization.

  An object of the present invention is to provide a recording medium capable of realizing parallel processing for a display set even if there is only one controller-processor pair.

In order to achieve the above object, a recording medium according to the present invention is a recording medium on which a digital stream obtained by multiplexing a moving image stream and a graphics stream is recorded, and the graphics stream displays a graphics screen. A plurality of display sets to be implemented, each display set including a presentation control segment, an object definition segment defining a graphics object, and an end segment;
Presentation control segment has a time information, the time information, on the playback time axis of the video stream is information that defines the active period of the presentation control segment,
Each segment is stored in a packet;
The time information includes a packet decoding time stamp and a presentation time stamp,
If the active period of the presentation control segment belonging to the preceding display set DSn and the active period of the presentation control segment belonging to the subsequent display set DSn + 1 overlap,
The value PTS (DSn [ODSlast]) of the presentation time stamp PTS of the packet storing the last object definition segment ODSlast belonging to the preceding display set DSn is for the object buffer in the active period of the presentation control segment belonging to the preceding display set DSn . Represents the time when the graphics object has been written and drawing of the graphics object to the graphics plane is possible. The presentation time stamp PTS value PTS (DSn [END] of the packet storing the end segment END belonging to the display set DSn ) With the relationship shown in the following formula,

PTS (DSn [END]) = PTS (DSn [ODSlast])

The value DTS (DSn + 1 [PCS]) of the decoding time stamp DTS of the packet storing the presentation control segment PCS belonging to the display set DSn + 1 is the starting point of the active period of the presentation control segment belonging to the subsequent display set DSn + 1 Satisfying the relationship expressed by the following formula with PTS (DSn [END])

PTS (DSn [END]) ≦ DTS (DSn + 1 [PCS])
It is characterized by that.

  According to the configuration described above, the processing for one display set is started in the middle of the active period of the control segment belonging to the preceding display set, so that the active period of the control segment belonging to the preceding display set is not terminated. , Processing of subsequent display sets can begin. The middle of the active period of the control segment is the time when the transfer of the decoded graphics is completed in the preceding display set. The processing start of the subsequent display set is advanced only during the period from the completion point to the end point of the active period of the control segment.

  Even if it starts in the middle of the active period of the control segment belonging to the preceding display set, the period for writing the graphics belonging to the subsequent display set to the buffer does not overlap with the period for writing the graphics belonging to the preceding display set to the buffer. Therefore, even if there is one processor that decodes graphics, processing for two or more display sets can be executed in a pipeline manner. By executing such a pipeline, it is possible to increase the processing efficiency for the display set without complicating the internal configuration.

(First embodiment)
Hereinafter, embodiments of the recording medium according to the present invention will be described. First of all, a description will be given of a usage act among implementation actions of a recording medium according to the present invention. FIG. 1 is a diagram showing a form of usage of a recording medium according to the present invention. In FIG. 1, a recording medium according to the present invention is a BD-ROM 100. The BD-ROM 100 is used for supplying movie works to a home theater system formed by a playback device 200, a television 300, and a remote controller 400.

The above is the description of the usage pattern of the recording medium according to the present invention.
Next, a description will be given of a production act among implementations of the recording medium according to the present invention. The recording medium according to the present invention can be implemented by improving the application layer of BD-ROM. FIG. 2 is a diagram showing the configuration of the BD-ROM.
The BD-ROM is shown in the fourth level of the figure, and the tracks on the BD-ROM are shown in the third level. The track in this figure is drawn by extending the track formed in a spiral shape from the inner periphery to the outer periphery of the BD-ROM in the horizontal direction. This track includes a lead-in area, a volume area, and a lead-out area. The volume area in this figure has a layer model of a physical layer, a file system layer, and an application layer. When the application layer format (application format) of the BD-ROM is expressed using the directory structure, it becomes like the first level in the figure. As shown in this figure, the BD-ROM has a BDMV directory under the ROOT directory, and under the BDMV directory, a file (XXX.M2TS) storing AVClip, a file storing management information of AVClip (XXX .CLPI), a file (YYY.MPLS) that defines a logical playback path (Playlist) in AVClip exists. By creating an application format as shown in the figure, the recording medium according to the present invention is produced. If there are multiple files such as XXX.M2TS, XXX.CLPI, and YYY.MPLS, three directories, STREAM directory, CLIPINF directory, and PLAYLIST directory, are provided under the BDMV directory. It is desirable to store files of the same type as M2TS, files of the same type as XXX.CLPI in the CLIPINF directory, and files of the same type as YYY.MPLS in the PLAYLIST directory.

A description will be given of AVClip (XXX.M2TS) in this application format.
AVClip (XXX.M2TS) is a digital stream in MPEG-TS (Transport Stream) format, and is obtained by multiplexing a video stream, one or more audio streams, and a presentation graphics stream. The video stream indicates the moving image portion of the movie, the audio stream indicates the audio portion of the movie, and the presentation graphics stream indicates the subtitles of the movie. FIG. 3 is a diagram schematically showing how the AVClip is configured.

  AVClip (middle) converts a video stream consisting of multiple video frames (pictures pj1,2,3) and an audio stream consisting of multiple audio frames (top 1) into a PES packet sequence (top 2) ), And further converted to TS packets (upper 3rd stage). Similarly, the presentation graphics stream (lower 1st stage) is converted to PES packet sequence (lower 2nd stage), and further converted to TS packets (lower 3). (Stage), these are multiplexed.

In this figure, there is one presentation graphics stream multiplexed. However, when the BD-ROM is multilingual compatible, the presentation graphics stream for each language is multiplexed on the AVClip. The AVClip generated through such a process is divided into a plurality of extents and recorded in an area on the BD-ROM, like a normal computer file.
Next, the presentation graphics stream will be described. FIG. 4A is a diagram illustrating a configuration of a presentation graphics stream. The first level shows a TS packet sequence constituting the AVClip. The second level shows a PES packet sequence constituting the graphics stream. The PES packet sequence in the second stage is configured by extracting the payload from the TS packets having a predetermined PID among the TS packets in the first stage and concatenating them.

  The third level shows the structure of the graphics stream. The graphics stream is composed of functional segments called PCS (Presentation Composition Segment), WDS (Window Define Segment), PDS (Palette Definition Segment), ODS (Object_Definition_Segment), and END (END of Display Set Segment). Among these functional segments, PCS is called a screen composition segment, and WDS, PDS, ODS, and END are called definition segments. The correspondence between PES packets and functional segments is a one-to-one relationship or a one-to-many relationship. That is, the functional segment is converted into one PES packet and recorded on the BD-ROM, or fragmented, converted into a plurality of PES packets, and recorded on the BD-ROM.

  FIG. 4B is a diagram showing a PES packet obtained by converting functional segments. As shown in FIG. 4B, the PES packet includes a packet header and a payload, and this payload corresponds to a functional segment entity. In the packet header, there are DTS and PTS corresponding to this functional segment. In the following description, the DTS and PTS existing in the header of the PES packet in which the functional segment is stored are treated as the DTS and PTS of the functional segment.

These various types of functional segments form a logical structure as shown in FIG. FIG. 5 is a diagram showing a logical structure composed of various types of functional segments. This figure shows functional segments in the third row, Display Set in the second row, and Epoch in the first row.
The second level Display Set (abbreviated as DS) refers to a set of a plurality of functional segments constituting a graphics stream that constitute graphics for one screen. The broken line in the figure indicates the attribution relationship to which DS the functional segment in the third row belongs. It can be seen that a series of functional segments called PCS-WDS-PDS-ODS-END constitutes one DS. The playback device can configure graphics for one screen by reading a plurality of functional segments constituting the DS from the BD-ROM.

  The first-stage Epoch refers to one period having memory management continuity on the playback time axis of the AVClip, and a data group assigned to this period. The memory assumed here is a graphics plane for storing graphics for one screen and an object buffer for storing graphics data in an expanded state. There is continuity in memory management for these, because the graphics plane and object buffer are not flushed during the period corresponding to this Epoch, and only within a certain rectangular area in the graphics plane. Erasing and redrawing are performed (* Flash here means clearing all stored contents of the plane and buffer). The vertical and horizontal sizes and positions of the rectangular area are fixed throughout the period corresponding to Epoch. In the graphics plane, as long as the graphics are erased and redrawn in this fixed area, synchronization between the video and the graphics is guaranteed. In other words, Epoch can be said to be a unit on the playback time axis that can ensure synchronization of video-graphics. In the graphics plane, if it is desired to change the area where the graphics should be erased / redrawn, the change time point must be defined on the playback time axis, and after that change point, a new Epoch must be set. In this case, video-graphics synchronization is not guaranteed at the boundary between two Epochs.

  For example, the time period in which captions appear in a certain rectangular area on the screen on the playback time axis can be referred to as Epoch. FIG. 6 is a diagram illustrating the relationship between the subtitle display position and Epoch. In this figure, consideration is given to changing the position of the subtitle according to the pattern of each picture of the moving image. In other words, out of the five subtitles “real”, “it was a lie”, “sorry”, “after that”, “three years have passed”, the three subtitles “actually”, “it was a lie”, “sorry” “After that” and “3 years have passed” are located at the top of the screen. This is intended to arrange subtitles in a position corresponding to a blank space in the screen in consideration of easy viewing of the screen. When there is such temporal variation, on the playback time axis of AVClip, the period in which subtitles appear in the lower margin is one Epoch1, and the period in which subtitles appear in the upper margin is another Epoch2. . Each of these two Epochs has its own subtitle drawing area. In Epoch1, the lower margin of the screen is the subtitle drawing area (window 1). On the other hand, in Epoch2, the upper margin of the screen becomes the subtitle drawing area (window 2). In these Epochs 1 and 2, since the continuity of memory management in the buffer plane is guaranteed, the above-described caption display in the margin is performed seamlessly. This completes the explanation of Epoch. Next, Display Set will be described.

  The broken lines hk1,2 in FIG. 5 indicate the attribution relationship to which Epoch the second-stage functional segment belongs. It can be seen that a series of DSs of Epoch Start, Acquisition Point, and Normal Case constitute the first stage Epoch. “Epoch Start”, “Acquisition Point”, and “Normal Case” are types of DS. The order of Acquisition Point and Normal Case in this figure is only an example, and either may be the first.

“Epoch Start” is a DS that provides a display effect of “new display” and indicates the start of a new Epoch. Therefore, Epoch Start includes all functional segments necessary for the next screen composition. Epoch Start is placed at a position where it has been found to be cued, such as a chapter in a movie work.
“Acquisition Point” is a Display Set that brings about a display effect of “display refresh”, and is exactly the same Display Set as the preceding Epoch Start. The acquisition point DS is not at the beginning of Epoch, but includes all the functional segments necessary for the next screen composition, so if you cue from the acquisition point DS, the graphics display will be realized reliably. be able to. In other words, Acquisition Point DS has the role of enabling screen composition from the middle of Epoch.

  The Display Set as an acquisition point is incorporated at a position where it can be a cue point. Such positions include positions that can be specified by time search. The time search is an operation of accepting a time input of minutes and seconds from the user and cuing from the playback point corresponding to the time input. Since such time input is performed in rough units such as 10 minutes and 10 seconds, the reproduction position at intervals of 10 minutes and the reproduction position at intervals of 10 seconds are positions that can be designated by time search. In this way, by providing Acquisition Points at positions that can be specified by time search, it is possible to suitably perform graphics stream playback during time search.

  “Normal Case” is a DS that provides a display effect of “display update”, and includes only the difference from the previous screen composition. For example, a subtitle of a certain DSv has the same content as the preceding DSu, but if the screen configuration is different from the preceding DSu, a DSv of only PCS and END is provided and this DSv is set as a DS of the normal case. In this way, there is no need to provide overlapping ODS, which can contribute to capacity reduction in the BD-ROM. On the other hand, since the DS of the Normal Case is only a difference, the screen configuration cannot be performed with the Normal Case alone.

Next, Definition Segment (ODS, WDS, PDS) will be described.
“Object_Definition_Segment” is a functional segment that defines a graphics object. This graphics object will be described below. Since the AVClip recorded on the BD-ROM has a high image quality equivalent to that of a high-definition TV, the resolution of the graphics object is also set to a high-definition size of 1920 × 1080 pixels. Since it has a resolution of 1920x1080, the BD-ROM can vividly reproduce the subtitle display for theater screenings, that is, the handwritten subtitle display. The graphics object consists of a plurality of run-length data. The run-length data is data representing a pixel column by a Pixel Code indicating a pixel value and a continuous length of the pixel value. Pixel Code is an 8-bit value and takes a value from 1 to 255. In run-length data, any 256 colors can be selected and set as pixel colors from 16,777,216 full-color colors using this Pixel Code. When displayed as subtitles, the graphics object must be drawn by placing a character string on a transparent background.

  The definition of a graphics object by ODS has a data structure as shown in FIG. As shown in FIG. 7A, the ODS uniquely identifies “segment_type” indicating that it is an ODS, “segment_length” indicating the data length of the ODS, and a graphics object corresponding to this ODS in the Epoch. It consists of “object_id”, “object_version_number” indicating the version of ODS in Epoch, “last_in_sequence_flag”, and continuous byte length data “object_data_fragment” which is a part or all of the graphics object.

  “Object_id” is an identifier that uniquely identifies this graphics object and the area occupied by this graphics object in the object buffer when the corresponding graphics object is read onto the object buffer by ODS decoding. is there. In a state where one or more graphics objects are stored, each area in the memory space of the object buffer is identified by this object_id. If an object_id is added to more than one Display Set, the graphics object corresponding to the preceding ODS will be overwritten by subsequent graphics objects with the same object_id after being stored in the object buffer. . Such an update is a consideration to avoid the occurrence of worm-feeding in empty areas and the discrete arrangement of graphics objects. The reason is as follows. In order to perform graphics display, the graphics object on the object buffer is constantly transferred to the graphics plane. If free space occurs in the object buffer, or if one graphics object with the same object_id is separated and arranged discretely, the overhead for reading the graphics object occurs, and the object buffer → Transfer efficiency is reduced when transferring graphics plane. This reduction in efficiency can adversely affect the graphics-video synchronization display. For this reason, an area shared by a graphics object identified by an object_id on the object buffer can be overwritten only by a graphics object having the same size and the same object_id.

  In overwriting a graphics object, the size of the succeeding graphics object must be the same size as the preceding one. It must not be too small or too large for the preceding graphics object. It is the responsibility of the authoring person to create the graphics object so that the size of the graphics object becomes the same when updating. The limitation that graphics objects with the same ID have the same vertical / horizontal size is only a limitation within one Epoch. When a graphics object belonging to a certain Epoch and a graphics object belonging to the next Epoch have the same object_id, the vertical and horizontal sizes of these graphics objects may be changed.

  “Last_in_sequence_flag” and “object_data_fragment” will be described. Due to PES packet payload limitations, uncompressed graphics that make up subtitles may not be stored in a single ODS. In such a case, one part (fragment) obtained by dividing the graphics is set in object_data_fragment. When one graphics object is stored in multiple ODSs, all fragments except the last fragment have the same size. That is, the last fragment is smaller than the previous fragment size. ODSs that store these fragments appear in the same order in the DS. The end of the graphics object is indicated by the ODS having last_in_sequence_flag. The ODS data structure described above presupposes a storage method in which fragments are packed from the previous PES packet, but a storage method in which each PES packet is packed so that a space is generated may be used. The above is the explanation of ODS.

  “Palette Difinition Segment (PDS)” is information that defines a palette for color conversion. The palette is data indicating a combination of a pixel code of 1 to 255 and a pixel value. Here, the pixel value is composed of a red color difference component (Cr value), a blue color difference component (Cb value), a luminance component (Y value), and transparency (T value). By replacing the Pixel Code of each run-length data with the pixel value shown in the palette, the run-length data is colored. The data structure of PDS is shown in FIG. As shown in FIG. 7B, the PDS has “segment_type” indicating that it is a PDS, “segment_length” indicating the data length of the PDS, “pallet_id” uniquely identifying the palette included in this PDS, and Epoch It consists of “pallet_version_number” indicating the version of the Epoch PDS and information “pallet_entry” for each entry. “Pallet_entry” indicates a red color difference component (Cr value), a blue color difference component (Cb value), a luminance component Y value, and transparency (T value) in each entry.

Next, WDS will be described.
“Window_definition_segment” is a functional segment for defining a rectangular area of the graphics plane. It has already been described in Epoch that continuity occurs in memory management only when clearing and redrawing are performed within a rectangular area in the graphics plane. The rectangular area in this graphics plane is called “window” and is defined by this WDS. FIG. 8A shows the data structure of WDS. As shown in this figure, the WDS has a window_id that uniquely identifies the window in the graphics plane, a window_horizontal_position that indicates the horizontal position of the upper left pixel in the graphics plane, and a vertical position of the upper left pixel in the graphics plane. It is expressed using “window_vertical_position” shown, “window_width” showing the horizontal width of the window in the graphics plane, and “window_height” showing the vertical width in the graphics plane.

The possible values of window_horizontal_position, window_vertical_position, window_width, and window_height will be described. The coordinate system assumed by these is an internal area of the graphics plane, and this graphics plane has a two-dimensional coordinate system defined by vertical: video_height and horizontal: video_width.
Since window_horizontal_position is the horizontal address of the upper left pixel in the graphics plane, it takes a value of 0 to video_width-1, and window_vertical_position is a vertical address of the upper left pixel in the graphics plane and takes a value of 0 to video_height-1. .

Since window_width is the horizontal width of the window in the graphics plane, it takes a value from 1 to (video_width)-(window_horizontal_position), and since window_height is the vertical width in the graphics plane, 1 to (video_height)-(window_vertical_position) Takes a value.
The window_horizontal_position, window_vertical_position, window_width, and window_height of the WDS can specify for each Epoch where the window is to be placed on the graphics plane and how large the window is. For this reason, during a period in which a picture belonging to a certain Epoch is displayed, it is possible to make an adjustment during authoring so that a window appears in a position corresponding to a margin on the picture so as not to disturb the picture in the picture. Thereby, it is possible to make the subtitle display by graphics easy to see. Since WDS can be defined for each Epoch, even if there is a temporal variation in the picture pattern, graphics can be displayed in an easy-to-read manner according to the variation. As a result, the quality of the movie work can be improved to the same level as when subtitles are incorporated into the video body.

  Next, “END of Display Set Segment” will be described. The END of Display Set Segment is an index indicating the end of transmission of the Display Set, and is arranged immediately after the last ODS among the functional segments in the Display Set. The internal structure of this END of Display SetSegment consists of “segment_type” indicating that it is the END of Display SetSegment and “segment_length” indicating the data length of the functional segment. There are no elements. Therefore, illustration is abbreviate | omitted.

This completes the explanation of ODS, PDS, WDS, and END. Next, PCS will be described.
PCS is a functional segment that constitutes an interactive screen. The PCS has a data structure shown in FIG. As shown in this figure, the PCS has “segment_type”, “segment_length”, “composition_number”, “composition_state”, “pallet_update_flag”, “pallet_id”, and “composition_object (1) to (m)”. Consists of

“Composition_number” identifies a graphics update in the Display Set using a numerical value from 0 to 15. As for how to identify, if there is a graphics update from the beginning of the Epoch to this PCS, the composition_number is set with a rule that it is incremented every time it passes through the graphics update.
“Composition_state” indicates whether the Display Set starting from this PCS is Normal Case, ACquisition Point, or Epoch Start.

“Pallet_update_flag” indicates whether or not PalletOnly Displey Update is performed in this PCS. PalletOnly Displey Update is an update made by switching only the previous palette to a new one. If this update is done in this PCS, this field is set to “1”.
“Pallet_id” indicates a palette to be used for PalletOnly Displey Update.

  “Composition_object (1)... (N)” is information indicating how to control each window in the Display Set to which this PCS belongs. A broken line wd1 in FIG. 8B closes up the internal configuration of an arbitrary composition_object (i). As shown by the broken line wd1, composition_object (i) is composed of “object_id”, “window_id”, “object_cropped_flag”, “object_horizontal_position”, “object_vertical_position”, “cropping_rectangle information (1) (2) ... (n ) ”.

“Object_id” indicates an identifier of an ODS existing in the window corresponding to composition_object (i).
“Window_id” indicates a window to be assigned to the graphics object in this PCS. A maximum of two graphics objects can be assigned to a window.

  “Object_cropped_flag” is a flag for switching whether to display a graphics object that has been cropped in the object buffer or not to display the graphics object. When “1” is set, the graphics object cropped in the object buffer is displayed. When “0” is set, the graphics object is not displayed.

“Object_horizontal_position” indicates the horizontal position of the upper left pixel of the graphics object in the graphics plane.
“Object_vertical_position” indicates the vertical position of the upper left pixel in the graphics plane.
“Cropping_rectangle information (1) (2)... (N)” is an information element that is valid when “object_cropped_flag” is set to 1. A broken line wd2 closes up the internal configuration of arbitrary cropping_rectangle information (i). As illustrated, cropping_rectangle information (i) is composed of "object_cropping_horizontal_position", "Object_cropping_vertical_ position", "object_cropping_width", "object_cropping_height".

“Object_cropping_horizontal_position” indicates the horizontal position of the upper left pixel of the crop rectangle in the object buffer . The crop rectangle is a frame for cutting out a part of the graphics object, and corresponds to “Region” in the ETSI EN 300 743 standard.
"Object_cropping_vertical_ position" indicates the vertical position of the top left corner of the cropping rectangle in the Object Buffer.

“Object_cropping_width” indicates the horizontal width of the crop rectangle in the object buffer .
“Object_cropping_height” indicates the vertical width of the crop rectangle in the object buffer .
The above is the data structure of PCS. Next, the specific description of PCS will be explained. In this example, the subtitles shown in FIG. 6 are displayed, that is, as the video playback progresses, it is gradually displayed as “true”, “lie” or “sorry” by writing to the graphics plane three times. It is to let you. FIG. 9 shows a description example for realizing such caption display. Epochs in this figure have DS1 (Epoch Start), DS2 (Normal Case), and DS3 (Normal Case). DS1 has a WDS that defines the window that is the display frame for subtitles, an ODS that represents the line “I was really a lie”, and a first PCS. DS2 (Normal Case) has a second PCS. DS3 (Normal Case) has a third PCS.

Next, how to describe each PCS is explained. Description examples of WDS and PCS belonging to the Display Set are shown in FIGS. FIG. 10 is a diagram illustrating a description example of the PCS in DS1.
In FIG. 10, window_horizontal_position and window_vertical_position of the WDS indicate the upper left coordinate LP1 of the window in the graphics plane, and window_width and window_height indicate the horizontal and vertical widths of the window display frame.

  The object_cropping_horizontal_position and object_cropping_vertical_position of the crop information in FIG. 10 indicate the reference SDT of the crop range in the coordinate system with the upper left coordinate of the graphics object in the object buffer as the origin. Then, the range indicated by object_cropping_width and object_cropping_height from the reference point (thick frame portion in the figure) becomes the cropping range. The cropped graphics object is arranged in a broken line range cp1 with object_horizontal_position and object_vertical_position as reference points (upper left) in the coordinate system of the graphics plane. By doing so, “true” is written in the window in the graphics plane. As a result, the subtitle “actually” is combined with the moving image and displayed.

  FIG. 11 is a diagram illustrating a description example of the PCS in DS2. The description of WDS in this figure is the same as in FIG. The description of the crop information is different from FIG. The crop information object_cropping_horizontal_position and object_cropping_vertical_position in FIG. 11 are subtitles “actually lies in the object buffer. “I'm sorry” shows the upper left coordinates of “It was a lie”, and object_cropping_height and object_cropping_width show the horizontal and vertical widths of “I was lie”. By doing so, “It was a lie” is written in the window in the graphics plane. As a result, the subtitle “It was a lie” is combined with the moving image and displayed.

  FIG. 12 is a diagram illustrating a description example of the PCS in DS3. The description of WDS in this figure is the same as in FIG. The description of the crop information is different from FIG. The crop information object_cropping_horizontal_position and object_cropping_vertical_position in FIG. 12 are subtitles “actually lies in the object buffer. "I'm sorry" indicates the upper left coordinates of "I'm sorry", and object_cropping_height and object_cropping_width indicate the horizontal and vertical widths of "I'm sorry". By doing so, “I'm sorry” is written in the window in the graphics plane. As a result, the subtitle “I'm sorry” is combined with the moving image and displayed. FIG. 13 is a diagram showing a memory space in the object buffer when realizing the graphics update as shown in FIGS. As shown in this figure, the object buffer has four storage areas A, B, C, and D whose fixed width, width, and position are fixed. Among these, the subtitles shown in FIG. The Each of these four storage areas A, B, C, and D is identified by object_id corresponding to the stored graphics object. That is, storage area A is identified by object_id = 1, storage area B is identified by object_id = 2, storage area C is identified by object_id = 3, and storage area D is identified by object_id = 4. In order to maintain the transfer efficiency to the graphics plane, the vertical and horizontal widths of these storage areas are fixed throughout. If a graphics object having the same object_id is newly obtained by decoding, those storage areas are overwritten by the newly obtained graphics object. For example, if it is desired to display a caption with the same size at the same position as the caption shown in FIG. 10, an ODS with the same object_id added may be provided in the subsequent Display Set. By simply adding the same object_id in this way, the graphics object existing in the object buffer is overwritten with a new graphics object, and the new graphics object is displayed at the same size and position as the previous graphics object. Will be.

The restrictions for realizing the display effect will be described. Realizing a smooth display effect requires window clear and window redraw. If window clear and window redrawing are implemented at video frame time intervals, how much transfer rate is required between the object buffer and the graphics plane?
Now consider the limits on how big a window can be. When the transfer rate between the object buffer and the graphics plane is Rc, in the worst case, window clear and window redrawing must be performed at this transfer rate Rc. Then, each of Windu Clear and Window Redraw must be realized at a transfer rate (Rc / 2) that is half of Rc.

To synchronize these window clear-window redraws with the video frame,
Window size x frame rate ≒ Rc / 2

It is necessary to satisfy. If this frame rate is 29.97,

Rc is the window size x 2 x 29.97.

When displaying subtitles, the size of the graphics plane must be at least 25% to 33%. Here, the total number of pixels of the graphics plane is 1920 × 1080, and if the bit length of the index per pixel is 8 bits, the total capacity of the graphics plane is 2 Mbytes (≈1920 × 1080 × 8).

If 1/4 of this total capacity is the window size, it becomes 500k bytes (= 2M bytes / 4). By substituting this into the above equation and deriving Rc, Rc can be calculated as 256 Mbps (500 Kbytes × 2 × 29.97).
If the size is 25% to 33%, as long as captions are displayed at a transfer rate of 256 Mbps, synchronization with a moving image can be maintained regardless of the display effect.

If the window clear and redraw rates may be 1/2 and 1/4 of the frame rate of the video frame, the window size can be doubled or quadrupled even if Rc is the same. This completes the description of the size of the window. Next, the position and range of the window will be described. As mentioned above, the position and range of the window are consistent in Epoch.
The reason for keeping Windu's position and range consistent in Epoch is as follows. This is because, if the window position / range is changed, the write destination address for the graphics plane must be changed, and an overhead is generated. This overhead reduces the transfer rate from the object buffer to the graphics plane.

  A window has a limited number of graphics objects. This number limitation is provided for the purpose of reducing overhead in transferring the decoded graphics object. The overhead here occurs when setting the address of the edge portion of the graphics object. In this case, the more edges there are, the more overhead is generated.

  If there is no limit to the number of graphics objects in the window, the number of overheads that occur during graphics transfer will be unknown, and the transfer load will increase and decrease drastically. On the other hand, if the number of graphics in the window is up to two, it is only necessary to set the transfer rate assuming that the worst four overheads occur, so it becomes easy to quantify the minimum standard transfer rate.

  The above is an explanation of Windu. Next, how the Display Set having these PCS and ODS is allocated on the playback time axis of AVClip will be described. Epoch is a period in which memory management is continued on the playback time axis. Since Epoch is composed of one or more Display Sets, it becomes a problem how to allocate the Display Set to the playback time axis of AVClip. Here, the playback time axis of the AVClip refers to a time axis that is assumed to define the decoding timing and playback timing of individual picture data that constitutes a video stream multiplexed on the AVClip. On this playback time axis, the decode timing and playback timing are expressed with a time accuracy of 90 KHz. DTS and PTS added to the PCS and ODS in the Display Set indicate the timing at which synchronization control should be realized on this playback time axis. Performing synchronous control using DTS and PTS added to the PCS and ODS is the assignment of the Display Set to the playback time axis.

First, what kind of synchronization control is performed by the DTS and PTS added to the ODS will be described.
The DTS indicates the time at which ODS decoding should be started with a time accuracy of 90 KHz, and the PTS indicates the decoding end time.
ODS decoding does not complete instantaneously, but has a length of time. In order to clarify the start and end points of this decoding period, DTS and PTS for ODS indicate the decoding start time and decoding end time.

Since the value of the PTS is a deadline, the ODS is decoded by the time indicated by the PTS, and an uncompressed graphic object must be obtained in the object buffer on the playback device.
The decoding start time of any ODSj belonging to DSn is indicated in the DTS (DSn [ODSj]) with a time accuracy of 90 KHz, so the time obtained by adding the longest time required for decoding to this is the decoding end time of the ODSj in the Display Set become.

When the ODSj size is “SIZE (DSn [ODSj])” and the ODS decoding rate is “Rd”, the maximum time (seconds) required for decoding is “SIZE (DSn [ODSj]) // Rd”. Here, “//” is an operator indicating division by rounding up after the decimal point.
By converting this longest time into a time accuracy of 90 KHz and adding it to the DTS of ODSj, the decoding end time (90 KHz) to be indicated by the PTS is calculated.

The PTS of ODSj belonging to DSn can be expressed by the following formula.

PTS (DS [ODSj]) = DTS (DSn [ODSj]) + 90,000 × (SIZE (DSn [ODSj]) // Rd)

And it is necessary to satisfy the following relationship between two adjacent ODSs (ODSj, ODSj + 1).

PTS (DSn [ODSj]) ≦ DTS (DSn [ODSj + 1])

Since END belonging to DSn indicates the end of DSn, the decoding end time of the last ODS (ODSlast) belonging to DSn may be indicated. Since this decoding end time is indicated in the PTS of ODS2 (ODSlast) (PTS (DSn [ODSlast])), the PTS of END is set to the value shown in the following equation.

PTS (DSn [END]) = PTS (DSn [ODSlast])

Next, PCS DTS and PTS settings will be described.

  The DTS value of PCS indicates the time point at which decoding of the first ODS (ODS1) in DSn is started or before that time. This is because the PCS starts decoding the first ODS (ODS1) in DSn (DTS (DSn [ODS1])) and the first PDS (PDS1) in DSn becomes valid (PTS (DSn [PDS1]) ) Must be loaded into the buffer on the playback device at the same time or before. Therefore, it must be set to a value that satisfies the relationship of the following expression.

DTS (DSn [PCS]) ≦ DTS (DSn [ODS1])
DTS (DSn [PCS]) ≦ PTS (DSn [PDS1])
The PCS PTS in DSn is calculated from the following equation.

PTS (DSn [PCS]) ≧ DTS (DSn [PCS]) + decodeduration (DSn)

Here, decodeduration (DSn) is the time required to decode and display all graphics objects used for updating the PCS in DSn. This decoding time is not a fixed value. However, it does not vary depending on the state of each playback device or the implementation of the playback device. When the object used for the screen composition of this DSn.PCSn is DSn.PCSn.OBJ [j], decoding (DSn) is the time required for window clear (i), the decoding period of DSn.PCSn.OBJ (ii) , DSn.PCSn.OBJ is a value that varies depending on the time (iii) required for writing. As long as Rd and Rc are determined in advance, the playback apparatus of any implementation has the same value. Therefore, when authoring, the length of these periods is calculated and the PTS is calculated from this value.

  The calculation of decode_duration is performed based on the program of FIG. FIGS. 15, 16 (a) and 16 (b) are flowcharts showing the algorithm of this program. Hereinafter, calculation of decode_duration will be described with reference to these drawings. In the flowchart of FIG. 15, first, the PLANEINITIALIZATINTIME function is called, and the return value is added to decode_duration (step S1 of FIG. 15). The PLANEINITIALIZATINTIME function (FIG. 16A) is a function for obtaining the time required to initialize the graphics plane in order to realize the display of DSn. In step S1 of FIG. 15, DSn, DSn.PCS.OBJ [0], decode_durtation are set as arguments and this function is called.

Next, the PLANEINITIALIZATINTIME function will be described with reference to FIG. In FIG. 16A, initialize_duration is a variable indicating the return value of the PLANEINITIALIZATINTIME function.
Step S2 in FIG. 16 is an if statement for switching processing depending on whether composition_state in the DSS PCS is Epoch Start. If the composition_state is Epoch Start (DSn.PCS.composition_state == EPOCH_START in FIG. 14, step S2 = Yes), the time required for clearing the graphics plane is set to initialize_duration (step S3).

  When the transfer rate Rc between the object buffer and the graphics plane is 256,000,000 as described above, and the total size of the graphics plane is video_width * video_height, the time (seconds) required for clearing is “video_width * video_height // 256,000,000”. Become. If this is multiplied by 90,000 Hz and expressed with PTS time accuracy, the time required to clear the graphics plane is “90000 × video_width * video_height // 256,000,000”. This time is added to initialize_duration and returned with initialize_duration as a return value.

  If composition_state is not Epoch Start (step S2 = No), the process of adding the time required for clearing window [i] defined in WDS to initialize_duration is repeated for all windows (step S4). If the transfer rate Rc between the object buffer and the graphics plane is 256,000,000 as described above, and the total size of the window [i] belonging to WDS is ΣSIZE (WDS.WIN [i]), the time (seconds) required for clearing is , “ΣSIZE (WDS.WIN [i]) // 256,000,000”. If this is multiplied by 90,000 Hz and expressed in PTS time accuracy, the time required to clear all windows belonging to the WDS is “90000 × ΣSIZE (WDS.WIN [i])) // 256,000,000”. This time is added to initialize_duration and returned with initialize_duration as a return value. The above is the PLANEINITIALIZATINTIME function.

  Step S5 in FIG. 15 switches processing depending on whether the number of graphics objects belonging to DSn is 2 or 1 (if (DSn.PCS.num_of_object == 2), if (DSn in FIG. 14). .PCS.num_of_object == 1)) If “1” (step S5 = 1), a waiting time for decoding the graphics object is added to decode_duration (step S6). The calculation of the waiting time is realized by calling the WAIT function (decode_duration + = WAIT (DSn, DSn.PCS.OBJ [0], decode_duration) in FIG. 14). The argument for calling this function is DSn, DSn.PCS.OBJ [0], decode_duration, and wait_duration indicating the waiting time is returned as a return value.

FIG. 16B is a flowchart showing the processing procedure of the WAIT function.
In this WAIT function, current_duration is set to the caller's decode_duration. object_define_ready_time is a variable in which the PTS of the graphics object [i] of the Display Set is set.
The current_time is a variable in which a value obtained by adding the current_duration to the DTS of the DSS PCS is set. When object_define_ready_time is larger than this current_time (Yes in step S7, if (current_time <object_define_ready_time)), the return value wait_duration is set to the difference between object_define_ready_time and current_time (step S8, wait_duration + = object_define_ready_time-current_time) ). The above is the Wait function. In step S9, decode_duration is set to the sum of the return value of this wait function and the time required for window redrawing (90,000 * (SIZE (DSn.WDS.WIN [0])) // 256,000,000) (Step S9).

  The above is the process when there is one DSn graphics object. Step S5 in FIG. 15 determines whether or not the number of graphics objects is two. If the DSn graphics object is 2 or more (if (DSn.PCS.num_of_object == 2) in FIG. 14), the wait function is called with the DSn, DSn.PCS.OBJ [0], decode_duration in the PCS as arguments, The return value is added to decode_duration (step S10).

  The subsequent step S11 is a determination as to whether the window to which OBJ [0] of DSn belongs is the same as the window to which graphics object [1] belongs (if (DSn.PCS.OBJ [0] .window_id == DSn.PCS .OBJ [1] .window_id) If they are the same, the wait function is called with DSn, DSn.PCS.OBJ [1], decode_duration as arguments, and the return value wait_duration is added to decode_duration (step S12). The time required for redrawing the window to which OBJ [0] belongs (90,000 * (SIZE (DSn.WDS.OBJ [0] .window_id) // 256,000,000)) is added to decode_duration (step S13).

  If the windows to which they belong are different ("different" in step S11), the time required to redraw the window to which OBJ [0] belongs (90,000 * (SIZE (DSn.WDS.OBJ [0] .window_id) // 256,000,000) Is added to decode_duration (step S15), the wait function is called with DSn, DSn.PCS.OBJ [1] and decode_duration as arguments, and the return value wait_duration is added to decode_duration (step S16), and then OBJ [1. ] Is added to decode_duration (90,000 * (SIZE (DSn.WDS.OBJ [1] .window_id) // 256,000,000)) necessary for redrawing the window to which the file belongs (step S17).

Decode_Duration is calculated by the above algorithm. How the PTS of the PCS is specifically set will be described.
FIG. 17A is a diagram assuming a case where one ODS exists in one window. FIGS. 17B and 17C are timing charts showing the temporal relationship between the numerical values quoted in FIG. This chart has three stages. Of these stages, the "Graphics Plane Access" stage and the "ODS Decode" stage show two processes that are performed in parallel during playback. The algorithm described above assumes parallel execution of these two processes.

  The graphics plane access consists of a clear period (1) and a write period (3). This clear period (1) is the period required to clear the entire graphics plane (90,000 x (graphics plane size // 256,000,000)), the time required to clear all windows in the graphics plane (Σ (90,000 x (window [i] size // 256,000,000))) means either.

The writing period (3) means a period (90,000 × (window size // 256,000,000)) required for drawing the entire window.
On the other hand, the decoding period (2) means a period indicated from DTS to PTS of ODS. The clear period (1) to the write period (3) may vary depending on the range to be cleared, the size of the ODS to be decoded, and the size of the graphics object to be written to the graphics plane. In this figure, for the sake of simplicity, it is assumed that the start point of the ODS decode period (2) is the same as the start point of the clear period (1).

FIG. 17B shows a case where the decoding period (2) becomes long, and Decode_Duration is the decoding period (2) + the writing period (3).
FIG. 17C shows a case in which the clear period (1) becomes long, and Decode_Duration has a period of clear period (1) + write period (3) as Decode_Duration.
18A to 18C are diagrams assuming a case where two ODSs exist in one window. The decoding period (2) in FIGS. 2B and 2C means the sum of periods required for decoding two graphics. The graphics writing period (3) also means the total period of writing two graphics to the graphics plane. Although there are two ODSs, Decode_Duration can be calculated if considered similarly to FIG. When the decoding period (2) for decoding two ODSs is long, Decode_Duration is calculated as decoding period (2) + writing period (3) as shown in FIG.

When the clear period (1) is long, as shown in FIG. 18C, Decode_Duration is the clear period (1) + the write period (3).
FIG. 19A assumes a case where one ODS exists in each of two windows. Even in this case, if the clear period (1) is longer than the total decode period (2) for decoding two ODSs, Decode_Duration is the clear period (1) + the write period (3). The problem is that the clear period (1) is shorter than the decode period (2). In this case, writing to the first window is possible without waiting for the decoding period (2) to elapse. Therefore, the length of the clear period (1) + the write period (3) and the decode period (2) + the write period (3) are not reached. Here, a period required for decoding the first ODS is a writing period (and a period required for decoding the second ODS is a writing period. In FIG. 19B, the decoding period (2) is cleared. Period (1) + Writing period (shows a longer case. In this case, Decode_Duration becomes Decoding period (2) + Writing period (.

FIG. 19C shows a case where the clear period (1) + write period (is longer than the decode period (2). In this case, Decode_Duration is the clear period (1) + write period (+ write period (N Become.
The size of the graphics plane is known in advance from the player model, and the size of the window, the size of the ODS, and the number of pieces are also known at the authoring stage. Therefore, the Decode_Duration is the clear period (1) + writing period ( 3), decode period (2) + write period (3), decode period (2) + write period (, clear period (1) + write period (+ write period). Such Decode_Duration calculation By setting the PCS PTS based on this, synchronized display with picture data can be realized with high time accuracy.Such high-accuracy synchronous control defines a window and defines the range to be cleared and redrawn. Because it consists of limiting to this window, the significance of introducing this window concept into the authoring environment is significant.

Next, the setting of DTS and PTS of WDS in DSn will be described. WDS DTS may be set to satisfy the following formula.
DTS (DSn [WDS]) ≧ DTS (DSn [PCS])

On the other hand, the PTS of WDS in DSn indicates a deadline at which writing to the graphics plane should start. Since writing to the graphics plane is sufficient only for the window, the time to start writing to the graphics plane is determined by subtracting the time required for writing all windows from the time indicated in the PCS PTS. If the total size of the WDS is ΣSIZE (WDS.WIN [i]), the time required for clearing and redrawing will be “ΣSIZE (WDS.WIN [i]) // 256,000,000”. And this is expressed as “90000 × ΣSIZE (WDS.WIN [i]) // 256,000,000” when expressed with a time accuracy of 90.000 KHz.

Therefore, the PTS of WDS should be calculated from the following formula.

PTS (DSn [WDS]) = PTS (DSn [PCS])-90000 × ΣSIZE (WDS.WIN [i]) // 256,000,000

Since the PTS indicated in the WDS is a deadline, the writing of the graphics plane may be started at an earlier point. That is, as shown in FIG. 19, when decoding of the ODS to be displayed in one of the two windows is completed, writing of the graphics object obtained by decoding may be started from that point.

As described above, a window can be assigned to an arbitrary time point on the playback time axis of the AVClip using the DTS and PTS added to the WDS.
The above is the explanation of DTS and PTS of PCS and WDS. Here, the PCS becomes active from the time indicated in the DTS to the time indicated in the PTS. The period when the PCS is active is called the PCS active period of the Display Set.

Next, overlap of active periods of PCS in Display Set will be described. When multiple Display Set processes are required, there is a desire to parallelize these processes. In order for the playback apparatus to perform such parallelization, the active periods of the PCS must be overlapped.
However, in the Blu-ray Disc Read-Only standard, it must be ensured that it can be decoded by a playback device having the minimum necessary configuration.

The Bluy-ray Disc Read-Only standard decoder model is a decoder model (pipeline decoder model) based on pipeline processing. The pipeline decoder model refers to a decoder model that can read a graphics object belonging to a certain Display Set from the object buffer and simultaneously write a graphics object belonging to the next Display Set to the object buffer.

  When the playback device is a pipeline decoder model, the input interval becomes a problem. Here, the input interval is an interval from the start of processing for the previous Display Set to the start of processing for the next Display Set. Here, of the processing for one Display Set, the one related to the object buffer is a process of decoding ODS and writing uncompressed graphics to the object buffer, and reading uncompressed graphics from the object buffer and writing to the graphics plane. There are two. Then, the breakdown of one Display Set during the PCS active period is as shown in FIG. As shown in FIG. 20, the breakdown of processing of one Display Set is “period required to decode ODS and write it to object buffer”, “period required to read graphics from object buffer and write to graphics plane” It consists of.

  In the pipeline decoder model, graphics writing to the object buffer and data reading from the object buffer can be performed at the same time. Therefore, the processing for the two Display Sets can be parallelized as shown in FIG. FIG. 21 is a diagram showing how processing for two Display Sets (DSn, DSn + 1) is parallelized in a pipeline decoder model.

As shown in this figure, the processing for DSn and DSn + 1 so that the “reading period from the object buffer” in the processing for DSn and the “writing period to the object buffer” in the processing for DSn + 1 overlap. Is parallelized.
The parallelization as shown in FIG. 21 means that the graphics object belonging to DSn is written to the object buffer, and the graphics object belonging to DSn + 1 is written to the object buffer after the completion of the writing.

The decoding end time of the ODS belonging to DSn is indicated in the PTS of the END segment, and the ODS belonging to DSn + 1 starts the earliest at the time indicated in DTS (DSn + 1 [PCS]) So

PTS (DSn [END]) ≦ DTS (DSn + 1 [PCS])

The time stamp for the DSn END segment and the time stamp for the DSn + 1 PCS are set so that Execution with a pipeline model becomes possible by providing the input interval in this way.

  When the PCS active periods for the three Display Sets are overlapped as shown in FIG. 22, how to set the time stamp of each functional segment belonging to each Display Set on the playback time axis will be described. . FIG. 23 is a diagram showing the setting for the time stamp of each functional segment belonging to each Display Set. In this figure, the DTS values of WDS, PDS, and ODS belonging to DS0 are set to be the same as the DTS value of PCS. This is intended to start decoding ODS immediately after the start of the Display SetPCS active period. Therefore, the decoding of the first ODS1 belonging to DS0 starts at the time indicated by DTS (PCS). The decoding of the last ODSn belonging to DS0 ends at the time PTS (END) indicated in the END segment. Note that the DTS values of WDS, PDS, and ODS belonging to DS0 may indicate a time point later than the DTS value of PCS.

On the other hand, since the DTS value of PCS1 belonging to DS1 indicates the time point after PTS (END), even if decoding of ODS belonging to DS1 is started at the same time as DTS (PCS1), these two Display Processing for Set can be parallelized in the pipeline.
Next, let us consider parallelization of writing processing to the graphics plane. When the process for DSn and the process for DSn + 1 are parallelized, the graphics object obtained by the process for DSn + 1 and the graphics object obtained by the process for DSn are simultaneously written to the graphics plane. There may be a phenomenon that a graphics object belonging to DSn does not appear on the display.

Therefore, PTS is added to the PCSs belonging to DSn and DSn + 1 so that the following relationship is established between DSn and DSn + 1.

PTS (DSn [PCS]) + (90000 × ΣSIZE (DSn [WDS] .window [i])) // 256,000,000
≦ PTS (DSn + 1 [PCS])

Where ΣSIZE (WDS.WIN [i]) is the total size of WDS [i]
(90000 × ΣSIZE (DSn [WDS] .window [i])) // 256,000,000 is the time required for redrawing. By delaying the display time of DSn + 1, it is possible to avoid a situation where a graphics object belonging to DSn + 1 overwrites a graphics object belonging to DSn. FIG. 24 is a diagram showing the PTS interval by a mathematical expression.

When the window size is 1/4 of the graphics plane, the interval between PTS (DSn [PCS]) and PTS (DSn + 1 [PCS]) is the video display period in the above formula. .
Next, the duplication prohibited items will be described. The PCS active period is not allowed to overlap when the graphics object belonging to a certain Display Set has the same object_id as the graphics object belonging to the preceding Display Set and the graphics object is scheduled to be updated. . Here, it is assumed that there is an ODSA with ID = 1 added to DS0, and an ODS with object_id = 1 added also to DS1.

  If there are overlapping PCS active periods, the DS1 ODS is loaded into the playback device and decoded by the end of DS0. Then, since the DS0 graphics object is overwritten by the DS1 graphics object, the graphics object belonging to DS1 is displayed instead of the graphics object in DS0 when the DS0 is displayed. To prevent such a situation, if the graphics object is scheduled to be updated, it is prohibited to overlap the PCS active period of the Display Set.

  FIGS. 25 (a) and 25 (b) are diagrams showing a case where two DS pipelines are recognized and a case where they are not. In this figure, DS0 has ODS, and DS1 has ODS. . When the same object_id as ODSA, B belonging to DS0 is assigned to ODSC, D belonging to DS1, duplication of two Display Sets is recognized as shown in FIG. On the other hand, when object_id different from ODSA, B belonging to DS0 is assigned to ODSC, D belonging to DS1, duplication of two Display Sets is prohibited as shown in FIG.

  This prohibition can be avoided by “advanced ODS transmission”. "ODS transmission advance" means assigning an object_id different from the graphics object to be updated to ODS, storing the graphics object corresponding to ODS in the object buffer, and then adding the object_id to this graphics object afterwards Is changed, and the graphics object previously stored in the object buffer is overwritten. If such a technique is taken, there is no application of the prohibited items described above. A graphics object that overwrites the graphics object can be loaded into the object buffer without waiting for the display of the graphics object previously loaded into the object buffer.

  In the update, the advance of such transmission is effective, so not only the ODS that is referenced by its own PCS but also the ODS that is referenced by the PCS of the later Display Set may be transmitted together in one Display Set. Become more. However, when an ODS that is not referred to by the PCS is transmitted, when loading a plurality of ODSs into the playback device, the boundary of how much belongs to the previous Display Set and how much belongs to the later Display Set becomes ambiguous. Therefore, in the Display Set, the END segment is placed after the ODS belonging to itself. By doing so, the playback device can know the end of transmission of the ODS belonging to one Display Set by referring to the END segment.

  FIG. 26 is a diagram for explaining the end of transmission by the END segment. The first level in the figure shows functional segments belonging to one Display Set, and the second level shows how the functional segments are arranged on the BD-ROM. Functional segments such as PCS, WDS, PDS, and ODS are converted into TS packets and recorded on a BD-ROM together with the video stream converted into TS packets.

At this time, TS packets corresponding to the functional segment and TS packets corresponding to the video stream are respectively added with time stamps called ATS and PCS. TS packets corresponding to the functional segment and TS packets corresponding to the video stream are arranged on the BD-ROM so that those having the same time stamp are arranged in order.
Then, PCS, WDS and PDS belonging to Display Set do not exist continuously on BD-ROM, and TS packets ("V" in the figure) corresponding to the video stream are inserted between them. Become. As a result, functional segments appear on the BD-ROM. If TS packets corresponding to a functional segment appear on the BD-ROM, it is impossible to distinguish how many TS packets belong to one Display Set. Also, the Display Set includes things that are not referenced by your PCS, so it is even more difficult to distinguish. However, in this embodiment, since the END segment is inserted after the ODS, it is easy to distinguish how far the ODS belongs to itself even if the functional segments that make up the Display Set appear suddenly. Get on.

  FIG. 27 summarizes the relationship between overlapping PCS active periods and object_id assignment. Hereinafter, FIG. 27A will be described. This figure assumes four Display Sets (DS0, DS1, DS2, DS3). DS0 is written to display nothing. The DS1 PCS displays object X and object Y on the screen, the DS2 PCS displays object A, object Y, and object C, and the DS3 PCS displays object D on the screen.

  FIG. 27B is a diagram showing the ODSs belonging to each Display Set and the active period of the PCS of each Display Set. DS0 includes an ODS that configures object X, DS1 includes an ODS that configures object Y, DS2 includes an ODS that configures object A, object B, and object C, and DS3 includes an ODS that configures object D. The reason why the displayed graphics object does not match the transmitted graphics object is that the transmission is intended to be advanced.

The active periods of the four Display Set PCSs are in the form of overlapping each other. On the other hand, the arrangement of each graphics object in the object buffer is as shown in FIG.
Assume that identifiers 0, 1, 2, 3, and 4 are assigned to these objects X and Y and objects A to C, respectively.

In the four Display Sets as described above, what kind of object_id is possible for the object D belonging to the last Display Set (DS3) will be considered.
At this time, object_id that can be assigned to DS1 belonging to DS3 is 5, 3, 0.
5 is assignable because object_id = 5 is not assigned in DS0 to DS2.
The reason why 3 is assignable is that the object B to which 3 is assigned is not referred to by any Display Set PCS included in DS2.

0 can be assigned because object X with object_id = 0 is displayed in DS1, and even if the same object_id is assigned if the active period of the DS1 PCS has passed, This is because there is no inconvenience that the object D is displayed instead.
Conversely, object_id (1,2,4) other than these cannot be assigned to object D. If assigned, the object D is displayed instead of any of the three graphics objects (object A, object Y, object C) to be displayed in the DS2.

In this way, for graphics objects that are not referenced in the PCS active period of the Display Set with overlapping PCS active periods, or for graphics objects that are referenced in the PCS of the Display Set that have already ended the PCS active period It becomes possible to add the same object_id.
The overlap of PCS active periods of the Display Set is based on the premise that DSn and DSn + 1 belong to the same Epoch in the graphics stream. If two Display Sets belong to different Epochs, the PCS active periods of these two Display Sets cannot be overlapped. This is because if the DSn + 1 PCS or ODS is read by the end of the DSn PCS active period, the object buffer and graphics plane cannot be flushed at the end of DSn.

If DSn is the last DS belonging to EPOCHm (this is called EPOCHm DSlast [PCS]) and DSn + 1 is the first DS belonging to EPOCHm + 1 (this is called EPOCHm + 1 DSfirst [PCS]), these The PTS value for PCS must be set to satisfy the following formula:

PTS (EPOCHm DSlast [PCS]) ≤ DTS (EPOCHm + 1 DSfirst [PCS])


The overlap of PCS active periods included in the Display Set is based on the premise that the graphics stream is a presentation graphics stream. In addition to the presentation graphics stream, there is an interactive graphics stream whose main purpose is interactive display.

When two consecutive Display Sets (DSn, DSn + 1) belong to an interactive graphics stream, no overlap is observed and the DSn + 1 PCS active period starts after the DSn PCS active period. Time information must be set to begin. In the interactive graphics stream, a segment storing control information is called an interactive composition segment. The end point of the DSn PCS active period is indicated by the PTS value of the ICS belonging to DSn, and the start point of the DSn + 1 PCS active period is indicated by the DTS value of the ICS belonging to DSn + 1. Here, PTS (DSn [ICS]) and DTS (DSn + 1 [ICS]) must be set to satisfy the following equations.

PTS (DSn [ICS]) ≤ DTS (DSn + 1 [ICS])
This completes the explanation of the overlap of PCS active periods.

  The data structure of the Display Set (PCS, WDS, PDS, ODS) described so far is an instance of a class structure described in a programming language, and authors who perform authoring use the Blu-ray Disc Read Only Format. These data structures on the BD-ROM can be obtained by describing the class structure according to the prescribed syntax. The above is the embodiment of the recording medium according to the present invention. Next, an embodiment of a playback apparatus according to the present invention will be described. FIG. 28 shows the internal structure of the playback apparatus according to the present invention. The reproducing apparatus according to the present invention is industrially produced based on the inside shown in the figure. The playback device according to the present invention is mainly composed of three parts, a system LSI, a drive device, and a microcomputer system, and can be industrially produced by mounting these parts on the cabinet and board of the device. The system LSI is an integrated circuit in which various processing units that fulfill the functions of the playback device are integrated. The reproduction apparatus thus produced includes a BD drive 1, a Read Buffer 2, a PID filter 3, Transport Buffers 4a, b, c, a peripheral circuit 4d, a video decoder 5, a video plane 6, an audio decoder 7, a graphics plane 8, a CLUT unit 9, An adder 10, a graphics decoder 12, a Coded Data Buffer 13, a peripheral circuit 13a, a Stream Graphics Processor 14, an Object Buffer 15, a Composition Buffer 16, and a Graphical Controller 17 are included.

The BD-ROM drive 1 performs loading / reading / ejecting of the BD-ROM, and executes access to the BD-ROM.
The Read Buffer 2 is a FIFO memory, and TS packets read from the BD-ROM are stored in a first-in first-out manner.
The PID filter 3 performs filtering on a plurality of TS packets output from the Read Buffer 2. Filtering by the PID filter 3 is performed by writing only TS packets having a desired PID in the Transport Buffers 4a, b, and c. In the filtering by the PID filter 3, buffering is not necessary. Therefore, the TS packet input to the PID filter 3 is written in the transport buffers 4a, b, and c without time delay.

The Transport Buffers 4a, b, and c are memories that store TS packets output from the PID filter 3 in a first-in first-out manner. The peripheral circuit 4d having the speed Rx at which the TS packet is extracted from the Transport Buffer 4a is a wire logic that performs processing for converting the TS packet read from the Transport Buffer 4a, b, c into a functional segment. The functional segment obtained by the conversion is stored in the Coded Data Buffer 13.

The video decoder 5 decodes the plurality of TS packets output from the PID filter 3 to obtain an uncompressed picture and writes it to the video plane 6.
The video plane 6 is a plane memory for moving images.
The audio decoder 7 decodes the TS packet output from the PID filter 3 and outputs uncompressed audio data.

The graphics plane 8 is a plane memory having an area for one screen, and can store uncompressed graphics for one screen.
The CLUT unit 9 converts the index color in the uncompressed graphics stored in the graphics plane 8 based on the Y, Cr, and Cb values indicated in the PDS.
The adder 10 multiplies the uncompressed graphics color-converted by the CLUT unit 9 by the T value (transmittance) indicated in the PDS and adds the uncompressed picture data stored in the video plane 6 for each pixel. To obtain and output a composite image.

The graphics decoder 12 decodes the graphics stream to obtain uncompressed graphics, and writes this into the graphics plane 8 as a graphics object. By decoding the graphics stream, subtitles and menus appear on the screen.
The pipeline by the graphics decoder 12 is executed by simultaneously executing a process of writing a graphics object belonging to DSn into the Object Buffer 15 and a process of reading out a graphics object belonging to DSn + 1 from the Object Buffer 15.

The graphics decoder 12 includes a Coded Data Buffer 13, a peripheral circuit 13a, a Stream Graphics Processor 14, an Object Buffer 15, a Composition Buffer 16, and a Graphical Controller 17.
The Coded Data Buffer 13 is a buffer in which functional segments are stored together with DTS and PTS. Such a functional segment is obtained by removing the TS packet header and the PES packet header from each TS packet of the transport stream stored in the Transport Buffer 4a, b, c, and sequentially arranging the payload. Of the removed TS packet header and PES packet header, PTS / DTS is stored in association with the PES packet.

  The peripheral circuit 13 a is wire logic that realizes transfer between the Coded Data Buffer 13 and the Stream Graphics Processor 14 and transfer between the Coded Data Buffer 13 and the Composition Buffer 16. In this transfer process, when the current time is the time indicated by the DTS of the ODS, the ODS is transferred from the Coded Data Buffer 13 to the Stream Graphics Processor 14. If the current time is the time indicated by the DTS of the PCS and PDS, a process of transferring the PCS and PDS to the Composition Buffer 16 is performed.

  The Stream Graphics Processor 14 decodes the ODS, and writes uncompressed uncompressed graphics composed of the index colors obtained by the decoding into the Object Buffer 15 as a graphics object. Decoding by the Stream Graphics processor 14 is performed instantaneously, and the Stream Graphics processor 14 temporarily holds the graphics object by decoding. Decoding by the Stream Graphics processor 14 is performed instantaneously, but writing from the Stream Graphics processor 14 to the Object Buffer 15 does not end instantaneously. This is because in the BD-ROM standard player model, writing to the Object Buffer 15 is performed at a transfer rate of 128 Mbps. Since the writing completion time to the Object Buffer 15 is indicated in the PTS of the END segment, processing for the next DS is awaited until the time indicated in the PTS of the END segment elapses. Writing of the graphics object obtained by decoding each ODS starts at the time of the DTS associated with the ODS and ends by the decoding end time indicated in the PTS associated with the ODS.

  When the object_id assigned to the DSn-side graphics data and the DSn + 1-side graphics data is different at the time of writing, the Stream Graphics processor 14 uses the DSn-side graphics data and the DSn + 1-side graphics. Data is written to separate areas in the Object Buffer 15. Thus, the graphics object referred to by the DSn PCS is displayed in the pipeline without being overwritten by the graphics object belonging to DSn + 1. When the object_id assigned to both graphics objects is the same, the Stream Graphics processor 14 sets the graphics data on the subsequent DS side in the same area as the area where the graphics data on the preceding DS side is stored in the Object Buffer 15. Is overwritten. In such a case, the pipeline is not performed. In addition, graphics objects belonging to a DS include those that are referenced by the PCS of the same DS and those that are not referenced. The Stream Graphics processor 14 sequentially decodes not only the graphics objects referenced by the PCS but also the graphics objects not referenced, and stores the graphics obtained by the decoding in the Object Buffer 15.

  The Object Buffer 15 is a buffer corresponding to a pixel buffer in the ETSI EN 300 743 standard, and a graphics object obtained by decoding by the Stream Graphics Processor 14 is arranged. The Object Buffer 15 must be set to a size twice or four times that of the graphics plane 8. This is because, considering the case of realizing Scrolling, it is necessary to store twice and four times as many graphics objects as the graphics plane 8.

The composition buffer 16 is a memory in which PCS and PDS are arranged. --If there are two Display Sets to be processed and the active periods of these PCS overlap, a plurality of PCSs to be processed are stored in the Composition buffer 16.
The Graphical Controller 17 decodes the PCS, and executes writing of the graphics object to the Object Buffer 15, reading of the graphics object from the Object Buffer 15, and display of the graphics object according to the PCS decoding result. The display by the Graphical controller 17 is executed at the time indicated by the PTS of the PES packet storing the PCS. The interval from the display of the graphics object belonging to DSn by the Graphical controller 17 to the display of the graphics object belonging to DSn + 1 is as described above.

  Next, the recommended values for the transfer rate and buffer size for configuring the PID filter 3, the transport buffers 4a, b, c, the graphics plane 8, the CLUT unit 9, and the Coded Data Buffer 13 to the Graphical Controller 17 will be described. FIG. 29 is a diagram showing the sizes of the write rates Rx, Rc, Rd, the graphics plane 8, the transport buffer 4a, the coded data buffer 13, and the object buffer 15.

The transfer rate Rc between the Object Buffer 15 and the graphics plane 8 is the highest transfer rate in this apparatus, and is calculated as 256 Mbps (= 500 Kbytes × 29.97 × 2) from the window size and the frame rate.
Unlike the Rc, the transfer rate Rd (Pixel Decoding Rate) between the Stream Graphics Processor 14 and the Object Buffer 15 is not required to be updated according to the period of the video frame, and may be 1/2 or 1/4 of Rc. Therefore, it becomes 128Mbps, 64Mbps.

The Transport Buffer Leak rate Rx between the Transport Buffers 4a, b, and c-Coded Data Buffer 13 is the ODS transfer rate in a compressed state. Therefore, the transport buffer leak rate may be a transfer rate obtained by multiplying Rd by the compression rate. If the ODS compression rate is assumed to be 25%, 16Mbps (= 64Mbps x 25%) is sufficient.
The transfer rate and buffer size shown in this figure are minimum standards, and implementation with values larger than this is not denied.

In the playback apparatus configured as described above, each component can perform a decoding process in a pipeline manner.
FIG. 30 is a timing chart showing pipeline processing by the playback device. The fifth row shows the Display Set in the BD-ROM, and the fourth row shows the PCS, WDS, PDS, and ODS reading period to the Coded Data Buffer 13. The third row shows the decoding period of each ODS by the Stream Graphics Processor 14, the second row shows the contents stored in the Composition Buffer 16, and the first row shows the processing contents of the Graphical Controller 17.

  The DTS (decoding start time) assigned to the ODSs 1 and 2 indicates the time points t31 and t32 in the figure. Since the decoding start time is defined in the DTS, each ODS must be read into the Coded Data Buffer 13 by the time indicated in its own DTS. Therefore, the reading of ODS1 is completed immediately before the decoding period dp1 of ODS1 to Coded Data Buffer 13. The reading of ODS2 into the Coded Data Buffer 13 is completed immediately before the decoding period dp2 of ODS2.

  On the other hand, the PTS (decoding end time) assigned to ODS 1 and 2 indicates the time points t32 and t33 in the figure. The decoding of ODS1 by the Stream Graphics Processor 14 is completed by t32, and the decoding of ODS2 is completed by the time indicated by t33. As described above, the Stream Graphics Processor 14 reads the ODS into the Coded Data Buffer 13 by the time indicated by the DTS of each ODS, and the ODS read by the Coded Data Buffer 13 by the time indicated by the PTS of each ODS. Decode and write to Object Buffer 15.

  A period cd1 in the first row in the figure is a period required for the Graphics Controller 17 to clear the graphics plane 8. The period td1 is a period required to write the graphics object obtained on the Object Buffer 15 to the graphics plane 8. The PTS of the WDS indicates a deadline at the start of the writing, and the PTS of the PCS indicates the end point of the writing and the display timing. At the time indicated by the PCS PTS, the uncompressed graphics constituting the interactive screen are obtained on the graphics plane 8. If the color conversion of the uncompressed graphics is performed by the CLUT unit 9 and the uncompressed picture stored in the video plane 6 is combined by the adder 10, a combined image can be obtained.

In the graphics decoder 12, the decoding of the Stream Graphics Processor 14 is continued even while the Graphics Controller 17 is executing the clearing of the graphics plane 8. With the pipeline processing as described above, it is possible to quickly display graphics.
In FIG. 30, it is assumed that the clearing of the graphics plane ends earlier than the decoding of the ODS, but FIG. 31 is a timing showing pipeline processing assuming that the decoding of the ODS ends earlier than the clearing of the graphics plane. It is a chart. In this case, when the ODS decoding is completed, writing to the graphics plane cannot be performed, and when the graphics plane is cleared, the graphics obtained by decoding are written to the graphics plane. Can do.

  The temporal transition of the buffer occupation amount in the playback device will be described with reference to FIG. FIG. 32 is a timing chart showing temporal transitions of the Composition buffer 16, Object Buffer 15, Coded Data buffer 13, and graphics plane 8 in FIG. This figure shows temporal transitions of occupation amounts in the graphics plane 8, the Object Buffer 15, the Coded Data buffer 13, and the Composition buffer 16 from the first stage to the fourth stage. The temporal transition of the occupation amount is expressed by a line graph notation with the horizontal axis as the time axis and the vertical axis as the occupation amount.

The fourth row in FIG. 32 shows the temporal transition of the occupation amount in the composition buffer 16. As shown in the figure, the temporal transition of the Composition buffer 16 includes a monotonically increasing vf0 that is output from the Coded Data buffer 13 and stores the PCS.
The third level shows the temporal transition of the occupation amount in the Coded Data buffer 13. As shown in the figure, the temporal transition of the Coded Data buffer 13 includes monotonically increasing Vf1 and Vf2 due to the ODS being stored, and monotonically decreasing Vg1 and Vg2 due to the stored ODS being sequentially extracted by the Stream Graphics processor 14. Including. The slopes of monotonically increasing Vf1, Vf2 are based on the output rate Rx from the Transport buffers 4a, b, c to the Coded Data buffer 13, and the slopes of monotonically decreasing Vg1, Vg2 are decoded by the Stream Graphics processor 14 and executed instantaneously. Is done. That is, the ODS is decoded instantaneously, and the Stream Graphics processor 14 holds the uncompressed graphics obtained by the decoding. Since the writing rate of the transmission path from the Stream Graphics processor 14 to the Object Buffer 15 is 128 Mbps, the occupation amount of the Object Buffer 15 increases by this writing rate.

  The second level shows the temporal transition of the occupation amount in the Object Buffer 15. As shown in the figure, the temporal transition of the Object Buffer 15 includes monotonically increasing Vh1 and Vh2 due to the ODS output from the Stream Graphics processor 14 being stored. The slopes of the monotonically increasing Vh1 and Vh2 are based on the transfer rate Rd from the Stream Graphics processor 14 to the Object Buffer 15. The period in which the monotonic decrease in the third stage and the period in which the monotonous increase in the second stage occur are decoding periods. The beginning of this decoding period is shown in the OTS DTS, and the end of the decoding period is shown in the ODS PTS. If the uncompressed graphics is stored in the object buffer 15 by the period indicated in the PTS of the ODS, the decoding for the ODS is completed. It is an essential requirement that uncompressed graphics be stored in the object buffer 15 by the period shown in the PTS of the ODS, and any monotonic increase or monotonic decrease in this decoding period may be used.

  The first level shows the temporal transition of the occupation amount in the graphics plane 8. As shown in the figure, the temporal transition of the graphics plane 8 includes a monotonically increasing Vf3 due to storing the decoded ODS output from the Object Buffer 15. The slope of the monotonically increasing Vf3 is based on the transfer rate Rc from the Object Buffer 15 to the graphics plane 8. The end of this monotonic increase is indicated in the PCS PTS.

By using the DTS and PTS assigned to the ODS, the DTS and PTS assigned to the ICS, and the size and transfer rate of each buffer shown in FIG. 29, the BD-ROM It becomes clear at the authoring stage how the state of each buffer changes during the playback of an AVClip to be supplied in the above.
This buffer state transition can be adjusted by rewriting the values shown in DTS and PTS. Therefore, it is possible to avoid the occurrence of a decoding load that exceeds the specifications of the decoder on the playback device side, Overflow can be avoided. This simplifies the implementation of hardware and software in developing the playback device.

The above is the internal configuration of the playback device. Next, how the control unit 20 and the graphics decoder 12 are mounted will be described. The control unit 20 can be implemented by creating a program for performing the processing procedure of FIG. 33 and causing the CPU to execute the program. Hereinafter, the processing procedure of the control unit 20 will be described with reference to FIG.
FIG. 33 is a flowchart of a process procedure of a function segment loading process. In this flowchart, SegmentK is a variable that means each of the read segments (PCS, WDS, PDS, ODS) during AVClip playback, and the ignore flag indicates whether to ignore or load this SegmentK. It is a flag to switch. This flowchart has a loop structure in which the process of steps S21 to S24 and steps S27 to S31 is repeated for all Segments K after the ignore flag is initialized to 0 (steps S25 and S26).

Step S21 is a determination of whether or not SegmentK is a PCS. If SegmentK is a PCS, the determinations of Steps S27 and S28 are performed.
Step S22 is to determine whether or not the ignore flag is 1. If the ignore flag is 0, the process proceeds to step S23, and if it is 1, the process proceeds to step S24. If the ignore flag is 1 (Yes in step S22), SegmentK is loaded into the Coded Data Buffer 13 in step S23.

If the ignore flag is set to 1 (No in step S22), SegmentK is ignored in step S24. As a result, all functional segments belonging to DS are ignored because step S22 is No (step S24).
Thus, whether SegmentK is ignored or loaded is determined by the setting of the ignore flag. Steps S27 to S31, S34, and S35 are processes for setting the ignore flag.

Step S27 is a determination as to whether or not composition_type in PCS is Acquisition Point. If SegmentK is Acquisition Point, the process proceeds to step S28, and if SegmentK is Epoch Start or Normal Case, the process proceeds to step S31.
Step S28 is a determination as to whether or not the preceding DS exists in any of the buffers (Coded Data Buffer 13, Stream Graphics processor 14, Object Buffer 15, Composition Buffer 16) in the graphics decoder 12, and if Step S27 is Yes To be executed. The case where there is no DS in the graphics decoder 12 is a case where cueing has been performed. In this case, since the display must be started from the Acquisition Point DS, the process proceeds to Step S30 (No in Step S28).

In step S30, the ignore flag is set to 0, and the process proceeds to step S22.
The case where the DS exists in the graphics decoder 12 refers to a case where normal reproduction is performed. In this case, the process proceeds to step S29 (Yes in step S28). In step S29, the ignore flag is set to 1, and the process proceeds to step S22.
Step S31 is determination of whether composition_type in PCS is Normal Case. If it is Normal Case, the process proceeds to step S34. If SegmentK is Epoch Start, the ignore flag is set to 0 in step S30. Step S34 is the same as step S28, and it is determined whether or not a preceding DS exists in the graphics decoder 12. If it exists, the ignore flag is set to 0 (step S30). If it does not exist, since a sufficient functional segment constituting the dialog screen cannot be obtained originally, the ignore flag is set to 1 (step S35). With this flag setting, when the preceding DS does not exist in the graphics decoder 12, the functional segments constituting the Normal Case are ignored.

Assuming that the DS is multiplexed as shown in FIG. 34, how the DS is read will be described. In the example of FIG. 34, three DSs are multiplexed with a moving image. Of these three DSs, the first DS1 has composition_type of Epoch_Start, DS10 of Acquisition Point, and DS20 of Normal Case.
It is assumed that cueing from the picture data pt10 is performed as indicated by an arrow am1 in the AVClip in which the three DSs are multiplexed with the moving image. In this case, the DS 10 closest to the cueing position is the target of the flowchart of FIG. In step S27, composition_type is determined to be Acquisition Point, but since the preceding DS does not exist on Coded Data Buffer 13, the ignore flag is set to 0, and this DS10 is Coded of the playback device as indicated by arrow md1 in FIG. The data buffer 13 is loaded. On the other hand, when the cueing position is after the position where DS10 exists (arrow am2 in FIG. 34), DS20 is the Display Set of Normal Case, and the preceding DS20 does not exist in Coded Data Buffer 13, so this Display Set is Will be ignored (arrow md2 in FIG. 35).

  The load of DS1, 10, 20 when normal reproduction is performed as shown in FIG. 36 is as shown in FIG. Of the three DSs, DS1 whose PCS composition_type is Epoch Start is loaded as it is into the Coded Data Buffer 13 (step S23), but for DS10 whose PCS composition_type is Acquisition Point, the ignore flag is set to 1. For this reason (step S29), the functional segments constituting this are not loaded into the Coded Data Buffer 13 but are ignored (arrow rd2, step S24 in FIG. 37). For DS20, since PCS composition_type is Normal Case, it is loaded into Coded Data Buffer 13 (arrow rd3 in FIG. 37).

Next, the processing procedure of the graphical controller 17 will be described. 38 to 40 are flowcharts showing the processing procedure of the graphical controller 17.
Steps S41 to S44 are the main routine of this flowchart, and wait for the establishment of any event defined in steps S41 to S44.
Step S41 is a determination as to whether or not the current playback time is the DTS time of the PCS, and if so, the processing of steps S45 to S53 is performed.

  In step S45, it is determined whether or not the composition_state of the PCS is Epoch_Start. If it is Epoch Start, the graphics plane 8 is completely cleared in step S46. Otherwise, in step S47, the windows indicated by the window_horizontal_position, window_vertival_position, window_width, and window_height of the WDS are cleared.

  Step S48 is a step that is executed after clearing in step S46 or step S47, and is a determination of whether or not the PTS time of any ODSx has already passed. That is, when clearing the entire graphics plane 8, it takes a long time to clear the graphics plane 8, so that decoding of a certain ODS (ODSx) may already be completed. Step S48 verifies the possibility. If not, return to the main routine. If the decoding time of any ODS has elapsed, steps S49 to S51 are executed. Step S49 is a determination as to whether or not object_cropped_flag indicates 0. If it indicates 0, the graphics object is not displayed (step S50).

  If 0 is not indicated, the graphics object cropped based on object_cropping_horizontal_position, object_cropping_vertival_position, cropping_width, and cropping_height is written in the position indicated by object_horizontal_position and object_vertival_position in the window of the graphics plane 8 (step S51). With the above processing, one or more graphics objects are drawn on the window.

Step S52 is a determination as to whether another PTS time of ODSy has elapsed. When writing ODSx to the graphics plane 8, if decoding of another ODS has already been completed, this ODSy is set to ODSx (step S53), and the process proceeds to step S49. As a result, the processing of steps S49 to S51 is repeated for another ODS.
Next, step S42 and steps S54 to S59 will be described with reference to FIG.

  Step S42 is a determination as to whether or not the current playback time is a PTS of WDS. If it is a PTS of WDS, it is determined whether or not there is one window in step S54. If there is, return to the main routine. If there is only one window, the loop processing from step S55 to step S59 is performed. This loop processing is to execute steps S57 to S59 for each of the two graphics objects displayed in the window. Step S57 is a determination as to whether or not object_cropped_flag indicates 0. If it indicates 0, the graphics object is not displayed (step S58).

  If 0 is not indicated, the graphics object cropped based on object_cropping_horizontal_position, object_cropping_vertival_position, cropping_width, and cropping_height is written in the position indicated by object_horizontal_position and object_vertival_position in the window of the graphics plane 8 (step S59). If the above processing is repeated, a maximum of two graphics objects are drawn on the window.

Step S44 is a determination as to whether or not the current playback time is a time indicated in the PTS of the PCS. If so, it is determined whether or not Pallet_update_flag indicates 1 in step S60. If 1 is indicated, the PDS indicated by pallet_id is set in the CLUT part (step S61). If 0 is indicated, step S61 is skipped.
After that, the color conversion of the graphics object in the graphics plane 8 is performed by the CLUT unit and synthesized with the moving image (step S62).

Next, step S43 and steps S64 to S66 will be described with reference to FIG.
Step S43 is a determination as to whether or not the current playback time is an ODS PTS. If it is an ODS PTS, it is determined whether or not there are two windows in step S63. If there is, return to the main routine.

  The determinations in step S43 and step S63 have the following meanings. That is, when there are two windows, one graphics object is displayed on each window. Then, every time decoding of each ODS is completed, it is necessary to write the graphics object obtained by decoding to the graphics plane (example: case of FIG. 19B). Therefore, if the current time is the time indicated in the PTS of the ODS and there are two windows, Steps S64 to S66 are executed in order to write each graphics object on the graphics plane. Step S64 is a determination of whether or not object_cropped_flag indicates 0, and if so, the graphics object is not displayed (step S65).

  If 0 is not indicated, the graphics object cropped based on object_cropping_horizontal_position, object_cropping_vertival_position, cropping_width, and cropping_height is written in the position indicated by object_horizontal_position and object_vertival_position in the window of the graphics plane 8 (step S66). If the above processing is repeated, a graphics object is drawn on each window.

  As described above, according to the present embodiment, the processing for one Display Set is started in the middle of the PCS active period of the preceding Display Set. Instead, the processing of the subsequent Display Set can be started. The middle of the PCS active period is the time when the transfer of the decoded graphics is completed in the preceding Display Set. The processing of the subsequent Display Set is advanced only during the period from the completion point to the end point of the PCS active period.

  Even if the processing for the subsequent Display Set starts in the middle of the PCS active period of the preceding Display Set, the graphics object belonging to the preceding Display Set is stored in the object buffer during the period in which the graphics object belonging to the subsequent Display Set is written to the object buffer. There is no overlap with the period of writing. Therefore, if the object buffer is a dual port memory and simultaneous read / write is possible, even if there is one Stream Graphics processor 14 that decodes graphics, processing for two or more Display Sets is pipelined. Can be performed on expressions. Such pipelined execution can increase the efficiency of the decoding process without complicating the internal configuration.

(Second embodiment)
The present embodiment is an embodiment relating to a BD-ROM manufacturing process. FIG. 41 is a diagram showing a manufacturing process for creating the PCS shown in the first embodiment.
The BD-ROM production process consists of a material production process S201 for creating materials such as video recording and audio recording, an authoring process S202 for generating an application format using an authoring device, and a BD-ROM master, It includes a pressing step S203 for performing bonding to complete a BD-ROM.

Among these processes, the authoring process for BD-ROM includes the following steps S204 to S213.
In step S204, control information, window definition information, palette definition information, and graphics are described. In step S205, the control information, window definition information, palette definition information, and graphics are converted into functional segments. In step S206, the PCS PTS is set based on the appearance timing of the picture to be synchronized. In step S207, DTS [ODS] and PTS [ODS] are set based on the value of PTS [PCS]. In step S208, DTS [PCS], PTS [PDS], DTS [WDS], and PTS [WDS] are set based on the value of DTS [ODS], and in step S209, the time of occupation of each buffer in the player model is set. Graph transitions. In step S210, it is determined whether or not the graphed temporal transition satisfies the constraints of the player model. If not, the DTS and PTS of each functional segment are rewritten in step S211. If satisfied, a graphics stream is generated in step S212, and an AVClip is obtained by multiplexing the graphics stream with the separately generated video stream and audio stream in step S213. Thereafter, the application format is completed by adapting the AVClip to the BD-ROM format.

(Remarks)
The above description does not show all implementation modes of the present invention. The present invention can be implemented also by the form of the implementation act in which the following (A), (B), (C), (D). Each invention according to the claims of the present application is an extended description or a generalized description of the above-described embodiments and their modifications. The degree of expansion or generalization is based on the technical level characteristics of the [Technical Field] of the present invention at the time of filing.

  (A) In all the embodiments, the recording medium according to the present invention is implemented as a BD-ROM. However, the recording medium according to the present invention is characterized by a graphics stream to be recorded. It does not depend on the physical properties. Any recording medium that can record a graphics stream may be used. For example, it may be an optical disk such as DVD-ROM, DVD-RAM, DVD-RW, DVD-R, DVD + RW, DVD + R, CD-R, CD-RW, or a magneto-optical disk such as PD, MO. . Further, it may be a semiconductor memory card such as a compact flash (registered trademark) card, smart media, memory stick, multimedia card, and PCM-CIA card. It may be a flexible disk, a magnetic recording disk (i) such as SuperDisk, Zip, or Clik !, or a removable hard disk drive (ii) such as ORB, Jaz, SparQ, SyJet, EZFley, or a microdrive. Furthermore, a built-in hard disk may be used.

  (B) The playback device in all embodiments decodes the AVClip recorded on the BD-ROM and outputs it to the TV. However, the playback device is only a BD-ROM drive, and other components are the TV. In this case, the playback device and the TV can be incorporated into a home network connected by IEEE1394. In addition, the playback device in the embodiment is a type that is used by being connected to a television, but may be a playback device integrated with a display. Furthermore, in the playback device of each embodiment, only a system LSI (integrated circuit) that forms an essential part of the processing may be implemented. Since these playback devices and integrated circuits are the inventions described in this specification, playback is performed based on the internal configuration of the playback device shown in the first embodiment, regardless of which aspect. The act of manufacturing the device is an act of implementing the invention described in the specification of the present application. The act of transferring the playback device shown in the first embodiment for a fee or free of charge (sale if paid, or gift if free), lending or importing is also an implementation of the present invention. The act of offering this transfer or lending to a general user through store display, catalog solicitation, or pamphlet distribution is also an implementation of this playback device.

(C) Since the information processing by the program shown in each flowchart is specifically realized using hardware resources, the program showing the processing procedure in the flowchart is established as an invention as a single unit. Although all the embodiments have been described with respect to the implementation of the program according to the present invention in a mode incorporated in the playback device, the program alone shown in the first embodiment is implemented separately from the playback device. May be. The act of implementing the program alone includes the act of producing these programs (1), the act of transferring the program for a fee (2), the act of lending (3), the act of importing (4), There is an act of offering to the public through electronic communication lines (5), and an act of offering to the general user transfer or lending of the program by storefront, catalog solicitation, or pamphlet distribution.
(D) The “time” element of each step executed in time series in each flowchart is considered as an indispensable matter for specifying the invention. Then, it can be seen that the processing procedure by these flowcharts discloses the usage mode of the reproduction method. If the processing of these flowcharts is performed so that the original purpose of the present invention can be achieved and the operations and effects can be achieved by performing the processing of each step in time series, the recording according to the present invention is performed. Needless to say, this is an implementation of the method.

(E) When recording on a BD-ROM, it is desirable to add an extension header to each TS packet constituting an AVClip. The extension headers, which are called TP_extra_header, include an "Arrival _Time_Stamp", a data length of 4 bytes and a "copy_permission_indicator". TS packets with TP_extra_header (hereinafter abbreviated as TS packets with EX) are grouped every 32 and written to three sectors. A group consisting of 32 EX-attached TS packets is 6144 bytes (= 32 × 192), which matches three sector sizes of 6144 bytes (= 2048 × 3). 32 EX-attached TS packets stored in 3 sectors are called “Aligned Unit”.

When used in a home network connected via IEEE1394, the playback device transmits Aligned Units by the following transmission process. That is, the device on the sender side removes TP_extra_header from each of the 32 EX-attached TS packets included in the Aligned Unit, and encrypts and outputs the TS packet body based on the DTCP standard. When outputting TS packets, isochronous packets are inserted everywhere between TS packets. The insertion point is a position based on a time shown in Arrival _Time_Stamp the TP_extra_header. With the output of the TS packet, the playback device outputs DTCP_Descriptor. DTCP_Descriptor indicates copy permission / inhibition setting in TP_extra_header. If DTCP_Descriptor is described so as to indicate “copy prohibited”, TS packets are not recorded in other devices when used in a home network connected via IEEE1394.

(F) The digital stream in each embodiment is an AVClip of the BD-ROM standard, but may be a VOB (Video Object) of the DVD-Video standard or the DVD-Video Recording standard. The VOB is a program stream conforming to the ISO / IEC13818-1 standard obtained by multiplexing a video stream and an audio stream. The video stream in AVClip may be in MPEG4 or WMV format. Furthermore, the audio stream may be a Linear-PCM system, a Dolby-AC3 system, an MP3 system, an MPEG-AAC system, or a dts system.

(G) The movie work in each embodiment may be obtained by encoding an analog video signal broadcast by analog broadcasting. It may be stream data composed of a transport stream broadcast by digital broadcasting.
Further, the content may be obtained by encoding an analog / digital video signal recorded on a video tape. Further, the content may be obtained by encoding an analog / digital video signal taken directly from the video camera. In addition, a digital work distributed by a distribution server may be used.

  (H) The graphics object shown in the first to second embodiments is run-length encoded raster data. The reason why the run-length encoding method is adopted as the compression / encoding method for graphics objects is that run-length encoding is most suitable for compression / decompression of subtitles. Subtitles have a characteristic that the continuous length of the same pixel value in the horizontal direction is relatively long, and a high compression rate can be obtained by performing compression by run-length encoding. Also, the load for decompression is light and suitable for software for decryption processing. However, adopting a run-length encoding method for subtitles is not an essential matter of the present invention, and the graphics object may be PNG data. Further, it may be vector data instead of raster data, or a more transparent picture.

(I) The display effect by PCS may be subtitle graphics selected according to the language setting on the device side. As a result, it is possible to realize the display effect using characters as expressed in the main body of the moving image in the current DVD with the subtitle graphics displayed according to the language setting on the device side. Is big.
The target of the display effect by PCS may be subtitle graphics selected according to the display setting on the apparatus side. In other words, graphics for various display modes such as wide vision, pan scan, and letterbox are recorded on the BD-ROM, and the device chooses one of these according to the setting of the TV connected to itself. indicate. In this case, the display effect based on the PCS is applied to the subtitle graphics displayed in this manner, so that the appearance is improved. Thus, a display effect using characters as expressed in the moving image main body can be realized by subtitles displayed according to the display settings on the apparatus side, and thus has practical value.

  (J) In the first embodiment, the graphics plane write rate Rc is set to 25% of the entire window so that the graphics plane can be cleared and redisplayed in one video frame. Rc may be determined so that these clear / redraw operations are completed in the vertical blanking period. Assuming that the vertical blanking period is 25% of 1 / 29.93 seconds, Rc is 1 Gbps. By setting Rc in this way, graphics can be displayed smoothly, so the practical effect is great.

In addition to writing in the vertical blanking period, writing synchronized with the line scan may be used in combination. As a result, smooth display can be realized even at a write rate of Rc = 256 Mbps.
(K) Although the graphics plane is mounted on the playback device in each embodiment, a line buffer that stores uncompressed pixels for one line may be provided instead of the graphics plane. This is because the conversion to the video signal is performed for each horizontal line (line), so that the conversion to the video signal can be performed as long as the line buffer is provided.

(L) The graphic subtitles were explained as a character string representing the line of the movie, but the combination of figures, characters, colors that constitute a trademark, national emblem, flag, insignia, state It may include official symbols and seals for supervision / certification employed, emblems of international intergovernmental organizations, flags, insignia, and indications of origin of specific goods.
(M) In the first embodiment, subtitles are displayed horizontally on the upper and lower sides of the screen, and windows are defined on the upper and lower sides of the graphics plane. However, subtitles are displayed on the right and left sides of the screen. As a matter of course, a window may be defined on the right and left sides of the graphics plane. By doing this, Japanese subtitles can be displayed vertically.

(O) The graphics decoder 12 pipelines DSn and DSn + 1 when DSn and DSn + 1 belong to the same Epoch in the graphics stream, and the DSn and DSn + When 1 belongs to different Epochs, the processing for DSn + 1 is started after the graphics display in DSn is started.
In addition, the graphics stream includes a presentation type mainly for synchronization with a moving image and an interactive type mainly for interactive display. The graphics decoder has a graphics stream as a presentation type. In some cases, a pipeline expression for two DSs is performed, and when a graphics stream is an interactive system, a pipeline expression for two DSs is not performed.

  Although the above-described modifications can be made, each claimed invention reflects the means for solving the technical problems of the prior art, so the technical scope of each claimed invention is the conventional The technical problem solution of the technology does not exceed the technical scope recognized by those skilled in the art. Therefore, each invention according to the claims of the present application has a substantial correspondence with the description of the detailed description.

  The recording medium and the playback apparatus according to the present invention have the internal configuration disclosed in the above embodiment, and can be mass-produced based on the internal configuration, so that they can be industrially utilized in qualities. Therefore, the recording medium and the reproducing apparatus according to the present invention have industrial applicability.

It is a figure which shows the form about the usage act of the recording medium based on this invention. It is a figure which shows the structure of BD-ROM. It is a figure which shows typically how AVClip is comprised. (A) It is a figure which shows the structure of a presentation graphics stream.

(B) It is a figure which shows the PES packet obtained by converting a functional segment.
It is a figure which shows the logical structure comprised by various types of functional segments. It is a figure which shows the relationship between the display position of a caption, and Epoch. (A) It is a figure which shows the definition of the graphics object by ODS.

(B) It is a figure which shows the data structure of PDS.
(A) It is a figure which shows the data structure of WDS. (B) It is composed of a PCS data structure. It is a description example of Display Set for realizing subtitle display. It is a figure which shows the example of a description of WDS and PCS in DS1. It is a figure which shows the example of description of PCS in DS2. It is a figure which shows the example of description of PCS in DS3. It is a figure which shows the memory space in an object buffer in implement | achieving the graphics update as shown in FIGS. It is a figure which shows an example of the calculation algorithm of decode_duration. FIG. 15 is a flowchart schematically showing an algorithm of the program of FIG. (A) (b) It is the flowchart which represented the algorithm of the program of FIG. (A) It is the figure which assumed the case where one ODS exists in one window.

(B) (c) It is a timing chart which shows the time context of each numerical value quoted in FIG.
(A) It is the figure which assumed the case where two ODS exists in one window. (B) (c) It is a timing chart which shows the time context of each numerical value quoted in FIG. (A) A timing chart assuming a case where one ODS exists in each of two windows.

(B) It is a timing chart which shows the case where decoding period (2) becomes longer than clear period (1) + writing period (31).
(C) is a timing chart showing a case where the clear period (1) + the write period (31) is longer than the decode period (2).
It is a figure which shows the breakdown of the process with respect to one Display Set. It is a figure which shows how the process with respect to two Display Sets (DSn, DSn + 1) is parallelized in a pipeline decoder model. It is a figure which shows an example in the case of overlapping the active period of PCS with respect to three Display Sets. It is a figure which shows the setting with respect to the time stamp of each function segment which belongs to each Display Set. It is a figure which shows the time stamp of PCS which belongs to each Display Set. (A) It is a figure which shows an example in the case of overlapping the active period of PCS with respect to two Display Sets.

(B) It is a figure which shows an example when not overlapping the active period of PCS with respect to two Display Sets.
It is a figure for demonstrating the transmission end by an END segment. (A)-(c) is a figure which shows the relationship between duplication of the active period of PCS, and object_id allocation. It is a figure which shows the internal structure of the reproducing | regenerating apparatus based on this invention. FIG. 4 is a diagram illustrating the sizes of write rates Rx, Rc, Rd, graphics plane 8, Coded Data Buffer 13, Object Buffer 15, and Composition Buffer 16. It is a timing chart which shows the pipeline process by a reproducing | regenerating apparatus. 10 is a timing chart illustrating pipeline processing assuming that ODS decoding ends earlier than clearing the graphics plane. 10 is a timing chart showing temporal transitions of accumulated amounts in the composition buffer 16, the object buffer 15, the coded data buffer 13, and the graphics plane 8. It is a flowchart which shows the process sequence of the load process of a functional segment. It is a figure which shows an example in which skip operation is made. FIG. 35 is a diagram showing a state in which DS10 is loaded into Coded Data Buffer 13 of the playback device when a skip operation is performed in FIG. It is a figure which shows the case where normal reproduction | regeneration is performed. FIG. 37 is a diagram showing loading of DSs 1, 10, and 20 when normal reproduction is performed as shown in FIG. 4 is a flowchart showing a processing procedure of the Graphical Controller 17. 10 is a flowchart showing a processing procedure of the Graphical Controller 17. 10 is a flowchart showing a processing procedure of the Graphical Controller 17. It is a figure which shows the manufacturing process for manufacturing BD-ROM on which PCS shown in 1st Embodiment was recorded.

Explanation of symbols

1 BD drive 2 Read Buffer
3 PID filter 4a, b, c Transport Buffer
4d peripheral circuit 5 video decoder 6 video plane 7 audio decoder 8 graphics plane 9 CLUT section 12 graphics decoder 13 Coded Data Buffer
14 Stream Graphics Processor
15 Object Buffer
16 Composition Buffer
17 Graphical Controller
20 Adder 100 BD-ROM
200 Playback device 300 TV 400 Remote control

Claims (2)

A playback system including a recording medium and a playback device,
In the recording medium, a digital stream obtained by multiplexing a video stream and a graphics stream is recorded,
The graphics stream includes multiple display sets that implement a graphics screen.
Each display set includes a presentation control segment, an object definition segment that defines a graphics object, and an end segment.
The presentation control segment has time information, and the time information is information that defines an active period of the presentation control segment on the playback time axis of the video stream.
Each segment is stored in a packet;
The time information includes a packet decoding time stamp and a presentation time stamp,
If the active period of the presentation control segment belonging to the preceding display set DSn and the active period of the presentation control segment belonging to the subsequent display set DSn + 1 overlap,
The value PTS (DSn [ODSlast]) of the presentation time stamp PTS of the packet storing the last object definition segment ODSlast belonging to the preceding display set DSn is for the object buffer in the active period of the presentation control segment belonging to the preceding display set DSn. Represents the time when the graphics object has been written and drawing of the graphics object to the graphics plane is possible. The presentation time stamp PTS value PTS (DSn [END] of the packet storing the end segment END belonging to the display set DSn ) With the relationship shown in the following formula,

PTS (DSn [END]) = PTS (DSn [ODSlast])

The value DTS (DSn + 1 [PCS]) of the decoding time stamp DTS of the packet storing the presentation control segment PCS belonging to the display set DSn + 1 is the starting point of the active period of the presentation control segment belonging to the subsequent display set DSn + 1 And satisfying the relationship expressed by the following formula with the PTS (DSn [END]),

PTS (DSn [END]) ≦ DTS (DSn + 1 [PCS])

The active period of the presentation control segment is a period from the time indicated in the decoding time stamp of the presentation control segment to the time indicated in the presentation time stamp of the presentation control segment,
The presentation time stamp of the presentation control segment indicates when the drawing of the graphics object on the graphics plane is completed,
The playback device reads a digital stream from a recording medium, plays back a video stream multiplexed on the digital stream, and a graphics stream,
A video decoder that decodes the video stream to obtain moving images;
A graphics decoder that decodes the graphics stream to obtain and display a graphics object;
The graphics decoder includes an object buffer. The recording medium has a sector structure, and a video stream and a graphics stream are composed of PES packet sequences, and each PES packet is multiplexed in a state of being converted to a TS packet, and each TS packet has an Arrival Time 2. The reproduction system according to claim 1 , wherein the reproduction system is grouped with a TS header including a stamp and is written in a plurality of continuous sectors.

Images