RU2415483C2 - Player - Google Patents

Abstract

FIELD: information technology.

SUBSTANCE: device plays from the recording medium a stream of the main track and a stream of the auxiliary track. At any moment in time, total volume of data of the played streams does not exceed a predefined value.

EFFECT: prevention of buffer overflow.

5 cl, 43 dwg

Description

The present invention belongs to the field of technology of the infrastructure of super-compaction (Oub-of-MUX).

BACKGROUND OF THE INVENTION

Over-compaction infrastructure is a technology that simultaneously reads a digital stream recorded on a read-only recording medium such as a BD-ROM (read-only memory device on a Blue-ray standard disc) and a digital stream recorded on a local storage device such as a recording medium with multiple dubbing, feeds them to the decoder, and then plays them synchronously.

Here we assume that the digital stream recorded on the BD-ROM is the main part of the movie, while the digital stream recorded in the local storage device is the narration of the movie director. In this case, by implementing the aforementioned over-compaction infrastructure, the main part of the movie on the BD-ROM and the narration text can be played together, which thereby refines and extends the content on the BD-ROM.

The prior art regarding read-only recording media includes the following patent application.

<Patent reference 1> Laid-open Application No. H8-83478 for Japanese Patent

SUMMARY OF THE INVENTION

[Problems to be solved by the invention]

In the above supercompression infrastructure, the stream recorded on the BD-ROM and the stream recorded in the local storage must be read at the same time, and the TS packets (transport stream) constituting these streams must be supplied to the decoder. According to a study of how much bandwidth is required to feed to the decoder in the worst case, where the BD-ROM delivery bit rate is 48 Mbit / s and the local storage device's bit delivery rate is 48 Mbit / s, data is supplied in as much as 96 Mbit (48 Mbps + 48 Mbps) may occur during the simultaneous read period. If such a worst case is likely to occur, the bandwidth in the device should be expanded in order to produce TS packets at 96 Mbps. If this cannot be done, it is necessary to provide a large buffer in the decoder and force the decoder to perform a pre-read operation in order to read TS packets in advance so that the delivery does not focus at some point in time. If the simultaneous reading period is short, this may be possible; however, in the case of playing a movie lasting two hours, the buffer capacity is insufficient, and the preliminary reading operation is therefore not successful.

Because the pre-read operation is not successful, a negative overflow occurs in the buffer due to the pre-read operation. In this case, this causes loss of video and audio, and therefore the playback quality is significantly reduced. However, the delivery of data with a high bit rate results in an obstacle to lowering the price of such playback devices.

The present invention aims to provide a recording medium capable of supplying digital streams supplied from different recording media to the decoder without the need for a wider bandwidth.

[Means for solving the problem]

In order to achieve the aforementioned goal, the recording medium of the present invention is characterized in that: playlist information includes main path information and auxiliary path information; information about the main path sets from among the number of digital streams one digital stream as the main stream and determines the primary playback section in the main stream; the sub-path information sets among the rest of the plurality of digital streams one digital stream as an auxiliary stream and determines in the auxiliary stream a secondary reproduction section to be synchronized with the primary reproduction section; playlist information further includes a stream table showing at least one pair of elementary streams that are allowed to play simultaneously, this pair of elementary streams consisting of one of a plurality of elementary streams multiplexed into the main stream and one of a plurality of elementary streams multiplexed into the auxiliary stream; and the total amount of data of the digital stream per unit time is less than or equal to a predetermined value, and the digital stream includes the aforementioned pair of elementary streams and does not include the elementary stream, which in the stream table is not allowed to play simultaneously.

[Useful Results of the Invention]

The total amount of data per unit time of the set of elementary streams allowed in the stream table for reproduction is less than or equal to said predetermined value. Even in the worst case, the number of TS packets transmitted per unit of time does not exceed this predetermined value.

For example, in the case where the unit of time is one second and the predefined value is 48 Mbit, if the TS packet delivery locally reaches 96 Mbit due to the simultaneous reading of the streams, the number of bits per second is kept less than or equal to 48 Mbit. Accordingly, the worst case — that is, a 96 Mbit data delivery volume — does not last for 0.5 seconds or more.

Since it is guaranteed that “the worst case does not last for 0.5 seconds or more”, at any time on the time axis of the stream playback, negative overflow in the decoder buffer can be prevented by constructing the playback device so that TS packets with a size of 96 Mbit × 0.5 seconds are always read in advance and fed to the decoder.

The pre-read operation with an upper limit of “96 Mbps × 0.5 seconds” prevents negative overflow, and therefore TS packets can be stably fed to the decoder. This eliminates the risk that simultaneous reads to implement overcompression infrastructure will affect the quality of the digital stream. It is possible to implement an over-compression infrastructure in a reproducing apparatus that performs BD-ROM reproduction, but without requiring a bandwidth that needs to be increased. As a result, playback devices that implement over-compaction infrastructure can be brought to market at low prices.

In addition, with a limitation of “48 Mbps or less per second”, if the reproducing apparatus performs simple control of “continuously executing the pre-read operation” as described above, it is possible to prevent the occurrence of negative overflow even if the worst-case data supply occurs. This eliminates the need to implement a sequence of operations for predicting time relationships at which the worst-case data supply could occur, thereby contributing to the development of reproducing devices.

List of drawings

Figure 1 shows the consumer use of a recording medium according to the present invention;

figure 2 shows the internal structure of the BD-ROM;

figure 3 - schematic structure of the file with the extension.m2ts assigned to it;

4 shows additional details of how video and audio streams are stored in a sequence of PES packets;

5 shows how video and audio are multiplexed into a program stream and a transport stream;

6 shows a detail of the transport stream;

7 shows the internal structures of a PAT packet and a PMT packet;

Fig. 8 shows what operation sequences TS packets constituting the AV clip (AVClip, AV clip) are subjected to before being recorded onto the BD-ROM;

Fig.9 shows the internal structure of the aligned block (Aligned Unit);

10 shows an internal structure of clip information;

11 shows the settings of an EP card (EP_map) for a movie video stream;

12 shows a data structure of information of a playlist (PlayList);

13 shows the relationships between the AV clip and the playlist information;

Fig shows the internal structure of the local storage device 200;

FIG. 15 shows a method by which a primary TS (Primary TS) and a secondary TS (Secondary TS) forming an over-densification application are supplied to a decoder within a BD-ROM reproducing apparatus;

16 shows a data structure of information of a playlist;

Fig. 17 shows a disclosure of the internal structure of the auxiliary path information;

Fig. 18 shows the relationship of auxiliary clips in local storage 200, playlist information in local storage 200, and main clip (MainClip) on a BD-ROM;

figa shows the internal structure of the table STN (STN_table);

FIG. 19B shows a stream attribute (Stream_attribute) corresponding to a video stream;

Fig. 19C shows a stream attribute (Stream_attribute) corresponding to an audio stream;

fig.19D shows the input stream (Stream_entry) audio stream;

FIG. 20 shows TS packets read from a BD-ROM and from a local storage device, and illustrates from these TS packets those to be supplied to a decoder; FIG.

figa-21D show the shift of the window (Window);

FIG. 22 is a graph showing temporal modulation regarding the amount of data of the TS packets read from the BD-ROM, as well as the data volume of the TS packets read from the local storage;

23A and 23B show a comparison between the transmitted volume and the volume supplied to the decoder during each window;

Fig. 24 shows a connection state of playitems (PlayItem) and auxiliary playitems (SubPlayItem) constituting over-compaction;

Fig. 25 shows the relationship between start times (In_Time) and finish times (Out_Time) of playitems and start times (In_Time) and finish times (Out_Time) of auxiliary playitems in the case where the connection condition information (connection_condition) of the item playback and connection condition information SP (sp_connection_condition) of the sub playback element shown in FIG. 24 is set to “= 5”;

FIG. 26 shows an STC value to be processed when a part existing from a start time (In_Time) to an end time (Out_Time) of a playitem is reproduced, and an STC value to be processed when a part existing from a moment is played start time (In_Time) to the end time (Out_Time) of the sub play item;

Fig. 27 shows how TS1 and TS2 are identified in the main clip indicated by the link in the previous playitem and the auxiliary clip indicated by the link in the current playitem;

Fig. 28 shows details on CC = 5 and SP_CC = 5;

Fig. 29 shows the relationship between the plurality of video presentation units defined by the previous playitem and the current playitem, the multiple audio presentation units and the time axes of the STC;

30 shows an internal structure of a reproducing apparatus of the present invention;

FIG. 31 is a flowchart showing a reproducing procedure based on the information of the playlist; FIG.

32 is a flowchart showing a process for processing a seamless connection of sub-play elements;

Fig. 33 shows the internal structure of the authoring system of Embodiment 2;

Fig. 34 is a flowchart showing a verification procedure for primary TS and secondary TS;

Fig. 35 is a flowchart showing a verification procedure for a primary TS and a secondary TS when multiple streams of the same type occur;

Fig. 36 shows a detailed explanation of CC = 6;

Fig. 37 shows the relationship between playitems and sub playitems;

Fig. 38 schematically shows a method by which multiple TS packets present on an ATC time axis are multiplexed;

Fig. 39 schematically shows, in a case where the subtitle (PG) and menu (IG) are also replaced in addition to audio, the manner in which multiple TS packets constituting the primary TS and multiple TS packets constituting the secondary TS are multiplexed together;

40 shows a method by which a primary TS and a secondary TS constituting an audio mixing application are supplied to a decoder within a BD-ROM reproducing apparatus;

41 shows an internal structure of a reproducing apparatus according to Embodiment 5;

Fig. 42 shows a relationship between playitems and sub playitems defined by a playlist indicating audio mixing, and

Fig. 43 shows an example of playlist information forming a stage version and directorial cut.

Legend

1a - BD-ROM drive

1b, c - read buffer

1b, a, c - ATC counter

2a, d - source depacketizer (depacketizer) of the source

2c, d - ATC counter

3a, c - STC counter

3b, d - PID filter

4 - video decoder

5 - video card

6 - transport buffer

7 - elementary buffer

8 - audio decoder

10a, b, c, d - switch

11 - interactive graphics decoder

12 - interactive graphics board

13 - demo decoder

14 - demo board

17 - synthesis module

21 - memory

22 - controller

23 - PSR kit

24 - PID conversion module

25 - network module

26 - operations receiving module

100 - BD-ROM

200 - local storage device

300 - playback device

400 - TV

500 - AV amplifier

Best Mode for Carrying Out the Invention

Option 1 implementation

The following reports on a preferred embodiment of the recording medium according to the present invention. First of all, consumer use is described with respect to the implementation of the recording medium of the present invention. Figure 1 shows the consumer use of a recording medium according to the present invention. The local storage device 200 of FIG. 1 is a recording medium of the present invention. The local storage device 200 is used to supply a movie to a home theater system consisting of a playback device 300, a television 400, an AV amplifier 500, and speakers 600.

The following explains the BD-ROM 100, the local storage device 200, and the playback device 300.

The BD-ROM 100 is a recording medium on which a movie is recorded.

Local storage device 200 is a hard disk drive that is built into the playback device and is used to store content distributed from the movie distributor server.

The playback device 300 is a digital home appliance supported with respect to networks and has a function for playing the BD-ROM 100. The playback device 300 is also capable of downloading content from the movie distributor server 700 via the network, storing the downloaded content on the local storage device 200, and combining this content with content recorded on the BD-ROM 100 to expand / update the content of the BD-ROM 100. A technology called the “virtual software package” combines content recorded on the BD-ROM 100 with content th stored in the local storage 200, and interprets the data not recorded on the BD-ROM 100 in a manner as if they were recorded on the BD-ROM 100.

This is a description of the traditional use of the recording medium of the present invention.

The following describes the manufacturing use of the recording medium of the present invention. The recording medium of the present invention can be implemented as a result of improvements to the BD-ROM file system.

<General description of BD-ROM>

Figure 2 shows the internal structure of the BD-ROM; Level 4 in the figure shows a BD-ROM, and level 3 shows a track on the BD-ROM. The figure depicts a track in an elongated form, although the track is actually formed into a spiral winding from the inside towards the outside of the BD-ROM. A track consists of a start region, a volume region, and an end region. The volume region in the figure contains a layered model composed of a physical layer, a file system layer, and an application layer. Layer 1 in the figure shows the BD-ROM application layer format by using a directory structure. At level 1, the BD-ROM contains the EDMV directory under the root (Root) directory.

In addition, three subdirectories are located under the BDMV directory: the PLAYLIST (playlist) directory; CLIPINF directory (information about clips) and STREAM directory (streams).

The PIAYLIST directory includes a file for which the mpls extension (00001.mpls) is assigned.

The CLIPINF directory includes files, each of which has a clip extension (00001.clip and 00002.clip).

The STREAM directory includes files that are assigned the m2ts extension (00001.m2ts and 00002.m2ts).

Thus, it can be seen that numerous files of various types are arranged on the BD-ROM according to the directory structure given above.

<Structure 1 BD-ROM: AV clip>

First of all, the files for which the “m2ts” extension is assigned are explained. FIG. 3 shows a schematic structure of a file that is associated with the “m2ts” extension. Files, each of which is assigned the extension m2ts (00001.m2ts and 00002.m2ts), store the AV clip. An AV clip is a digital stream in the MPEG2 transport stream format (standard for moving picture and sound compression). The digital stream is generated by converting the digitized video and audio (upper level 1) into an elementary stream consisting of PES packets (upper level 2), and by converting the elementary stream into TS packets (upper level 3) and, similarly, converting the demo graphic stream (PG ) for subtitles or the like, and an interactive graphics (IG) stream for interactive purposes (lower level 1 and lower level 2) into TS packets (lower level 3), and then final multiplexing of these TS packets.

The following describes a video stream, an audio stream, a PG stream, and an IG stream.

<Video Stream>

A video stream is a stream forming moving images of a movie, and consists of movie data of SD images (with standard resolution) and HD images (with high resolution). A video stream is a VC-1 video stream, MPEG4-AVC or MPEG2-Video format. When the video stream is an MPEG4-AVC video stream, time stamps such as PTS and DTS are attached to the IDR, I, P, and B frames, and playback control is performed in the blocks of the frames. A video stream unit, which is a unit for controlling playback, with PTS and DTS attached to it, is called a “video presentation unit”.

<Audio Stream>

An audio stream is a stream for a movie soundtrack, and audio stream formats include an LPCM audio stream format, a DTS-HD audio stream format, a DD / DD + audio stream format, and a DD / MLP audio stream format. Time stamps are attached to audio frames in an audio stream, and playback control is performed in blocks of audio frames. An audio stream unit, which is a unit for controlling playback, with a time stamp attached to it, is called an “audio presentation unit”.

<PG stream>

A PG stream is a graphic stream constituting a subtitle recorded in a language. There are many threads that correspond to many languages, such as English, Japanese, and French, respectively. The PG stream consists of functional segments, such as: PCS (presentation control segment); PDS (Palette Definition Segment); WDS (window definition segment); ODS (object definition segment) and END (display set end segment). ODS (Object Definition Segment) is a functional segment that defines a graphic object that is a subtitle.

WDS (window definition segment) is a functional segment that determines the number of bits of a graphic object on the screen. PDS (Palette Definition Segment) is a functional segment that defines color when drawing a graphic object. PCS (presentation control segment) is a functional segment that defines page management when displaying a subtitle. Such page management includes Cut-In / Out, Fade-In / Out, Color Change, Scroll, and Wipe-In / Out . By controlling the pages with PCS, it is possible to achieve a display effect, for example by making the current subtitle diminish the image brightness when the next subtitle is displayed.

<Stream IG>

The IG stream is a graphic stream for successful interactive management. The interactive control defined by the IG stream is an interactive control that is compatible with the interactive control in a DVD (multi-function digital disc) reproducing apparatus. The IG stream consists of functional segments, such as: ICS (interactive layout segment); EDS (segment definition palette) and ODS (segment definition objects). ODS (Object Definition Segment) is a functional segment that defines a graphic object. Buttons on the interactive screen are drawn through a variety of such graphic objects. PDS (Palette Definition Segment) is a functional segment that defines color when drawing a graphic object. ICS (Interactive Layout Segment) is a functional segment that performs state changes in which the state of the button changes according to the user's operation. ICS includes a button command that is executed when a button confirmation operation is performed.

Here, the AV clip is composed of at least one "STC_Sequence" ("STC sequence"). "STC_Sequence" is a section in which there is no gap (rupture of the system time base) in the STC (system time clock signal), which is a system time base of the AV streams. The gap in the STC is the point at which the gap information (discontinuity_indicator) of the PCR packet carrying the PCR (program time stamp) indicated by the decoder link to receive the STC is ON (on state).

4 shows additional details of how video and audio streams are stored in a sequence of PES packets. Level 1 in the figure shows a video stream, and level 3 shows an audio stream. Level 2 shows the sequence of PES packets. As shown by the arrows yy1, yy2, yy3 and yy4 in the figure, it can be seen that IDR frames, B frames and P frames, which are multiple blocks for presenting a video in a video stream, are divided into multiple sections and each of the separated sections is stored in one of the payloads (V # 1, V # 2, V # 3 and V # 4 in the figure) PES packets. It may also be understood that each of the audio frames, which are audio presentation units constituting the audio stream, is stored in one of the payloads (A # 1 and A # 2 in the figure) of the PES packets, as shown by arrows aa1 and aa2.

5 shows how video and audio are multiplexed into a program stream and a transport stream. The lower part of the figure shows numerous PES packets (V # 1, V # 2, V # 3, V # 4, A # 1 and A # 2 in the figure), which contain video and audio streams stored therein. The figure shows that the video and audio streams are stored in different PES packets. The upper part shows the program stream and the transport stream in which the PES packets shown in the lower part are stored. When multiplexed into a program stream, each PES packet fits in one packet. When multiplexed into a transport stream, a PES packet is divided into sections, each of which is then stored in one of the payloads of multiple TS packets. Not the program stream format, but the transport stream format is used for the BD-ROM storage format. It is well known that the PES video packets used for the transport stream contain one frame or two paired fields, although FIG. 5 does not illustrate such a case.

6 shows a detail of a traffic stream. The layer 1 in the figure shows the sequence of multiple TS packets forming the MPEG2 transport stream, and layer 2 shows the internal structure of the TS packet. As shown in Layer 2, one TS packet consists of a “header”, “adaptation field”, and “payload”. Lead line th1 shows the exposed details of the TS packet header structure. As shown by the extension line, the TS packet header includes: a “block start flag (payload block start sign (payload_unit_start_indicator))” is stored indicating the start of the PES packet; “PID (Packet Identifier)”, indicating the type of elementary stream that is multiplexed into the transport stream; and “adaptation field management” indicating whether the adaptation field is present in the TS packet.

Lead line th2 shows exposed details of the internal structure of the adaptation field. The adaptation field is set for the TS packet when the control of the adaptation field of the TS packet header is set to “1”. More precisely, the adaptation field stores in it: “random access flag (random_access_indicator)”, indicating that the TS packet is the beginning of the video or audio frame and the entry point; and “PCR (Program Timestamp),” which defines the STC (system time clock) of the T-STD (Target Transport Decoder).

7 shows the internal structures of a PAT packet and a PMT packet. These packages describe the software structure of the transport stream.

The lead line hm1 of the figure shows the exposed details of the TS packet structure with PID = 0 in the transport stream. Such a TS packet is called a PAT (program link table) packet and indicates the program structure of the full transport stream. The PID of the PAT packet is always "0". The PAT package contains the PAS (program links section). The lead line hm2 shows the exposed details of the internal structure of the PAS. As shown by the extension line, the PAS shows the correspondence between program_number (program number) and the program map table (PID PMT). Extension line hm3 shows the opened details of the TS packet structure with PID = 0 × 100 present in the transport stream. Such a TS packet is called a PMT packet. As shown by the extension line hm4, the TS packet header includes: “stream type” indicating the type of stream included in the program corresponding to the PMS; and an “elementary PID”, which is the stream PID. According to the example in the figure, a program with program number # 1 contains a PMT with PID = 0 × 100, and MPEG2 video with PID = 0 × 200 and ADTS audio with PID = 0 × 201 constitute a program with program number # 1. The program in the transport stream, as well as the PID of the stream forming the transport stream, and the type of stream can be found by obtaining the PID from PMT from PAT, whose PID is always 0, then receiving the PMT packet according to the PID from PMT and accessing the PMS.

Then, it is explained how an AV clip having the above structure is recorded on the BD-ROM. Fig. 8 shows what process sequences TS packets constituting the AV clip are subjected to before being recorded to the BD-ROM; Level 1 of the figure shows TS packets constituting the AV clip.

As shown at level 2 in FIG. 8, a 4-byte TS additional header (TS_extra_header) (shaded portions in the figure) is attached to each 188-byte TS packet constituting the AV clip to form each 192-byte source packet. The optional TS header (TS_extra_header) includes an arrival time stamp (Arrival_Time_Stamp), which is information indicating the point in time at which the TS packet is input to the decoder. The reason for attaching the ATS header to each TS packet to form a stream is to assign each TS packet a point in time at which the TS packet is input to the decoder (STD). In digital broadcasting, a transport stream is interpreted as a stream having a fixed bit rate. Therefore, empty TS packets, called NULL packets, are also multiplexed together to form a transport stream, so that the transport stream is broadcast at a fixed bit rate. However, in the case where the streams are recorded on an optical disc or other recording medium having a limited recording capacity, such a fixed bit rate recording method is a drawback, since it consumes capacity unproductively. Therefore, null (NULL) packets are not written to the BD-ROM. In order to satisfy the variable bit rate recording method, the ATS is attached to each TS packet, and then the transport stream is recorded on the BD-ROM. Using ATS involves recovering a decoder input timing for each TS packet, and thus can satisfy a variable bit rate recording method. Subsequently, the pair from the ATS header and the TS packet is called the source packet.

The AV clip shown at level 3 includes one or more “ATC_Sequence” (“ATC sequences”), each of which is a sequence of source packets. “ATC_Sequence” is a sequence of source packets where the arrival time clock (Arrival_Time_Clock) referenced by the arrival timestamps (Arrival_Time_Stamp) included in the ATC sequence (ATC_Sequence) does not include an “arrival timeline break”. In other words, “ATC_Sequence” is a sequence of source packets where the arrival time clocks (Arrival_Time_Clock) indicated by the reference timestamps (Arrival_Time_Stamp) included in the ATC sequence (ATC_Sequence) are continuous.

Such ATC sequences (ATC_Sequence) form an AV clip and are recorded on a BD-ROM with the name "xxxxx.m2ts" of the file.

An AV clip, as is the case with normal computer files, is divided into one or more file extents, which are then recorded in areas on the BD-ROM. Level 4 shows how the AV clip is recorded on the BD-ROM. At level 4, each file extent that makes up a file has a data length that is equal to or greater than a predefined length called Sextent.

Sextent is the minimum data length of each file space, where the AV clip is divided into many file extents to be recorded.

The time period required for the optical head to read information passed to the cell on the BD-ROM is obtained according to the following equation:

Tjump = Taccess + Toverhead.

"Taccess" ("Access T") is the required time, which corresponds to the distance of the transition (the distance of the target physical address of the transition).

TS packets read from the BD-ROM are stored in a buffer called a read buffer, and then output to a decoder. “Toverhead” (“T overhead”) is obtained according to the following equality when the input to the read buffer is performed at a bit rate called “Rud” and the number of sectors in the ECC block is represented by Secc:

Toverhead ≤ (2 × Secc × 8) / Rud = 20 ms.

The TS packets read from the BD-ROM are stored in the read buffer in the state of the source packets, and then are output to the decoder at a bit rate called “TS_Recording_rate” (“TS write speed”).

In order to keep the bit rate at the TS write speed (TS_Recording_rate) while the packets are being sent to the decoder, it is necessary that during Tjump the TS packets are continuously output from the read buffer to the decoder. Here, source packets, not TS packets, are output from the read buffer. As a result, when the ratio of the TS packet to the original packet in size is 192/188, it is necessary that during Tjump the source packets are continuously output from the read buffer at a transmission speed of “192/188 × TS_Recording_rate”.

Accordingly, the volume of the occupied buffer capacity of the read buffer, which does not cause negative overflow, is represented by the following equality:

Boccupied≥ (Tjump / 1000 × 8) × ((192/188) × TS_Recording_rate).

The write speed to the read buffer is represented by Rud, and the output speed from the read buffer is represented by TS_Recording_rate × (192/188). Therefore, the speed of filling the read buffer is obtained by executing "(input speed) - (output speed)" and, thus, it is obtained according to "Rud - TS_Recording_rate) × (192/188)".

The time “Tx” required to fill the read buffer according to “Boccupied” (“B busy”) is obtained according to the following equation:

Tx = Boccupied / (Rud - TS_Recording_rate × (192/188)).

When reading from a BD-ROM, it is necessary to continue to introduce TS packets with a bit rate of Rud for a “Tx” time period. As a result, the minimum length of the Sextent data for the extent, when the AV clip is divided into many file extents that must be recorded, is obtained according to the following equations:

Sextent = Rud × Tx = Rud × Boccupied / (Rud-TS_Recording_ratex (192/188)) ≥Rud × (Tjump / 1000 × 8) × (192/188) × TS_Recording_rate) / (Rud-TS_Recording_rate × (192/188) ) ≥ (Rud × Tjump / 1000 × 8) × TS_Recording_rate × 192

/ (Rud × 188-TS_Recording_rate × 192).

From here

Sextent≥ (Tjump × Rud / 1000 × 8) × (TS_Recording_rate × 192 / (Rud × 188-TS_Recording_rate × 192)).

If each file extent that makes up the AV clip has a data length equal to or greater than Sextent, which is calculated as a value that does not cause negative decoder overflow, even if the file extents that make up the AV clip are located on the BD-ROM by individually, TS packets are continuously output to the decoder so that data is read continuously during playback.

The smallest constituent block of the aforementioned file extent is a aligned block (6 KB data volume), which consists of a group of 32 source packets. Accordingly, the size of the stream file (XXXX.AVClip) on the BD is always a multiple of 6 kbytes.

Fig.9 shows the internal structure of the "aligned block". The aligned block is composed of 32 source packets, and then written to a set of three successive sectors. A group of 32 source packets are 6144 bytes (= 32 × 192), which are equivalent to the size of three sectors (2048 × 3). For sectors on the BD-ROM, an error correction code is assigned to every 32 source packets to thereby form an ECC block. Once access to the BD-ROM is in units of aligned blocks, the playback device can receive 32 complete source packets. This is a description of the process for recording an AV clip on a BD-ROM. An AV clip that is recorded on a BD-ROM and with which high-resolution video streams are multiplexed is hereinafter referred to as “MainClip” (“main clip”). On the other hand, the AV clip, which is stored on the local storage device and played back with the main clip, is called “SubClip” (“auxiliary clip”).

A partial transport stream is obtained by demultiplexing the main clip recorded on the BD-ROM. A partial transport stream corresponds to each elementary stream. The partial transport stream obtained by demultiplexing the main clip and corresponding to each elementary stream is called “Primary TS” (“primary TS”).

<Structure 2 BD-ROM: information about the clips>

The following are the files that are assigned the clip extension. Files (00001.clpi and 00002.clpi), which are assigned the extension "clpi", store information about the clips. The clip information is control information about each AV clip. 10 shows an internal structure of clip information. As shown on the left side of the figure, the clip information includes:

i) “ClipInfo ()”, storing in it information regarding the AV clip;

ii) a “Sequence Info ()" storing information regarding an ATC sequence and an STC sequence therein;

iii) “Program Info ()”, which stores information regarding the program sequence and

iv) “Characteristic Point Info (CPI ())” (“Characteristic Point Information (CPI ())”).

“ClipInfo” (“clip information”) includes “application_type” (“application type”) indicating the application type of the AV clip indicated by the clip information link. Pointing to clip information provides an opportunity to identify whether the application type is a main clip or a sub clip, whether video is contained or still images are contained (slide show). In addition, the aforementioned TS recording speed (TS_recording_rate) is described in the clip information.

Sequence Info is information regarding one or more STC sequences and ATC sequences contained in an AV clip. The reason this information is provided is to preliminarily notify the playback device of a system time base gap and a time gap gap of the receipt. That is, if there is such a gap, it is likely that PTS and ATS, which have the same value, take place in the AV clip. This may cause defective playback. Sequence information is provided to indicate where and from where, in the transport stream, the STC or ATC are sequential.

Program Information (Program Info) is information that indicates the section (called the "program sequence") of the program, where the contents are unchanged. Here, a “program” is a group of elementary streams that has a common time axis for synchronous playback. The reason that the program information is provided is to notify the playback device of the point at which the contents of the programs change in advance. It should be noted here that the point at which the contents of the programs change, for example, is the point at which the PID of the video stream changes, or the point at which the type of video stream changes from SDTV (standard definition television) to HDTV (high definition television).

The following describes the characteristic point information. Lead line cu2 in FIG. 9 shows the disclosure of the CPI structure. As indicated by the cu2 extension line, the CPI consists of Ne parts of the EP cards for the PID of one stream (EP_map_for_one_stream_PID) (from the EP card for PID [0] of one stream (EP_map_for_one_stream_PID [0]) on the EP card for the PID [Ne-1] of one stream ( EP_map_for_one_stream_PID [Ne-1]). These EP cards for the PID of one stream (EP_map_for_one_stream_PID) are EP cards (EP_map) of elementary streams that belong to the AV clip. The EP card (EP_map) is information that indicates in connection with the start time ( PTS_EP_start), packet number (SPN_EP_start) at the entry position, where the access unit is present in one elementary stream. Extension line cu3 in the figure indicates the rytie inner EP map patterns for one stream PID (EP_map_for_one_stream_PID).

From the disclosure it is clear that the EP card for the PID of one stream (EP_map_for_one_stream_PID) consists of Ne parts EP_High (from EP_High (0) to EP_High (Nc-l)) and Nf parts EP_Low (from EP_Low (0) to EP_Low (Nf-l) ) Here EP_High plays the role of indicating the most significant bits in the SPN_EP_start and PTS_EP_start of the access unit (not related to the IDR of the I-frame, IDR-movie frame), and EP_Low plays the role of indicating the least significant bits in the SPN_EP_start and PTS_EP_start of the access unit (not related to the IDR I-kin , IDR movie frame).

Lead line cu4 in the figure indicates the disclosure of the internal structure of EP_High. As indicated by the extension line cu4, EP_High (i) consists of: "ref_to_EP_Low_id [i]" ("reference to EP_Low_id [i]"), which is the value of the link to EP_Low; "PTS_EP_High [i]", which indicates the most significant bits of the PTS of the access unit (not related to the IDR of the I-movie frame, IDR-movie frame); and "SPN_EP_High [i]", which indicates the most significant bits of the SPN of the access unit (not related to the IDR of the I-movie frame, IDR-movie frame). Here “i” is the identifier of this EP_High.

Lead line cu5 in the figure indicates the disclosure of the EP_Low structure. As indicated by the cu5 extension line, EP_Low (i) consists of: “is_angle_change_point (EP_Low_id)” (“is the angle change point (EP_Low_id)”, which indicates whether the corresponding access unit is an IDR movie frame; “I_end_position_offset (EP_Low_id)” (“ end position offset I (EP_Low_id) "), which indicates the size of the corresponding access unit;" PTS_EP_Low (EP_Low_id) ", which indicates the least significant bits of the PTS of the access unit (not related to the IDR of the I-movie frame, IDR-movie frame); and" SPN_EP_Low ( EP_Low_id) ", which indicates the least significant bits of the SPN of the access unit (not related to the IDR of the I-movie frame, IDR-movie frame). Here l "EP_Low_id" is the identifier for identifying this EP_Low.

<Explanation 2 of the clip information: EP card>

Here, the EP map (EP_map) is explained using a separate example. 11 shows the settings of an EP card (EP_map) for a movie video stream; Level 1 shows a plurality of motion pictures (IDR motion picture, I-motion picture, B-motion picture and P-motion picture defined in MPEG4-AVC) arranged in display order. Level 2 shows the time axis for movie frames. Level 4 indicates the sequence of TS packets on the BD-ROM, and level 3 indicates the settings of the EP card (EP_map).

Here, suppose that on the time axis of level 2, an IDR film frame or I-film frame is present at each time instant from t1 to t7. The interval between adjacent times t1 through t7 is approximately one second. The EP card (EP_map) used for the movie is configured to indicate t1 through t7 with start times (PTS_EP_start) and indicate entry positions (SPN_EP_start) in relation to start times.

<Playlist Information>

The following describes playlist information. The file (00001.mpls), which is assigned the extension "mpls", is a file that stores playlist (PL) information in it.

12 shows a data structure of playlist information. As indicated by the extension line mp1 in the figure, the information of the playlist includes: information about the main path (MainPath ()), which defines the main path; playlist label information (PlayListMark ()) that defines the section; and other extension data.

<Clarification 1 of the information of the playlist: information about the main path>

First of all, the main path is described. The main path is the playback path, which is determined based on the video stream, such as the main video, and the audio stream.

As indicated by the arrow mp1, the main path is defined by many pieces of information about the playitem: from information # 1 about the playitem to information #m about the playitem. Information about the playitem defines one or more logical playback sections that make up the main path. Extension line hs1 in the figure indicates the disclosure of the structure of information about the playitem. As indicated by the extension line hs1, information about the playitem consists of: “Clip_Information_file_name” (“name of the file of information about the clips”), which indicates the name of the information file of the playback section of the AV clip, which belongs to the entry point (IN) and exit point (OUT) playback sections; “Clip_codec_identifier”, which indicates the encoding method of the AV clip; "Is_multi_angle" ("is multi-angle"), which indicates whether or not the play item is multi-angle; “Connection_condition”, which indicates whether or not to connect the current playitem and the previous playitem in a seamless manner; "Ref_to_STC_id [0]" ("reference to STC_id [0]"), which uniquely indicates the sequence STC (STC_Sequence) targeted by the playitem; "In_time" ("start time"), which is time information indicating the starting point of the playback section; “Out_time”, which is time information indicating an end point of a playback section; “UO_mask_table” (“user operation mask table”), which indicates which user operation should be masked by a playitem; "PlayItem_random_access_flag" ("sign of random access to the playitem"), which indicates whether to allow random access to the midpoint in the playitem; “Still_mode” (“snapshot mode”), which indicates whether to continue the still display of the last movie frame after the playback of the playback item ends; and “STN_table” (“STN table”). Among these, the In_time time information (“start time”) indicating the start point of the playback section, and the “Out_time” time information (“start time”) indicating the end point of the playback section constitute the playback path. Information about the playback path consists of "In_time" ("start time") and "Out_time" ("end time").

13 shows the relationship between the AV clip information and the playlist. Level 1 shows the time axis of the playlist information (time axis of the playlist). Levels 2 through 5 show the video stream, which is indicated by the reference map EP (EP_map).

The playlist information includes two pieces of information about the playitem: information # 1 about the playitem and information # 2 about the playitem. Two playback sections are defined by “In_time” and “Out_time” (“start time” and “end time”) included in the information # 1 about the playitem and information # 2 about the playitem, respectively. When these playback sections are arranged, a time axis is determined that is different from the time axis of the AV clip. This is the time axis of the playlist, shown at level 1. Thus, it is possible to determine a playback path that is different from the AV clip by determining information about the playitem.

This is the description of the BD-ROM 100.

<Local Storage 200>

The following describes a local storage device 200, which is a recording medium of the present invention. Fig. 14 shows an internal structure of a local storage device 200. As shown in the figure, the recording medium of the present invention can be created by improving the application layer.

The level 4 of the figure shows the local storage device 200, and the level 3 shows the track on the local storage device 200. The figure depicts the track stretched to the side, although the track is actually formed in a spiral, winding from the inside towards the outside of the local storage device 200 A track consists of a start region, a volume region, and an end region. The volume region in the figure contains a layered model composed of a physical layer, a file system layer, and an application layer. Level 1 in the figure shows the application layer format of the local storage 200 by using a directory structure.

In the directory structure shown in FIG. 13, there is a subdirectory “organization # 1” under the root directory. In addition, there is a subdirectory "disk # 1" under the directory "organization # 1". The organization # 1 directory is assigned to a specific software provider. A “disk # 1” directory is assigned to each BD-ROM supplied by the vendor.

With this design, in which the directory assigned to a particular vendor includes directories that correspond to the BD-ROM, the downloaded data for each BD-ROM is stored separately. Like the information stored in the BD-ROM, the following information is stored under the “disk # 1” subdirectory: playlist information (“00002.mpls”); information about the clips (“00003.clpi” and “00004.clpi”) and the AV clips (“00003.m2ts” and “00004.m2ts”).

The following describes the components of local storage 200: playlist information, clip information, and AV clips.

<Structure 1 of local storage device 200: AV clip>

The AV clips (00003.m2ts and 00004.m2ts) in the local storage 200 constitute auxiliary clips. A partial transport stream is obtained by demultiplexing an auxiliary clip. The partial transport stream obtained by demultiplexing the auxiliary clip is called “Secondary TS”. Such a secondary TS is a component of the over-compaction application. The following describes the application of over-compaction.

(Superconsolidation Application)

An over-compaction application is an application that, for example, selects two TSs — a primary TS on a BD-ROM and a secondary TS, which is received through a network or the like, and recorded to a local storage device — and plays them simultaneously, thereby enabling elementary associations flows between these two TS.

FIG. 15 shows a method by which a primary TS and a secondary TS constituting a super-densification application are supplied to a decoder within a BD-ROM reproducing apparatus. In the figure, among the internal structural components of the BD-ROM reproducing apparatus, a BD-ROM drive, a local storage device and a network are shown on the left side, while a decoder is shown on the right side. A PID filter that performs stream demultiplexing is shown in the center. Primary TS (video 1, audio 1 (English), audio 2 (Spanish), PG 1 (subtitle in English), IG 1 (menu in English)) and secondary TS (audio 2 (Japanese), audio 3 (Korean), PG 2 (subtitle in Japanese), PG 3 (subtitle in Korean), IG 2 (menu in Japanese), IG 3 (menu in Korean) in the figure are the transport streams supplied with BD-ROM and local storage, respectively. Since only English (audio 1) and Spanish (audio 2) are recorded on the disc, for example, an option duplicated in Japanese cannot be selected on the disc. However, by downloading to the local storage a secondary TS that includes a Japanese-dubbed version (audio 2) provided by the content provider, Japanese-dubbed audio (audio 2), a Japanese subtitle (PG 2), and a menu screen on Japanese (IG 2) can be sent to the decoder. Through this, the user is able to select any of the dubbed in Japanese audio (audio 2), Japanese subtitle (PG 2) and the Japanese menu screen (IG 2) and play it from the video (video 1).

The over-compaction application allows the user to freely make selections for audio and subtitles, provided that the choice can be made for up to one, regarding each type of elementary streams, which are stored in two TS, which must be played simultaneously (in other words, up to one video, one audio, one subtitle and one menu stored in the primary and secondary TS).

Any BD-ROM playback device is capable of decoding a primary TS, but cannot decode two TSs at the same time. Accordingly, the introduction of an overcompression application without limitation could cause an increase in the dimensions of the hardware and / or a serious addition to the software, which results in an increase in the cost of BD-ROM playback devices. Therefore, when it comes to the execution of a supercompression application, the key question is whether the supercondensation application can be implemented on resources capable of decoding only the primary TS.

The restriction of providing the ability to play up to one for each type of elementary streams may be allowed as a “replacement” of elementary streams of the primary TS with those of the secondary TS. By this means, the overcompression application can be executed on resources capable of decoding only a single TS, avoiding an increase in the cost of decoders. According to the example of the figure, the audio stream, the subtitle stream (PG) and the menu stream (IG) of the primary TS are replaced by those of the secondary TS.

The secondary TS can be entered not only from the built-in HDD, such as the aforementioned local storage device, but also from the temporary storage memory, the primary storage device memory and the HDD through the network or by streaming data through network software. For convenience of explanation, suppose that the secondary TS is supplied from the built-in HDD, similar to that shown in figure 1.

<Structure 2 of the local storage device 200: information about the clips>

The clip information (00003.clpi, 00004.clpi) in the local storage device has the same data structure as the clip information recorded on the BD-ROM. Here, the recording speed TS (TS_Recording_Rate) of the clip information in the local storage is set to be the same as the bit rate for reading the AV clip from the BD-ROM. That is, the recording speed TS (TS_Recording_Rate) recorded in the clip information of the sub clip is the same as the recording speed TS (TS_Recording_Rate) recorded in the clip information of the main clip. If the recording speed TS (TS_Recording_Rate) of the main clip is different from the recording speed TS (TS_Recording_Rate) of the auxiliary clip, the data rate for transmission from each source depacketizer to the buffer according to which the TS is transmitted is changed. This fails to establish the assumption that the overcompression application can be regarded as one input TS.

In addition, since the elementary streams to be reproduced are freely selected from two TSs, everything, the source depacketizer and the buffer in the decoder, are tuned to the bit rate of the primary TS when the audio of the primary TS is selected, and everything, the source depacketizer and the buffer in the decoder are tuned to the bit rate of the secondary TS when the audio of the secondary TS is selected. This makes the sequence of operations and verification of the playback device cumbersome and complex.

<Structure 2 of local storage 200: playlist information>

The following describes the information of the playlist in the local storage 200. The file (00002.mpls), which is assigned the extension “mpls”, is information that defines the group created by linking the two types of playback paths, called the main path (MainPath) and the auxiliary path ( Subpath), as a playlist (PL). 16 shows a data structure of playlist information. As shown in the figure, the information of the playlist includes: information about the main path (MainPath ()), which defines the main path; information of the label playlist (PlayListMark ()), which defines the section, and information about the auxiliary path (Subpath ()), which defines the auxiliary path. The internal structures of the playlist information and information about the playitems are the same as those in the BD-ROM, and therefore their descriptions are omitted here. The following describes helper path information.

<Clarification 1 of playlist information: auxiliary path information>

Whereas the primary path is a playback path specific to the primary clip, which is the primary video, the secondary path is the playback path specific to the secondary clip, which is synchronized with the primary path.

17 shows a disclosure of the internal structure of the auxiliary path information. As indicated by the hc0 arrow in the figure, each sub path includes a “SubPath_type” (“sub path type”) indicating the type of the sub clip and one or more pieces of information about the sub playback element (... SubPlayItem () ...) .

The lead line hc1 in the figure indicates the disclosure of the information structure of the sub play item. As indicated by the arrow hc1 in the figure, information about the sub-play item includes: “Clip_information_file_name” (“name of the clip information file”); "Clip_codec_identifier" ("identifier of the codec of the clip"); “SP_connection_condition”; "Ref_to_STC_id [0]" ("reference to STC_id [0]"); “SubPlayItem_In_time” (“time point of the start of the sub play item”); "SubPlayItem_Out_time" ("time point of the end of the sub-play item"); “Sync_PlayItem_id” (“id of the sync play item” and “sync_start_PTS_of_PlayItem” (“initial PTS of the sync play item”).

“Clip_information_file_name” is information that uniquely identifies an auxiliary clip corresponding to the auxiliary playback element by describing the name of the clip information file.

"Clip_codec_identifier" indicates the coding system of the AV clip.

"SP_connection_condition" indicates the state of the connection between the sub play item (the current sub play item) and the sub play item (the previous sub play item) immediately preceding the sub play item (the current sub play item).

"Ref_to_STC_id [0]" uniquely indicates the STC sequence (STC_Sequence) that the playitem is targeting.

"SubPlayItem_In_time" is information indicating the starting point of the sub-play item on the time axis of the playback of the sub-clip.

“SubPlayItem_Out_time” is information indicating an end point of an auxiliary playback element on a time axis of playback of the auxiliary clip.

"Sync_PlayItem_id" is information that uniquely specifies from among the playitems constituting the main path, the playitem with which the auxiliary playitem is synchronized. “SubPlayItem_In_time” is represented on the time axis of the playback of the playitem specified by the id of the synchronization playitem (sync_PlayItem_id).

"Sync_start_PTS_of_PlayItem" indicates with a temporal accuracy of 45 kHz, where on the playback time axis of the playitem specified by the id of the synchronization playitem (sync_PlayItem_id), there is a starting point of the sub play item specified by the start time of the sub play item (SubPlayItem_In_time).

<Detailing information 2 about the auxiliary path. The relationship of the three objects>

Here, three objects mean auxiliary clips in the local storage 200, information of the playlist in the local storage 200 and the main clip on the BD-ROM.

Fig. 18 shows the relationship of auxiliary clips in local storage 200, playlist information in local storage 200 and main clip on a BD-ROM. The level 1 of the figure indicates auxiliary clips present in the local storage 200. As shown in level 1, there are different types of secondary TS in the auxiliary clips of the local storage 200: audio stream, PG stream, and IG stream. Any one of them is used as an auxiliary path for synchronizing playback.

Level 2 indicates two time axes defined by playlist information. The lower time axis at level 2 is the time axis of the playlist defined by the playitem information, and the upper time axis is the time axis of the sub playitem defined by the sub playitem.

As shown in the figure, it can be seen that the name of the file of information about the clips of the auxiliary playback element (SubPlayItem_Clip_information_file_name) information about the element of auxiliary playback plays the role of selecting from the number of files.m2ts storing auxiliary clips, file.m2ts as the target for the playback section.

The end time of the sub play item (SubPlayItem.Out_time) plays a role in determining the start point and end point of the play section.

The pointer Id of the synchronization playitem (Sync_PlayItem_Id) plays a role in setting which playitem is synchronized with the sub playitem. The initial PTS of the synchronization of the playitem (sync_start_PTS_of_PlayItem) plays a role in determining the time from the start time of the auxiliary playitem (SubPlayItem_In_time) on the time axis of the playlist.

This is a description of the auxiliary path information.

<STN table (STN_table)>

A sign of the playlist information in the local storage 200 is the table STN (STN_Table). The following describes the playlist information in the local storage 200.

The STN table (STN_table) is a table showing at least one combination of elementary streams that are allowed to play simultaneously. The combination of elementary streams was selected from numerous elementary streams multiplexed into the main clip given by the name of the clip information file (Clip_Information_file_name) information about the playitem, as well as elementary streams multiplexed into the auxiliary clip given by the name of the clip information file (Clip_Information_file_name) information about sub play item. Such numerous elementary streams allowed for simultaneous playback in the STN table (STN_table) in the playlist information form the so-called "system stream".

More precisely, the STN table (STN_table) is formed by associating the stream input (Stream_entry) of each of the many elementary streams multiplexed into the main clip and those multiplexed into the auxiliary clip using the stream attribute (Stream_attribute).

Figa shows the internal structure of the table STN (STN_table). As shown in the figure, the STN table (STN_table) includes numerous input and attribute pairs (input-attribute) and has a data structure showing the total amount of these input-attribute pairs (number of video stream inputs (number_of_video_stream_entries), number of audio stream inputs (number_of_audio_stream_entries) , the number of inputs of the PG stream (number_of_PG_stream_entries), the number of inputs of the IG stream (number_of_IG_stream_entries)).

The input-attribute pairs correspond to each of the video streams, audio streams, PG streams and IG streams that can be played in the playitem, as shown by the symbol "{" in the figure.

The following describes the detail of the input attribute.

FIG. 19B shows a stream attribute (Stream_attribute) corresponding to a video stream.

The stream attribute (Stream_attribute) of the video stream includes “Video_format” (“video format”) indicating the display format of the video stream, and “frame_rate” (“frame rate”) indicating the frequency for displaying the video stream.

Fig. 19C shows a stream attribute (Stream_attribute) corresponding to an audio stream.

The stream attribute (Stream_attribute) of an audio stream is composed of: “stream_coding_type” (“stream encoding type”) indicating a method for encoding an audio stream; “Audio_presentation_type” (“audio presentation type”) indicating a channel structure of a corresponding audio stream; “Sampling_frequency”, indicating the sampling rate of the corresponding audio stream, and “audio_language code”, indicating the attribute of the audio stream language.

Fig. 19D shows an input of a stream (Stream_entry) of an audio stream. As shown in the figure, the input of the stream (Stream_entry) of the video stream includes “ref_to_Stream_PID_of_Main_Clip” (“reference to the PID of the main clip stream”) indicating the PID used to demultiplex the video stream.

The stream attribute (Stream_attribute) of the audio stream, IG stream, and PG stream multiplexed into the main clip has the format shown in FIG. 19D.

<Restriction on the amount of data of elementary streams allowed for playback>

The STN table (STN_table) shows, among the elementary streams read from the BD-ROM and the local storage, allowed for playback. However, if the STN table (STN_table) provides elementary streams with the ability to play without limitation, the decoder system may be disabled.

The reason for this is as follows. According to the MPEG2 decoder system standard, overlap between TS packets on the ATC time axis in the same transport stream is not allowed. This is a basic principle in order to make the decoder system perform the proper decoding process. On the other hand, in the case where it is possible for both, to play a stream read from a BD-ROM and to play a stream read from a local storage device, and furthermore, play an AV clip read from a BD-ROM and play an AV clip, read from the local storage device are performed simultaneously, an overlap is created between the TS packet from the BD-ROM and the TS packet from the local storage device.

Under the condition of this factor, the following restriction is imposed on elementary decoding streams.

The elementary decoding streams are a video stream, an audio stream, a PG stream, and an IG stream that were enabled in the STN table (STN_table) for playback and were selected for simultaneous playback. Some elementary decoding streams are read from the local storage device, others are read from the BD-ROM.

The limitation imposed on elementary decoding streams is that the number of bits of TS packets (TS decoding packets) constituting an AV clip (main clip, auxiliary clip), which includes the elementary streams allowed, in the STN table (STN_table) , for simultaneous playback, but does not include elementary streams not allowed for playback, should be 48 Mbit / s or less.

The one second time unit is called “Window” and can be located anywhere on the time axis of the ATC sequence. That is, the number of bits of the elementary decoding streams for one second at any time should be 48 Mbit or less.

FIG. 20 shows TS packets read from the BD-ROM and from the local storage device, and illustrates from these TS packets those to be supplied to the decoder. Level 1 of the figure shows multiple TS packets read from the BD-ROM; level 3 shows multiple TS packets read from a local storage device. Of the TS packets at levels 1 and 3, the shaded ones in the figure are TS packets constituting an elementary decoding stream (TS decoding packets). Layer 2 in the figure shows from the TS decoding packets shown in layers 1 and 3 occurring over a period of one second. As described above, according to the MPEG2 decoder system standard, overlap is not allowed between TS packets on the ATC time axis in the same transport stream. However, it can be seen from the figure that overlaps rp1, rp2 and rp3 between the TS packets occur on the time axis of the ATC. Thus, overlaps in operations on TS packets are allowed within a unit window time. However, another requirement is imposed that does not apply to the MPEG2 decoder system standard. That is, the above limitation is 48 Mbps or less. Level 4 represents mathematical expressions of the conditions that TS decoding packets must satisfy. Mathematical expressions indicate that the value obtained by converting the total amount of the aforementioned TS decoding packets to the total amount of bits (the total sum of TS decoding packets is multiplied by the number of bytes of the TS packet, 188, and the result is expressed in 8 bits) is 48 Mbit or less.

The imposition of the aforementioned condition on decoding TS packets for any period of one second is a limitation of the number of bits according to the present embodiment. When the authoring is performed for a supercompression application, it is checked whether the number of bits of the TS decoding packet is 48 Mbps or less for a period of one second, while observing a window shift in the sequence of source packets each time by one packet. When the constraint is satisfied, the window shifts to the next TS packet. If the restriction is not satisfied, it is determined that there is a violation of the BD-ROM standard. When the window's End Time (Out_Time) reaches the last source packet after repeating such shifts, it is determined that the source packets are BD-ROM compliant.

<Window shift>

An ATS having a temporal accuracy of 27 MHz is attached to each TS packet. The coordinates on the ATC time axis have a time accuracy of 1/27000000 seconds; however, ATS is not always present at every coordinate on the time axis of the ATC. On the ATC time axis, periods not containing ATS and periods containing ATS appear irregularly. The appearance of ATS varies, and therefore, when the window shifts, how to adjust the window's end time (Out_Time), it becomes a problem in the case where ATS is absent 1 second after the start time (In_Time).

The end time (Out_Time) of the window is, in principle, set to be 1 second after the start time (In_Time). Here, if the ATS is present on the ATC time axis in a coordinate corresponding to 1 second after the start time (In_Time), the coordinate in In_Time + 1 second is set as the end time (Out_Time). If there is no ATS at the coordinate corresponding to 1 second after the start time (In_Time), the coordinate at which ATS appears on the ATC time axis for the first time after In_Time + 1 second is set as the end time (Out_Time). Since the window shift end time (Out_Time) is adjusted to take into account time periods during which no ATS is present, different bit values are calculated each time the window is shifted. The start time (IN_Time) is shifted each time by one TS packet, and the end time (Out_Time) is adjusted according to the shift, and therefore, the modulation of bit values on the ATC time axis can be calculated with accuracy.

21A-21D show window shifts. In each of the figures 21A through 21D, the upper part shows the sequence of the source packet, which is the target for verification, and the lower part shows the start time (In_Time) and the end time (Out_Time) of the window. In FIG. 21A, the window start time (In_Time) defines the source packet #i. Package #j TS corresponding to 1 second after the start time (In_Time) of the window is set as the end time (Out_Time) of the window.

In FIG. 21B, the window start time (In_Time) defines the source packet # i + 1. On the other hand, there is no ATS in the coordinate corresponding to the original packet # j + l, which is 1 second after the window's start time (In_Time). The end time (Out_Time) of the window of FIG. 21B should specify one TS packet outside TS packet #j; however, since the source packet is not present immediately after the TS packet #j, the bit rate of the window of FIG. 21B becomes lower than the bit rate of the window of FIG. 21A. In this case, there is no point for the window of FIG. 21B with verification. Under the condition of this factor, by setting the window end time (Out_Time), the packet # j + 2 TS, which appears for the first time after 1 second from the window start time (In_Time), is set as the end time point (Out_Time). Setting the end time (Out_Time) in this way makes the window check of FIG. 21B appropriate for execution.

In FIG. 21C, the window start time (In_Time) sets the source packet # i + 2. On the other hand, the packet # j + 2 TS is located at the position corresponding to 1 second after the start time (In_Time) of the window. The total amount of TS packets for the window of FIG. 21C is the same as that of the window of FIG. 21B, and therefore there is no point to perform the check. Accordingly, no verification is performed in FIG. 21C, and the window start time (In_Time) is shifted.

In FIG. 21D, the window start time (In_Time) defines the source packet # i + 3. On the other hand, no source package is present at the position corresponding to source package # j + 3, which is 1 second after the window's start time (In_Time). Under the condition of this factor, by controlling the window end time (Out_Time) in the manner described above, the packet # j + 4 TS, which appears for the first time after 1 second from the window start time (In_Time), is set as the end time point (Out_Time). By this, the total amount of TS packets in the window becomes different from that for the window of FIG. 21B, and the window check of FIG. 21D is made suitable for execution.

By checking the number of bits with a window shift in the manner described above when authoring is performed, it is ensured that no negative overflow or overflow is caused when TS packets are read from the local storage and the BD-ROM and supplied to the decoder.

Providing window shift is described below with reference to the individual examples of FIGS. 22-26.

Level 1 in FIG. 22 is a graph showing temporal modulation regarding the amount of data of the TS packets read from the BD-ROM, as well as the data volume of the TS packets read from the local storage device. The horizontal axis is time, and the vertical axis is the transmission volumes relative to each point on the time axis. On the graph of the number of bits at the point in time when the TS packets are read from the BD-ROM and the local storage device, modulation is tested as indicated by dashed curves.

Level 2 in FIG. 22 shows the total amount of data from the number of TS packets read from the BD-ROM and the local storage, TS packets to be supplied to the decoder. Temporal modulation of the total transmission volume is as shown by the solid line. The total amount of data is the total amount of packets belonging to the streams that were allowed in the STN table (STN_table). In the worst case, the total amount of transmission could come close to 95 Mbit, and TS packets containing this amount of data could be fed to the decoder. Here, the time axis of the graph is divided into seven windows and a comparison is made between the amount of feed in each window and the transmitted volume for each window.

Level 3 in FIG. 22 is a graph of a level divided up to every 1 second. 23A and 23B show a comparison between the transmitted volume and the volume supplied to the decoder during each window. The transmitted volume for the window is 48 Mbit per second and is 96 Mbit if the volume is converted to bits in 0.5 seconds. The hatch pattern pn1 in the figure indicates the amount of data supplied to the decoder. The hatch pattern pn2 in the figure indicates the transferred volume in each window. In any window, the area with the hatch pattern pn1 has the same or less area than the area with the hatch pattern pn2. This indicates that the amount of data supplied from the BD-ROM and the local storage device is limited to the transferred amount or less in any window.

At any point on the ATC time axis, the transmitted volume to the decoder is 48 Mbps or less. Therefore, even if the transmitted volume to the decoder locally fits close to 96 Mbit, transmission at 96 Mbit never lasts for 0.5 second, which is confirmed by the calculation of 48 Mbit = 96 bit × 0.5 second. Accordingly, if the decoder performs a preliminary read operation to pre-read the source packets from the BD-ROM and the local storage device before the peak is reached, no negative overflow or overflow is caused in the decoder buffer.

The transmitted volume in each window, i.e. 48 Mbit / s, was determined using as a guide the volume that the MPEG-based decoder can read into the buffer in advance. If the amount of data that can be read into the buffer in advance is large, the volume of data per second can be made large or the period for the window can be set large. Thus, the present invention is not limited to a speed of 48 Mbps.

This is a description of the limitation of the amount of data on the secondary TS, which in the STN table (STN_table) is given the opportunity to play.

<Configuring connection condition information (connection_condition) and connection condition information SP (sp_connection_condition)>

The following describes the settings for the connection condition information (connection_condition) in the playitem and the connection condition information SP (sp_connection_condition information) in the auxiliary playitem for implementing the overcompression application. Fields of information about the connection condition (connection_condition) and information about the condition of the connection SP (sp_connection_condition information) can take the values "1", "5" and "6", the meanings of which are as follows.

connection_condition = 1 (CC = 1): There is no guarantee for a seamless connection between the playitem (current playitem) and the immediately preceding playitem (previous playitem). That is, this is a connection mode, which provides an opportunity to occur in a freeze frame, and playback is interrupted (disconnected connection).

connection_condition = 5 (CC = 5): There is a guarantee for a seamless connection between the video stream, the PG stream and the IG stream multiplexed into the main clip of the current playitem and the video stream, PG stream and the IG stream multiplexed into the main clip of the previous playitem. On the other hand, this is not true with an audio stream multiplexed into the main clip.

connection_condition = 6 (CC = 6): The corresponding TS streams belonging to the current playitem and the previous playitem are logically continued (they are continuous on the time axis, and the encoding methods are the same), and there is a guarantee for a seamless connection as audio, and video streams.

The connection condition information SP (sp_connection_condition) recorded in the sub play element can be determined as follows.

Connection condition information SP (sp_connection_condition) (SP_CC = 1): There is no guarantee for a seamless connection between the sub playback element (the current sub playback element) and the immediately preceding sub playback element (the previous sub playback element).

Connection condition information SP (sp_connection_condition) (SP_CC = 5): There is a guarantee for a seamless connection between the PG stream and the IG stream multiplexed into the auxiliary clip of the current auxiliary playback element, as well as the PG stream and IG stream multiplexed into the auxiliary clip of the previous auxiliary element reproduction. On the other hand, this is not the case with an audio stream multiplexed into an auxiliary clip.

Connection condition information SP (sp_connection_condition) (SP_CC = 6): Corresponding TS streams belonging to the current sub playback element and the previous sub playback element are logically continuous (they are continuous on the time axis and the encoding methods are also the same), and there is a guarantee for seamless connection of both audio and video streams.

The sub-play item that must be set for the play item that implements the overcompression application will not cause a mismatch even if the video stream, audio stream, PG stream or IG stream of the sub play element is within the play item. Therefore, they have identical connection conditions. That is, if the playitem # 1 and the playitem # 2 are connected according to CC = 1, the auxiliary playitem # 1 and the auxiliary playitem # 2 corresponding thereto are also connected according to CC = 1. Similarly, if the playitem # 1 and the playitem # 2 are connected according to CC = 5, the corresponding auxiliary playitem # 1 and the auxiliary playitem # 2 are connected when the condition CC = 5 is satisfied.

The following describes the relationship of the start times (In_Time) and the end times (Out_Time) of the playitems and sub-playitems constituting the overcompression application, as well as the details of the connection condition information (connection_condition), with reference to FIGS. 24, 25 and 26.

<The relationship of the start time (In_Time) and the end time (Out_Time)>

Fig. 24 shows a connection state of reproducing elements and sub-reproducing elements constituting over-compaction. The figure 1 level is the time axis of the sub clip; and levels 2 and 3 are the time axis of the sub play item and the time axis of the play list, respectively. Level 4 is the time axis of the main clip. In the figure, in the case where the connection condition information (connection_condition) of the playitem is “= 5”, the connection condition information (connection_condition) of the sub playitem is also SP_CC = 5.

Fig. 25 shows the relationship between the start time points (In_Time) and the end time points (Out_Time) of the playitems and the start time points (In_Time) and the end time (Out_Time) of the auxiliary playitems in the case where the connection condition information (connection_condition) of the item the playback and connection condition information SP (sp_connection_condition) of the sub playback item shown in FIG. 24 are set to “= 5”. Levels 1 and 4 are the same as those in FIG. Of the two playitems (playitem information # 1 and playitem information # 2) shown in FIG. 24, playitem information # 1 contains a start time (In_Time) indicating a time t1 and an end time ( Out_Time) indicating the time t2. The start time corresponding to the information # 2 about the playitem indicates the time t3, and the end time (Out_Time) corresponding to the information # 2 of the playitem indicates the time t4.

When the connection state of the playitem is CC = 5, the initial PTS of the synchronization of the playitem (Sync_Start_Pts_of_PlayItem) of the sub playitem indicates the same time point as the start time (In_Time) of the playitem. The start time (In_Time) and the end time (Out_Time) of the sub play item show the same time points as the start time (In_Time) and the end time (Out_Time) of the play item. Thus, in the case where the connection condition information (connection_condition) of the playitem is “= 5”, the connection condition information SP (sp_connection_condition) of the auxiliary playitem is also set to “= 5”, and the start time (In_Time) and the time point endings (Out_time) indicate the same time points as the starting time point (In_Time) and the ending time point (Out_Time) of the sub play element.

The start time (In_Time) and the end time (Out_Time) of the playitem and the start time (In_Time) and the end time (Out_Time) of the sub playitem refer to the PTS of the video presentation block and the audio presentation block, respectively. The start time (In_Time) and the end time (Out_Time) of the playitem and the start time (In_Time) and the end time (Out_Time) of the sub playitem consistent with each other mean that the PTS values of the video presentation block and the audio presentation block indicated by the reference point in time (In_Time) and the point in time (Out_Time) of the playitem are the same as the PTS values of the video presentation unit and the audio presentation block indicated by the reference time point the start (In_Time) and the end time (Out_Time) of the sub play item. In this case, it is necessary that the primary TS and secondary TS are encoded so that they have the same length of time, and that the PTS of the video presentation unit and the audio presentation unit are motivated to be the same when the authoring is performed. Creating a primary TS and a secondary TS in this way is also a condition for implementing CC = 5 and SP_CC = 5.

<STC values to be referenced for synchronous playback>

FIG. 26 shows an STC value to be referenced when a part existing from a start time (In_Time) to an end time (Out_Time) of a playitem is reproduced, and an STC value to be referenced when a part existing from a moment is played. start time (In_Time) to the end time (Out_Time) of the sub play item. Levels 2 and 3 are the same as in the previous figure. Level 1 shows in graphical form the value of the STC, which should be indicated by reference when playing back the part that exists from the start time (In_Time) to the end time (Out_Time) of the sub play element. Level 4 shows in graphical form the value of the STC, which should be indicated by reference when playing back the part that exists from the start time (In_Time) to the end time (Out_Time) of the playitem. The horizontal axis of level 1 is the time axis, and the vertical axis shows the STC values relative to each point in time on the time axis. Level 1 STC values include a monotonic increase in zk1 from the start time point (In_Time) to the end time (Out_Time) of the auxiliary playback element information # 1 and a monotonic increase in zk2 from the start time point (In_Time) to the end time (Out_Time) of information # 2 elements of auxiliary playback. Level 4 STC values include a monotonic increase in zk3 from the start time point (In_Time) to the end time (Out_Time) of the playitem item # 1 and a monotonic increase in zk4 from the start time point (In_Time) to the end time (Out_Time) of the information # 2 play item.

Since the start time (In_Time) of the playitem indicates the same point in time with the start time (In_Time) of the sub playitem, the initial STC values in the above graph are the same, and the STC value at the intermediate time points is also the same. That is, STC2 (i), which is the STC value, which should be indicated by reference, when the source packet located at the discretionary time moment i between the start time (In_Time) and the end time (Out_Time) of the playitem is supplied to the decoder, is the same as STC1 (i), which is the STC value to be referenced when the source packet located at the same time i between the start time (In_Time) and the end time (Out_Time) of the auxiliary element The performance is fed to the decoder. When the STC values are the same, all STC counters in the device must be kept in such a state that they create the same values of the clock signal and feed them to the demultiplexing units, thereby simplifying the controls in the playback device.

Presumably speaking, in the case where two or more sub-play items are prepared for one play item as opposed to the controls illustrated in FIGS. 25 and 26, video and audio are interrupted at the border of these sub-play items, and inconveniences such as pausing playback in the middle playback element - will occur as a result. Additionally, when the primary TS substitution process is implemented in an over-compaction application, the time axis of the STC must change during substitution, which leads to more complex synchronous controls in the reproducing apparatus.

On the other hand, by determining both the start time (In_Time) and the end time (Out_Time) of the playitem or auxiliary playitem on the continuous time axis of the STC, it is possible to prevent the aforementioned inconveniences, i.e. interruption of video and audio, as well as the replacement of transport streams . Due to these situations, with respect to one playitem, one auxiliary playitem is assigned having the same start and end times as those of the playitem.

<Inaccuracies of the start time (In_Time) and the end time (Out_Time)>

Here, an exact match between the start time point (In_Time) and the end time point (Out_Time) of the playitem and those of the auxiliary playitem is not required, and some level of errors may be acceptable. The following describes the errors of the start time (In_Time) and the end time (Out_Time).

The STC time instants of the start time (In_Time) and the end time (Out_Time) of the playitem are set for the video frames of the playitem. On the other hand, the time characteristics of the start time point (In_Time) and the end time point (Out_Time) of the sub play element are set for the audio frames of the sub play element. This is because the auxiliary playback element is mainly used for director's explanations, and therefore, it is a frequent case that the video stream is not multiplexed into it. In this case, due to, in the sense of accuracy, the differences in the duration of the reproduction period of the respective presentation blocks, the time moments of their start and end do not coincide with each other. Accordingly, it is necessary to provide the possibility of an error of at least one frame. The start and end times of the playitem #n and the auxiliary playitem #n are also set on the same time axis STC, as follows:

| (PlayItem # n.Out - PlayItem # n.In) - (SubPlayItem # n.Out_time -SubPlayItem # n.In_time) | ≤ a playback period of 1 progressive frame or two interlaced fields of a video having the shortest playback period in the playback element #n ≤1 / 60 seconds. For the value on the left side, a playback period of 1 progressive frame or two interlaced fields of a video having the longest playback period in the playback element #n (≤1 / 25) can be used, or the value can be set to be 1 second or less.

This is a description of the relationship between the start times (In_Iime) and the end times (Out_Time) of the playitem and sub playitem.

The following describes in detail the connection condition information (connection_condition) and the connection condition information SP (sp_connection_condition). In order to meet the requirements of CC = 5 and SP_CC = 5, the following conditions must be satisfied at all levels of the AV stream, transport stream, video presentation unit and audio presentation unit, and elementary stream.

<AV stream level>

The connection condition information (connection_condition) of the current playitem and the connection condition information (sp_connection_condition) set to “5” mean that there is a “clean break” between the end point of the AV stream played in the previous playitem and the start point The AV stream being played in the current playitem.

In order to realize a clean break, the AV stream played in the previous playitem and the AV stream played in the current playitem must satisfy the following requirements.

(1) An optional access unit is missing at the end point of the main clip specified in the previous playitem, and the optional access unit containing PTS has been excluded from the period following the Out_Time of the previous playitem.

Similarly, the optional access unit is missing at the end point of the sub clip specified in the previous sub-item, and the optional access unit containing PTS has been excluded from the period following the end time (Out_Time) of the previous sub-item.

(2) At the beginning of the AV stream specified in the current playitem, an optional access unit containing PTS was excluded from the period preceding the start time (In_Time) of the current playitem. In addition, the first audio presentation block of the main clip includes a sample to be played at the start time (In_Time) on the time axis of the STC.

Similarly, at the beginning of the AV stream specified in the current sub-play item, an optional access unit containing PTS was excluded from the period preceding the start time (In_Time) of the current sub-play item. In addition, the first sub-audio clip presentation unit includes a sample to be reproduced at the start time (In_Time) on the time axis of the STC.

(3) The source packets constituting the main clip specified in the previous playitem must be multiplexed in some way so that all of them are received in the decoder system before the first packet of the main clip specified in the current playitem is sent to the decoder.

Similarly, the data of the auxiliary clip specified in the previous auxiliary playback element must be multiplexed in some way so that all of them are received in the decoder system before the first packet of the auxiliary clip specified in the current auxiliary playback element is sent to the decoder.

This is a description of the conditions that must be satisfied at the level of the AV stream. The following describes conditions that must be satisfied at the level of traffic flows.

<Traffic Flow Level>

Here, two primary TSs, which are the targets of a seamless connection when CC = 5, are called primary TS1 and primary TS2. The two primary TSs that are the targets of the continuous connection when SP_CC = 5 are called secondary TS1 and secondary TS2.

Fig. 27 shows how TS1 and TS2 are identified in an AV clip indicated by a link in a previous playitem and a previous sub playitem and an AV clip indicated by a link in the current playitem and the current auxiliary playitem. Level 4 in the figure shows primary TS1 and primary TS2; and level 3 shows the main clip 1 of the previous playitem and the main clip 2 of the current playitem. Level 1 shows the secondary TS1 and secondary TS2; and level 2 shows the sub clip 1 of the previous sub play item and the sub clip 2 of the current sub play item.

Primary TS1 consists of a chunk of data that is hatched in the main clip 1 in the figure. The data portion in the main clip 1 begins with the source packet, from which the decoding of the start time (In_Time) in the previous playitem can begin. This source package is located at the beginning of the video presentation unit and the audio presentation unit, which are indicated by reference at the start time point (In_Time). Then the data portion ends with the last packet of the main clip 1.

Primary TS2 consists of a chunk of data that is hatched in the main clip 2 in the figure. This piece of data in the main clip 2 begins with the first source packet of the main clip 2. In this case, the piece of data in the main clip 2 ends with the original packet, at which the decoding of the current playitem ends. This source package is a source package located at the end of the video presentation unit and the audio presentation unit, which are indicated by the reference point in time (Out_Time) of the current playitem.

Secondary TS1 consists of a piece of data that is hatched in the auxiliary clip 1 in the figure. The data portion in the auxiliary clip 1 begins with the source packet, from which the decoding of the start time (In_Time) in the previous auxiliary playback element can begin. This source package is located at the beginning of the video presentation unit and the audio presentation unit, which are indicated by reference at the start time point (In_Time). Then, the data portion ends with the last packet of the auxiliary clip 1.

Secondary TS2 consists of a chunk of data that is shaded in sub clip 2 in the figure. This data portion in the auxiliary clip 2 begins with the first source packet of the auxiliary clip 2. In this case, the data portion in the auxiliary clip 2 ends with the original packet, at which the decoding of the current playitem ends. This source package is the source package located at the end of the video presentation unit and the audio presentation unit, which are indicated by the reference point in time (Out_Time) of the current sub-play element.

According to the description above, it can be understood how transport streams are arranged, which should be connected in the main clip and the auxiliary clip when CC = 5 and SP_CC = 5. The main clip of the previous playitem should end with the video presentation unit and the audio presentation unit, which are indicated by the reference time point (Out_Time) of the previous playitem, and the main clip of the current playitem should start with the video presentation unit and the audio presentation unit, which are indicated by the point in time start (In_Time) of the current playitem. This relationship is also valid for the previous subplay item. That is, the auxiliary clip of the previous auxiliary playback element must end with an audio presentation unit, which is indicated by the reference time point of the end (Out_Time) of the previous auxiliary playback element, and the auxiliary clip of the current auxiliary playback element must start with the audio presentation unit, which is indicated by the reference time of the start time (In_Time ) of the current sub play item. This is because the optional audio presentation unit does not have to be present at or after the video presentation unit and the audio presentation unit, which are indicated by the reference point in time (Out_Time) of the previous auxiliary playback element, as described above. On the other hand, the auxiliary clip of the previous auxiliary playback element should not start with the audio presentation block, which is indicated by the reference point in time (In_Time) of the previous auxiliary playback element, and the auxiliary clip of the current auxiliary playback element should also not end with the audio presentation block, which is indicated by the link the end time (Out_Time) of the current sub-play item.

24 and 27, the primary TS and the secondary TS must be created having the same time duration, and the PTS values of the video presentation unit and the audio presentation unit must be created having the same value. In addition, the main clip of the previous playitem and the auxiliary clip of the previous playitem must be multiplexed so as to end with a video presentation unit and an audio presentation unit corresponding to the end time (Out_Time). The main clip of the current playitem and the auxiliary clip of the current playitem must be multiplexed in such a way as to start with the video presentation block and the audio presentation block corresponding to the start time (In_Time).

Additionally, these traffic flows must meet the following conditions:

- the number of programs in TS1 and TS2 is one;

- the number of video streams is one;

- the number of audio streams is the same;

- the contents of the STN table (STN_table) of the previous playitem is the same as that of the STN table (STN_table) of the current playitem, and

- the playback period of the transport stream in each playitem is three seconds.

These are conditions that must be satisfied at the level of transport streams for connecting two streams when CC = 5 and SP_CC = 5. Next, conditions that must be satisfied at the level of the video presentation unit and the audio presentation unit are described.

<Video Presentation Block and Audio Presentation Block Level>

Although the starting time of the last video presentation block in the primary TS1 video stream is initially different from the ending time of the first video representing block in the primary TS 2 video stream, CC = 5 makes the starting time and ending time coincide with each other. When the end point in time and the start time point of the video presentation blocks are made to coincide with each other, it becomes a problem how to process such video presentation blocks and audio presentation blocks for synchronous playback. This is because video and audio have different sampling rates, and the durations of the video presentation block and the audio presentation block do not coincide with each other.

Fig. 28 shows details on CC = 5 and SP_CC = 5. Levels 1 through 3 show the connection condition (connection_condition) of the sub play item, and levels 4 through 7 show the connection condition SP (sp_connection_condition) in the play item. Level 4 shows numerous video presentation blocks from TS1 and TS2, and level 5 shows audio presentation blocks in TS1 and audio presentation blocks in TS2. Level 6 shows the STC values in the main clip. Level 7 shows the sequence of source packets of the main clip.

The shaded parts in the figure depict video presentation blocks, audio presentation blocks, and TS1 source packets, while the non-shaded parts depict video presentation blocks, audio presentation blocks, and TS2 source packets.

In the figure, CC = 5 means a state in which the video presentation blocks are aligned to have a common border (level 4), there is a gap between the ATCs in the main clip (level 7), and there is an overlap between the audio presentation blocks in the main clip (level 5) . SP_CC = 5 means a state in which there is a gap between the ATCs in the auxiliary clip (level 1), and there is an overlap between the audio presentation block in the auxiliary clip (level 2).

The aforementioned boundary between the blocks of the video presentation is located, from the point of view of TS1, at the end point PTS1 (1st end) + Tpp of the last block of the presentation of the video level 4, and is located, from the point of view of TS2, at the starting point PTS2 (2nd beginning) video presentation block level 4.

Assume that in TS1, the endpoint of the audio presentation block corresponding to the boundary time moment T4 is T5a, and in TS2, the starting point of the audio presentation block corresponding to the time moment T4 is T3a. Here, the overlap of the audio presentation blocks in the main clip extends from T3a to T5a.

In the figure, each sub-audio clip presentation unit is set longer than each sub-clip audio presentation unit. This is because the audio stream of the auxiliary clip is set to have a low sampling rate because it is delivered via the network, and accordingly, the time period for each audio presentation unit becomes longer. In the sequence of level 1 packets, there is a gap similar to that at level 7. In addition, in the level 2 audio presentation blocks there is an overlap similar to that at level 4. Suppose that in TS1 of the auxiliary clip, the end point of the audio representation block corresponding to the boundary time moment T4 is T5b, and in TS2 of the auxiliary clip, the starting point of the audio presentation block corresponding to time point T4 is T3b. Here the overlap extends from T3b to T5b.

From the figure it can be seen that, in order to implement CC = 5 and SP_CC = 5, the following four conditions must be satisfied at the levels of video presentation blocks, audio presentation blocks, and packets.

(1) The last unit for presenting the audio audio stream in TS1 includes a sample having a playback time that coincides with the end of the display period of the last movie frame in TS1 specified in the previous playitem and the previous auxiliary playitem.

(2) The first unit for presenting the audio audio stream in TS2 includes a sample having a playback time that coincides with the start of the display period of the frames of the first movie frame in TS1 specified in the current playitem and the current sub playitem.

(3) There is no gap at the junction point in the sequence of audio presentation blocks. This means that overlapping in the sequence of audio presentation blocks may occur at the junction point. However, the length of such an overlap should be shorter than the playback period of two audio frames.

(4) The first TS2 packet includes a PAT, which may be directly followed by one or more PMTs. If the PMT is larger than the payload of the TS packet, the DMT may be divided into two or more packets. A TS packet containing PMT may include PCR and SIT.

<The relationship of the start time (In_Time) and the end time (Out_Time) with the video submission unit>

Fig. 29 shows the relationship between the plurality of video presentation units defined by the previous playitem and the current playitem, the multiple audio presentation units, and the time axis STC. Level 1 shows multiple video presentation blocks belonging to TS1 referenced by the previous playitem and multiple video presentation blocks belonging to TS2 referenced by the current playitem. Level 2 shows multiple audio presentation units belonging to a time stamp referenced by a previous sub-play item and multiple audio presentation units belonging to TS2 referenced by the current sub-play item. Level 3 shows the time axis STC of TS1 in the previous sub-play item and the time axis STC of TS2 in the current sub-play item. As shown in FIG. 28, within the audio presentation blocks from TS1 and the audio presentation blocks from TS2 at level 2, the portion from the start point T3b to the end point T5b overlaps. The start time (In_Time) of the current sub-play element and the end time (Out_Time) of the previous sub-play element respectively set the time moment T4, which is the boundary of the video presentation blocks. Since the start time (In_Time) of the current sub-play item and the end time (Out_Time) of the sub-play item also set the time moment T4 of the border of the video presentation blocks, the start time (In_Time) and the end time (Out_Time) of the play item coincide with the time the start (In_Time) and the end time (Out_Time) of the sub play item. Thus, although the start time (In_Time) of the previous sub play item and the end time (Out_Time) of the current sub play item are recorded on a recording medium other than the BD-ROM, it can be seen that they correspond to the boundary of the video presentation blocks in the main clip, and also correspond to the end time (Out_Time) of the previous playitem and the start time (In_Time) of the current playitem, respectively.

This is a detailed description of the conditions that must be satisfied at the level of video presentation blocks and audio presentation blocks.

<Elementary Flow Level>

The following describes the encoding conditions at the elementary stream level in order to implement CC = 5 and SP_CC = 5.

The following encoding conditions must be satisfied at the level of each elementary stream.

(1) Video Stream

- the video resolution and frame rate do not change before and after a seamless connection, and

- the video stream immediately after a seamless connection ends with the sequence completion code (sequence_end_code) (for MPEG-2 Video) and the end of the RBSP sequence (end_of_sequence_rbsp) (for MPEG-4 AVC).

(2) Audio stream

- the encoding format of audio streams having the same PID does not change and

- sampling frequency, quantization bit rate and number of channels do not change.

(3) PG stream

a) The number of PG streams in TS1 and TS2 is the same.

b) The PG stream from TS1 ends with a functional segment called the “End of Display Set”.

c) The PTS of a PES packet carrying the last PCS in TS1 indicates a point in time before a playback time corresponding to an Out_Time of a previous playitem and a previous sub-playitem.

d) The PG stream from TS2 shall begin with a mapping set of Epock origin type or Epock continuation type.

e) The PTS of the PES packet carrying the first PCS in TS2 indicates a point in time after or after a play time corresponding to a start time (In_Time) of the current playitem and the current sub playitem.

f) The extraction of source packets from TS1, which is accompanied by the extraction of source packets from TS2, can be defined as STC1 and STC2 on the same system time axis, and there is no overlap in their DTS / PTS values.

(4) Stream IG

a) The number of IG streams in TS1 and TS2 is the same.

b) The IG stream from TS1 ends with a functional segment called the “End of Display Set”.

c) The PTS of a PES packet carrying the last ICS in TS1 indicates a point in time before a playback time corresponding to an Out_Time of a previous playitem and a previous sub playitem.

d) The IG stream from TS2 shall begin with a mapping set of the Epock origin type or Epock continuation type.

e) The PTS of the PES packet carrying the first ICS in TS2 indicates a time point in or after the play time corresponding to the start time (In_Time) of the current playitem and the current sub playitem.

f) Retrieving the source packet from TS1, which is accompanied by retrieving the source packet from TS2, can be defined as STC1 and STC2 on the same system time axis, and there is no overlap in their DTS / PTS values.

In order to connect the previous playitem and the current playitem at CC = 5 and connect the previous auxiliary playitem and the current auxiliary playitem with SP_CC = 5, all of the above conditions for the levels of the AV stream, transport stream, video presentation blocks and audio presentation blocks , and elementary flows must be satisfied.

This is an explanation of the information of the playlist, which is a component of the contents of the storage device of the local storage device 200.

This is an explanation of the recording medium according to the present invention. The following explains the playback device of the present invention.

30 shows an internal structure of a reproducing apparatus of the present invention. The reproducing apparatus of the present invention is manufactured on an industrial scale based on the internal structure shown in the figure. The playback device consists mainly of two parts - the system LSI (large integrated circuit, LSI) and the drive device, and can be produced on an industrial scale by installing these parts in the housing and base of the device. System LSI is an integrated circuit that combines a variety of processing modules to perform the functions of a playback device. A reproducing device manufactured in this way comprises: a BD-ROM drive 1a; read buffers 1b and 1c; ATC counters 2a and 2c; depacketizing agents 2b and 2d of the source; 2c and 2d ATC counters; counters 3a and 3c STC; 3b and 3d PID filters; video decoder 4; transport buffer (TB) 4a; multiplexed buffer (MB) 4b; encoded movie frame buffer 4c (CPB); video decoder 4d; reordering buffer 4e; 4f switch video card 5; audio decoder 9; transport buffer 6; elemental buffer 7; decoder 8; switches 10a, 10b, 10c and 10d; interactive graphics decoder 11; transport buffer (TB) 11a; coded data buffer 11b (CDB); processor 11c streaming graphics (SGP); buffer 11d objects; layout buffer 11e; graphics controller 11f; board 12 interactive graphics; demo graphics decoder 13; transport buffer (TB) 13a; coded data buffer 13b (CDB); processor 13c streaming graphics (SGP); buffer 13d objects; layout buffer 13e; graphics controller 13f; board 14 demo graphics; transport buffer 15a; elemental buffer 15b; decoder 15c; transport buffer 16a; elemental buffer 16b; decoder 16c; synthesis module 17; memory 21; controller 22; set of 23 PSR; PID conversion module 24; network module 25; an operation receiving unit 26 and a local storage 200.

The BD-ROM drive 1a loads / ejects the BD-ROM and accesses the BD-ROM.

Read buffer 1b (RB) accumulates source packet sequences read from the BD-ROM.

Read buffer 1c (RB) accumulates source packet sequences read from the last playback chapter.

The ATC counter 2a is reset by using the ATS of the source packet located at the beginning of the playback section within the source packets constituting the primary TS, and later outputs the ATC to the source depacketizer 2b.

The source depacketizer 2b extracts the TS packets from the source packets constituting the primary TS and sends the TS packets. Upon sending, the source depacketizer 2b adjusts the input time to the decoder according to the ATS of each TS packet. To be more precise, when the ATC value generated by the ATC counter 2a becomes the same as the ATS value of the source packet, the source depacketizer 2b only sends the TS packet to the PID filter 3b at the TS recording speed (TS_Recording_Rate).

The ATC counter 2c is reset by using the ATS of the source packet located at the beginning of the playback section within the source packets constituting the secondary TS, and later outputs the ATC to the source depacketizer 2d.

The source depacketizer 2d extracts the TS packets from the source packets constituting the secondary TS, and sends the TS packets. Upon sending, the source depacketizer 2d adjusts the input time to the decoder according to the ATS of each TS packet. To be more precise, when the ATC value generated by the ATC counter 2c becomes the same as the ATS value of the source packet, the source depacketizer 2d only sends the TS packet to the 3d PID filter at the TS recording speed (TS_Recording_Rate).

The STC counter 3a is reset by the PCR of the primary TS and outputs the STC.

The PID filter 3b is a demultiplexing unit for the primary TS and deduces from the number of source packets output from the source depacketizer 2b those having PID reference values reported by the PID conversion module 24 to video decoder 4, audio decoder 9, interactive graphics decoder 11 and decoder 13 demo graphics. Each of the decoders receives elementary streams passed through the PID filter 3b and performs from decoding processing to reproduction processing according to the PCR of the primary TS (time axis STC1). Thus, the elementary streams introduced into each decoder after passing through the PID filter 3b are decoded and reproduced based on the PCR of the primary TS.

The STC counter 3c is reset by the PCR of the secondary TS and outputs the STC. The 3d PID filter performs demultiplexing based on this STC.

The 3d PID filter is a demultiplexing unit for the auxiliary clip and deduces from the source packets output from the source depacketizer 2d those having PID reference values reported by the PID conversion module 24 to an audio decoder 9, an interactive graphics decoder 11, and a demo graphic decoder 13. Thus, the elementary streams introduced into each decoder after passing through the 3d PID filter are decoded and reproduced based on the PCR of the secondary TS.

As described in the explanation of the recording medium above, the start time (In_Time) and the end time (Out_Time) of the playitem correspond to the start time (In_Time) and the end time (Out_Time) of the sub playitem. Therefore, if the ATC counters 2a and 2c contain the same value (time) and count the time at the same speed, the time axes of the primary TS and secondary TS become aligned with each other. As a result, the primary TS and secondary TS constituting the overcompression application can be processed as a single stream. In addition, the ATC time axes showing time instants of data input to the decoder can be synchronized, and STC time axes showing the time base of the decoder can also be synchronized.

According to the ATC time axis synchronization, the aforementioned two source depacketizers can process source packets read from the BD-ROM and source packets read from the local storage, respectively.

Counters 3a and 3c of the STCs contain the same time and count the time at the same speed according to the synchronization of the time axes of the STC, and therefore two TSs can be processed as a single TS. Since the decoder of the reproducing apparatus operates on a single STC time axis, the STC time management can be standardized in the same way as when normal reproduction of only the primary TS is performed. Having the ability to make everything, video decoder 4, decoder 11 IG, decoder 13 PG, system decoders 15c and 16c and audio decoder 9, work on the same time axis STC, is desirable from the point of view of improving playback devices, since the control is exactly the same as used in conventional playback devices that only play back BD-ROMs. Moreover, when the authoring is performed, the state of the buffer can be observed by controlling the timing of the input of one TS, thereby facilitating verification at the authoring stage.

Video decoder 4 decodes the numerous PES packets output from the PID filter 3b, receives uncompressed motion pictures, and writes the motion pictures to video card 5. Video decoder 4 consists of transport buffer 4a, multiplexed buffer 4b, elementary buffer 4c, decoder 4d, reorder buffer 4e, and switch 4f.

The transport buffer (TB) 4a is a buffer in which TS packets belonging to the video stream are temporarily accumulated after being output from the PID filter 3b.

Multiplexed buffer (MB) 4b is a buffer in which PES packets are temporarily accumulated when a video stream is output from transport buffer 4a to elementary buffer 4c.

The elementary buffer (EB) 4c is a buffer in which the motion pictures are stored in the encoded state (I-motion pictures, B-motion pictures, P-motion pictures).

A decoder (DEC.) 4d receives multiple frame images by decoding individual frame images of a video elementary stream for each encoding time (DTS) and writes frame images to the video card 5.

The reordering buffer 4e is a buffer used to change the order of decoded movie frames so that they are arranged in a display order.

Switch 4f implements a change in the order of decoded movie frames so that they are arranged in display order.

Video card 5 is a card for storing uncompressed motion pictures in it. The board is the memory space of the playback device for these pixels with a capacity of a single screen. The resolution of video card 5 is 1920 × 1080, and the motion picture data stored in video card 5 consists of pixel data represented by 16-bit YUV (format “brightness - color blue - color red”).

The audio decoder 9 consists of a transport buffer 6, an elementary buffer 7, and a decoder 8 and decodes the audio stream.

The transport buffer 6 stores in it TS packets outputted from the PID filter 3b in a “first in, first out” manner and sends the TS packets to the audio decoder 8.

The elementary buffer 7 stores in it, from among the TS packets output from the PID filter 3b, only those having the PID of the audio stream, which must be reproduced in a “first in, first out” manner, and sends them to the audio decoder 8.

Decoder 8 converts the TS packets stored in transport buffer 6 into PES packets, decodes the PES packets to obtain uncompressed audio data in the LPCM state, and outputs the received audio data.

The switch 10a selectively provides TS packets read from the BD-ROM or TS packets read from the local storage 200 to the video decoder 4.

Switch 10b selectively provides TS packets read from the BD-ROM, or TS packets read from local storage 200 to interactive graphics decoder 11.

The switch 10c selectively provides TS packets read from the BD-ROM or TS packets read from the local storage 200 to the demo graphic decoder 13.

The interactive graphics (IG) decoder 11 decodes the IG stream read from the BD-ROM 100 or the local storage device 200, and writes uncompressed graphics to the IG board 12. The IG decoder 11 consists of a transport buffer (TB) 11a, a coded data buffer (CDB) 11b, a streaming graphics processor (SGP) 11c, an object buffer 11d, a layout buffer 11e, and a graphics controller (Ctrl) 11f.

Transport buffer (TB) 11a is a buffer in which TS packets belonging to the IG stream are temporarily accumulated.

The coded data buffer (CDB) 11b is a buffer in which the PES packets constituting the IG stream.

Stream graphic processor (SGP) 11c decodes the PES packets storing the graphic data therein and writes an uncompressed bitmap consisting of index colors obtained by decoding processing to the object buffer 11d as a graphic.

In the object buffer 11d, a graphic object is received obtained by decoding processing performed by the stream graphics processor 11c.

The layout buffer 11e is a memory in which control information for drawing graphic data is stored.

The graphics controller (Ctrl) 11f decodes the control information located in the layout buffer 11e, and performs control based on the result of decoding processing.

On the interactive graphics (IG) board 12, uncompressed graphics obtained by the decoding processing of the IG decoder 11 are recorded.

The demo graphics decoder (PG) 13 decodes the PG stream read from the BD-ROM or local storage 200 and writes uncompressed graphics to the demo graphics board 14. The PG decoder 13 consists of a transport buffer (TB) 13a, a coded data buffer (CDB) 13b, a streaming graphics processor (SGP) 13c, an object buffer (OB) 13d, a layout buffer 13e (CB), and a graphics controller 13f (Ctrl).

Transport buffer (TB) 13a is a buffer in which TS packets belonging to the PG stream are temporarily accumulated after being entered from the PID filter 4.

A coded data buffer (CDB) 13b is a buffer in which PES packets constituting a PG stream.

Stream graphics processor (SGP) 13c decodes PES packets (ODS) storing graphics data therein and writes an uncompressed bitmap consisting of index colors obtained by decoding processing to an object buffer 13d as a graphics object.

In the object buffer 13d, a graphic object is obtained obtained by decoding processing performed by the stream graphics processor 13c.

The layout buffer 13e (CB) is a memory that stores control information (PCS) for drawing graphic data.

The graphics controller 13f (Ctrl) decodes the PCS located in the layout buffer 13e and performs control based on the result of the decoding processing.

The demo graphics (PG) board 14 is a memory containing a single-screen capacitance region and is capable of storing uncompressed graphics with a single-screen capacitance therein.

The system decoder 15 processes the system control packets (PAT and PMT) of the secondary TS and manages the decoders taken as a whole.

The transport buffer 15a stores in it system control packets (PAT and PMT) present in the primary TS.

The elementary buffer 15b sends system control packets to the decoder 15c.

The decoder 15c decodes the system control packets stored in the elementary buffer 15b.

The transport buffer 16a stores in it system control packets present in the secondary TS.

The elementary buffer 16b sends the secondary TS system control packets to the decoder 16c.

The decoder 16c decodes the system control packets stored in the elementary buffer 16b.

The memory 21 is a memory for storing current playlist information and current clip information therein. The current playlist information is the playlist information that is currently being processed from among the plurality of pieces of playlist information stored on the BD-ROM. The current clip information is clip information that is currently being processed from among the plurality of pieces of clip information stored on the BD-ROM / local storage device.

The controller 22 achieves the playback control of the BD-ROM by performing playback of the playlist (i.e., playback control in accordance with the current information of the playlist). The controller 22 also performs the above control on the ATS and STC. With this control, the controller 22 performs a preliminary read operation to read in advance for a period of 1 second the source packets from the BD-ROM or local storage device into a buffer or decoder. By performing this pre-reading operation, the prevention of negative overflow and overflow can be guaranteed due to the aforementioned window control.

Set 23 PSR is a register built into the playback device, and consists of 64 pieces of player settings / status registers (PSR) and 4096 pieces of general purpose registers (GPR). Among the values (PSR) set in the player's settings / status registers, PSR4 through PSR8 are used to represent the current playback point.

The PID conversion unit 24 converts the audio streams and audio stream stream numbers stored in the PSR set 23 to the PID reference values based on the STN table (STN_table), and reports the PID reference values from the conversion results to the PID filters 3b and 3d.

The network module 25 performs the communication function of the playback device. When a URL is specified, the communication module 25 establishes a TCP connection or an FTP connection to the website at the specified URL. Establishing such a connection involves downloading from web sites.

The operation receiving unit 26 receives a description of the operation performed by the user on the remote control, and reports information about the user's operation, which indicates the operation specified by the user, to the controller 22.

This is a description of the internal structure of the playback device. The following describes the implementation of the controller 22 in the playback device. The controller 22 can be implemented in the playback device by creating a program that causes the CPU (central processing unit, CPU) to perform the sequence of operations according to the flowcharts of the sequence of operations of the methods shown in Figs. 31 and 32, recording the program in ROM (read-only memory, ROM ) and sending it to the CPU.

Fig. 31 is a flowchart showing a reproducing procedure based on the information of the playlist. The flowchart shows a cyclic structure in which the.mpls file structuring the playlist information is read (step S11), the playitem at the beginning of the playlist information is set as the current playitem (step S12), and steps S13 to S25 is repeated for the current playitem. This cyclic structure contains step S23 as a termination condition. The BD-ROM drive is instructed to read the access units, starting from the corresponding start time point (In_Time) and ending with the corresponding end time point (Out_Time) of the current playitem (step S13). An assessment is made whether the previous playitem is present in the current playitem (step S14). Step S15 or steps S16 to S21 are performed selectively according to the evaluation results. To be more precise, if the current playitem does not have a previous playitem (step S14: No), the decoder is instructed to perform playback from the start time of the playitem (PlayItem_In_Time) to the end time of the playitem (PlayItem_Out_Time) (Step S15).

If the current playitem has a previous playitem (step S14: Yes), an assessment is made whether the current playitem is in the state CC = 5 (step S16). When CC = 5 (step S16: Yes), the processing of steps S17 to S20 is performed.

When the previous playitem above is present, the ATC sequence (ATC_Sequence) in the main clip is switched. To switch the ATC sequence (ATC_Sequence), the offset value for the primary TS, called ATC delta 1 (ATC_delta1), is calculated (step S17). The ATC value (ATC2) for the new ATC sequence (ATC_Sequence) is obtained by adding the ATC delta 1 (ATC_delta1) to the ATC value (ATC1) of the original ATC sequence (ATC_Sequence) (step S18).

In addition, when the previous playitem above is present, the STC sequence (STC_Sequence) in the primary TS is switched. To switch the STC sequence (STC_Sequence), an offset value, called STC delta 1 (STC_delta1), is calculated (step S19). The STC value (STC2) for the new STC sequence (STC_Sequence) is obtained by adding delta 1 STC (STC_delta1) to the STC value (STC1) of the initial STC sequence (STC_Sequence) (step S20).

After the audio decoder 9 is instructed to suppress the overlap of the audio, the decoder is instructed to perform playback from the start time of the playitem (PlayItem_In_Time) to the end time of the playitem (PlayItem_Out_Time) (step S21). When the current playitem is not in the state CC = 5, processing CC = 1 and CC = 6 is performed.

After any one of the processing in step S15 and the processing in steps S16 to S21 are performed, the processing in step S25 is performed. Step S25 is a flowchart of checking whether the sub-play item should be played synchronously with the current play item. Here, the sub-play element constituting the sub-path information contains information called Id of synchronization with the play element (Sync_PlayItem_Id), and Sync_PlayItem_Id of the sub-play element to be played synchronously with the current play element is set to this current play element. Therefore, in step S25, a check is made to see if, in the multiple sub-play elements constituting the sub-path information, the sub-play element whose Sync_PlayItem_Id has been set to the current playback element.

If no such sub-play element is present, the flow proceeds to step S22. At step 22, an assessment is made whether the current playback time (current PTM (presentation time)) on the time axis of the AV clip reaches the Out_Time of the current playitem (step 22). If it has reached, the flow proceeds to step S23. At step S23, an assessment is made whether the current playitem is the last playitem of the playlist information. If it is not the last playitem, the next playitem in the playlist information is set as the current playitem (Step S24), and the flow proceeds to step S13. In this way, the processing in steps S13-S24 is performed on all the playitems in the playlist information.

32 is a flowchart showing a process for processing a seamless connection of sub-play elements.

When it is determined in step S25 that there is a sub play item whose Sync_PlayItem_Id is set to the current play item, the sub play item is set as the current sub play item (step S31). The local storage device 200 is instructed to output access units, starting from the corresponding start time point (In_Time) of the sub play element and ending with the corresponding end time point (Out_Time) (step S32). An assessment is then made whether the previous sub-playitem is present in the current playitem (step S33), and one of step S34, step S35, and steps S36-S41 is selectively performed according to the evaluation result. To be more precise, if the previous sub-playitem is not present in the current playitem (step S33: No), it is expected until the current PTM reaches the initial PTS of the playitem synchronization (Sync_Start_Pts_of_PlayItem) (step S34). When the current PTM has reached it, the decoder is instructed to play from the start time of the sub play item (SubPlayItem_In_Time) to the end time of the sub play item (SubPlayItem_Out_Time) (step S35).

When the previous sub-playitem is present in the current playitem (step S33: Yes), it is judged whether the current playitem is in the state SP_CC = 5 (step S36). When it is in the state SP_CC = 5 (step S36: Yes), the processing of steps S37-S41 is performed.

When the current playitem has the previous subtitle, the ATC sequence (ATC_Sequence) is switched. To switch the ATC sequence (ATC_Sequence), the offset value for the secondary TS called delta 2 ATC (ATC_delta2) is calculated (step S37), and the ATC value (ATC2) for the new ATC sequence (ATC_Sequence) is obtained by adding the ATC delta 1 (ATC_delta1) to the value ATC (ATC1) of the original ATC sequence (ATC_Sequence) (step S38).

Delta ATC (ATC_delta) means an offset value representing the offset from the time T1 of the input of the last packet TS of the transport stream (TS1), which was read initially, until the time T2 of the input of the last packet TS of the transport stream (TS2), which was read again. The ATC delta (ATC_delta) satisfies "ATC_delta ≥ N1 / TS_recording_rate" ("ATC delta ≥ N1 / (TS write speed)", where N1 is the total sum of TS packets following the last PES packet of video from TS1.

In addition, when the previous playitem above is present, the STC sequence (STC_Sequence) is switched. To switch the STC sequence (STC_Sequence), the STC delta 2 (STC_delta2) is calculated (step S39), and the STC (STC2) value of the new STC sequence (STC_Sequence) is obtained by adding the STC delta 2 (STC_delta2) to the STC value of the initial STC sequence (STC_Sequence) (step S40).

Assume that the start time for displaying the motion picture displayed finally in the previous STC (STC_Sequence) sequence is PTS1 (1st termination), the time period for displaying the motion picture is TPP, and the start time of the motion picture displayed at the beginning in the next STC (STC_Sequence) sequence is PTS2 (2nd beginning). Here for CC = 5, since it is necessary to coordinate the time PTS1 (1st end) + TPP with the time PTS2 (2nd beginning), delta 2 STC (STC_delta2) can be calculated using the following equation:

STC_delta2 = PTS1 (1st ending) + TPP-PTS2 (2nd ending).

After the audio decoder 9 is instructed to suppress audio overlap, the decoder is instructed to perform reproduction from the start time of the playitem (PlayItem_In_Time) to the end time of the playitem (PlayItem_Out_Time) (step S41).

The controller 22 performs the STC switching sequence of operations, as described above, and this sequence of operations is performed in the playback device with the conventional implementation, when the decoder is in a state of operation without external synchronization. A state without external synchronization means a state where the decoder is not performing synchronous control. Subsequently, when the STC returns to a condition where the time axis of the STC can be set, the decoder makes a transition from the state without external synchronization to synchronous control with the STC. On the other hand, when the current playitem is judged not to be in the state CC = 5 in step S36 (step S36: No), the processing of CC = 1 and CC = 6 is performed.

Thus, according to the present embodiment, the transmitted volume, called the window, is limited to 48 Mbps or less. Therefore, if TS packets with a size of 96 Mbit × 0.5 seconds are read into the decoder in advance, the decoder buffer will not cause negative overflow or overflow even when the transmitted volume locally reaches 96 Mbit within a period of 1 second. Since the data volume is “96 Mbps × 0.5 second” or less in any time period in the digital stream, and TS packets can be delivered without negative overflow or overflow, the loss of video and audio can be prevented. This eliminates the risk that simultaneous reads to implement overcompression infrastructure will affect the quality of the digital stream.

In addition, if the start time (In_Time) of the playitem and the end time (Out_Time) of the auxiliary playitem coincide with each other, and the connection state of the playitems is CC = 5, the connection state of the auxiliary playitem becomes SP_CC = 5. Therefore, when the playitem switches, switching from the playitem to another playitem and switch from the auxiliary playitem to another auxiliary playitem can be performed simultaneously, without resetting the demultiplexing units. Thus, although the time axes of the STCs on which the demultiplexing units are based are designed to synchronize each other, the sequence of playback operations based on the information of the playlist can continue.

Option 2 implementation

In the present embodiment, the manufacture of the BD-ROM according to the previous embodiment is described in detail. The BD-ROM of the previous embodiment may be made by performing one after another of the following sequences of operations.

<BD-ROM Writing Procedure>

First, a general structure is planned with which the BD-ROM (planning sequence of operations) is reproduced, materials such as motion picture recordings and audio recordings (sequence of operations for obtaining materials) are created, and volume configuration information is created based on the general structure created in the planning sequence ( script statement).

Volume configuration information is information indicating an application layer format on an optical disc using an abstract description.

Subsequently, each of the video materials, audio materials, subtitle materials, and menu materials are encoded to thereby create elementary streams (a sequence of operations for encoding materials). Then, multiple elementary streams are multiplexed (multiplexing process flow).

Then, an operation is performed to adjust the multiplexed streams and volume configuration information to the BD-ROM application layer format, and complete data (usually called “volume data”) is obtained, which must be written to the BD-ROM volume region (formatting flow).

The instances of the class structures described in the programming language are the application layer format of the recording medium according to the present invention. Clip information, playlist information, and the like can be created by describing instances of the class structure based on the syntax specified in the BD-ROM standard. In this case, the data in a table format can be determined using the "for" statements of the programming language, and the data required under certain conditions can be determined using the "if" statements.

When the volume data is obtained after such a sequence of fitting operations, the volume data is reproduced to understand whether the result of the scenario setting sequence (emulation sequence) is correct. In the emulation process, it is desirable to emulate the buffer state of the BD-ROM player model.

In conclusion, a stamping sequence is performed. With this punching sequence, the images of the volume are converted into physical data sequences, and the cutting of the first original disk is done by using physical data sequences to create the first original disk. Then the BD-ROMs are made from the first original created by the stamping device. The manufacture consists of various sequences of operations, including mainly forming the substrate, coating with a reflective film, coating with a protective film, laminating and printing labels.

By completing these workflows, a recording medium (BD-ROM) described in the embodiment above can be created.

<Sequence of operations for creating additional content>

When the motion picture consists of the contents of the BD-ROM and the additional contents, the aforementioned from the flow of scheduling to the flow of formatting are performed. Then, AV clips, clip information, and playlist information constituting the volume data are obtained. Those that will be delivered via BD-ROM are deleted from the received AV clips, clip information and playlist information, and the remaining information is collected into a single file as additional content by means of an archiver program or the like. When such additional content is obtained after these sequences of operations, the additional content is delivered to a web server and sent to playback devices upon request.

The verification described in the above embodiment is performed when the AV clips, clip information and playlist information are completed and the elementary streams to be played are determined by the STN table (STN_table) in the playlist information — that is, in the formatting flowchart. The following clarifies the authoring system that creates such an application format.

<Authoring System>

Fig. 33 shows the internal structure of the authoring system of Embodiment 2. As shown in the figure, the authoring system consists of: input device 51; encoding device 52; server device 53; a storage device 54 materials; a storage device 55 of ED configuration information; client devices 55-58; multiplexer 60; an ED script converter 61; formatter 62 and verification module 63.

A video cassette is installed in the input device 51, on which HD images and SD images are recorded, and then the input device 51 reproduces the video cassette and outputs playback signals to the encoding device 52.

The encoding device 52 encodes the playback signals output from the input device 51 to thereby obtain elementary streams such as video streams and audio streams. Elementary streams obtained in this way are output to the server device 53 via the LAN and recorded in the storage device 54 of the materials in the server device 53.

The server device 53 consists of two drive devices, a material storage device 54, and a BD configuration information storage device 55.

The material storage device 54 is an integrated disk device of the server device 53 and sequentially stores the elementary streams obtained through encoding operations by the encoding device 52. The memory device 54 contains two directories, a directory of HD streams and a directory of SD streams. Elementary streams obtained by encoding HD images are recorded in the HD stream catalog.

The ED configuration information storage device 55 is a drive device in which configuration information of a volume BD is stored.

A multiplexer 60 reads from among the elementary streams stored in the HD stream directory and the SD stream directory in the material storage 54 those specified in the configuration information of the BD volume, and then multiplexes the read elementary streams according to the configuration information of the HD volume to thereby obtain a multiplexed stream , i.e. an AV clip.

The BD script converter 61 obtains the BD script by converting the BD volume configuration information stored in the BD configuration information memory 55 to the application BD-ROM format.

The formatter 62 adapts the clip received by the multiplexer 60 and the BD script received by the BD script converter 61 to the application layer format on the BD-ROM. By this, the first original BD-ROM and download contents to be stored on the local storage device can be obtained from the adapted BD script.

Verification unit 63 evaluates, by referring to the STN table (STN_table) in the playlist information generated by the script converter 61, whether the primary TSs for the BD-ROM and the secondary local memory TSs obtained by the multiplexer 60 satisfy the constraints for implementing the supercompression application.

This is the internal structure of the authoring system. The following explains the implementation of the authoring system verification module 63.

<Processing Procedure for Implementing Verification Module 63>

Verification module 63 can be implemented in the authoring system by creating a program that prompts the CPU to perform processing according to the flowchart of the method shown in FIGS. 34 and 35, writing the program to ROM and sending it to the CPU.

Fig. 34 is a flowchart showing a verification procedure for primary TS and secondary TS. The flowchart shows that the ATS of the first source packet in the source packet sequence is set as the start time (In_Time) of the current window in step S1, and the process steps in steps S2 to S7 are repeated. The cyclic structure repeats the subsequent steps S2 to S5 until the assessment in step S6 becomes Yes: the ATS that appears 1 second after the start time (In_Time) of the current window is set as the Out_Time of the current window ( step S2); the TS packets present between the start time (In_Time) and the end time (Out_Time) of the current window are counted (step S3); the total amount of bits of the current window is calculated from the start time (In_Time) (step S4); and an assessment is made whether the bit value is 48 Mbps or less (step 55). Step S6 is an evaluation of whether the Out_Time has reached the current window of the last source packet on the ATC time axis. If step S6 is No, the next ATS in the source packet sequence is set at the start time (In_Time) of the current window (step S7), and steps S2-S6 are repeated. If, with any window, step S5 becomes No, it is determined that there is a violation of the standardization of the BD-ROM (step 9). When step S5 accepts Yes for all windows, and then step S6 accepts Yes, it is determined that the primary TS and secondary TS comply with the BD-ROM standard (step S8).

Since the primary TS and secondary TS were subjected to a verification process, the above limitations are always satisfied even when the primary TS and secondary TS are supplied from the BD-ROM and local storage, respectively.

With respect to video streams, audio streams, PG streams and IG streams, if there are multiple elementary streams of the same type, it is desirable to carry out a check according to the procedure shown in FIG. 35. In the verification procedure of FIG. 35, steps S3 and S4 of FIG. 34 are replaced by steps S81-S83.

Steps S81-S83 are that: regarding the TS belonging to the current window, from the number of TS packets constituting the elementary streams, which, in the STN table (STN_table), are allowed to play, the bit rate is calculated for each elementary stream every time the current window is determined (step S81); for each type of stream — that is, multiple video streams, multiple audio streams, multiple PG streams, and multiple IG streams — those having the highest calculated bit rate are selected (step S82); the highest bit rate of the video stream, the highest bit rate of the audio stream, the highest bit rate of the PG stream and the highest bit rate of the IG stream are summed (step S83); and an assessment is made whether the total amount is 48 Mbps or less (step S5).

In a super-densification application, an elementary flow is always selected solely and exclusively from among the same type of elementary flows, and therefore it is more acceptable that the verification be carried out in the above procedure.

Regarding verification, it is rational to check locations with locally high bit rates, that is, bit values of positions at which local peaks occur. The positions where local peaks occur are as follows.

(1) the start of the TS packet indicated by the start time (In_Time) of the window;

(2) the end of the TS packet indicated by the start time (In_Time) of the window;

(3) the start of the TS packet indicated by the window's end time (Out_Time), and

(4) the end of the TS packet indicated by the end time (Out_Time) of the window. The sequence of verification operations during authoring can be further simplified by a special focus on the number of bits of these positions.

Thus, according to the present embodiment, when an STN table (STN_table) is created that allows secondary TSs to be reproduced, it can be checked beforehand when authoring is being performed whether a negative overflow or overflow can be caused during playback based on the STN table (STN_table) .

Option 3 implementation

In the present invention, a new type of CC = 6 is provided for a connection between playitems and between sub playitems.

CC = 6 defines the state of the connection between the plural pieces of playitem information constituting the progressive playlist information. The progressive playlist information is the playlist information used to specify, as one playback path, multiple AV clips for streaming playback.

<Progressive playlist information>

Progressive playlist information has the advantage of becoming smaller in cache size or having the ability to start playback without waiting for all the files to be downloaded by dividing the secondary TS for download / streaming into partial files.

Since streaming content is defined by many short AV clips, progressive playlist information consists of many pieces of playlist information, each of which corresponds to a different one of the multiple AV clips. On the other hand, AV clips, divided into small blocks, were divided for streaming, and therefore, there are no gaps in the STC and ATC. Accordingly, such a connection state of the AV clips should be set as different from CC = 5. This type of connection state is defined as CC = 6.

<Conditions to be met when CC = 6>

When CC = 6, TS1 and TS2 defined by two playitems and TS1 and TS2 specified by two auxiliary playitems must satisfy the following conditions.

1) The video stream from TS2 must begin with a GOP (group of images or frames).

2) There is no gap, in the sequence of audio presentation blocks, at the connection point between the audio stream from TS2 and the audio stream from TS1 having the same PID as the audio stream from TS2.

The audio stream from TS1 may end up as an incomplete audio stream. Then, the audio stream from TS2 having the same PID as TS1 may begin with an incomplete audio presentation block. By reproducing these TS1 and TS2 based on the plurality of playitems and the plurality of auxiliary playitems, one complete audio presentation unit can be obtained from two audio presentation units.

In the case of CC = 6, the flow is actually continuous. Therefore, all elementary streams are connected seamlessly, in contrast to the case of CC = 5, where the video is connected only seamlessly, while the audio is connected in a discontinuous manner or set to suppress.

Thus, CC = 6 means the dividing boundary created when a logically continuous stream is divided into multiple parts according to the purpose of the streaming. Note that, since the stream to be recorded on the BD-ROM must consist of 32 source packets, the file of one stream forming one element of auxiliary playback must be values that are multiples of 6 kbytes.

<CC Details = 6>

Fig. 36 shows a detailed explanation of CC = 6. Level 1 shows a file (20000.m2ts) containing a single continuous ATC / STC time series, and the encoding method does not change. Level 2 shows three files (20001.m2ts, 20002.m2t and 20003.m2ts) that store three streams in them. These three files store three primary TSs in them, which were obtained by dividing a single level 1 stream into blocks of aligned blocks (6 kB).

Fig. 37 shows the relationship between playitems and sub playitems. Level 1 shows three playitems (information # 1 about a playitem, information # 2 about a playitem and information # 3 about a playitem) in the playlist information. These three playitems define the primary TS, and the connection between the playitem information # 1 and # 2 is set to CC = 1, while the connection between the playitem information # 2 and # 3 is set to CC = 5. Level 2 shows three sub-play items (sub-play item # 1, sub-play item # 2 and sub-play item # 3) in the playlist information. These three sub-play elements define the secondary TS, and the connection between the sub-play items # 1 and # 2 is set to CC = 1, while the connection between the sub-play items # 2 and # 3 is set to CC = 5. Level 3 shows nine sub-play items (sub-play item # 1, sub-play item # 2, from sub-play item # 3 to sub-play item # 9) in the progressive playlist information. These nine sub-play elements define the secondary TS. Here, the connection between the sub-play items # 3 and # 4 is set to CC = 1, the connection between the sub-play items # 6 and # 7 is set to CC = 5, and the remaining connections are set to CC = 6. Auxiliary play items in a progressive play list are usually connected at CC = 6; however, when the playitems are connected at CC = 1 and CC = 5, the auxiliary playitems are also connected to satisfy the conditions CC = 1 and CC = 5, respectively, like the playitem.

Thus, the present embodiment introduces a new CC = 6 connection state for playitems and sub playitems, thereby performing a process of dividing the AV clips constituting the progressive playlist information into small sections and delivering them via streaming.

Option 4 implementation

Embodiment 1 explains how to limit the number of bits for each window; the present embodiment provides how to perform multiplexing in order to satisfy such constraints.

<Multiplexing video and audio>

Fig. 38 schematically shows, in the case where the audio constituting the primary TS is replaced by the audio constituting the secondary TS, how the plural TS packets constituting the primary TS and the plural TS packets constituting the secondary TS are multiplexed together.

Fig. 38 schematically shows a method by which multiple TS packets represented on an ATC time axis are multiplexed together. Level 1 shows the primary TS. The primary TS consists of TS packets storing V, A1 and A2 in them (one set of video and two sets of audio). These TS packets are obtained by jointly multiplexing these three elementary streams of two types.

Level 2 shows the secondary TS. The secondary TS consists of TS packets storing in them two sets of audio A3 and A4. The time period p3 during which these secondary TS packets are multiplexed, on the time axis ATC indicating the time characteristics of input to the decoder, is composed of the time period p1 during which the audio packets of the primary TS are multiplexed and the time period p2 during which they are not transmitted TS packets forming the primary TS.

By multiplexing the streams in this way, it can be ensured that the total bit rate of the elementary streams to be decoded does not exceed the allowable maximum bit rate of the primary TS (48 Mbit / s), no matter which elementary stream is selected for each type of elementary streams. The example shown in FIG. 38 is the simplest case in which the secondary TS includes only audio.

<Multiplexing video, audio, PG and IG streams>

Fig. 39 schematically shows, in the case where the subtitles (PG stream) and menu (IG stream) are also replaced in addition to audio, the manner in which multiple TS packets constituting the primary TS and multiple TS packets constituting the secondary TS are multiplexed together.

In the figure, the period k3 of the time during which the packets of the secondary TS are forwarded is the total amount:

1) the period k1 of the time during which packets are sent whose type is the same as in the primary TS, and

2) the period of time during which the primary TS is not transmitted.

The above rules 1) and 2) apply in the same way to other types of streams (video, IG, PG and the like) stored in the secondary TS. Therefore, it is advisable if, for each stream, it is first evaluated whether the stream can be multiplexed to the secondary TS during the time period when the packet is forwarded, whose type is the same as in the primary TS, and when the rating is negative, multiplexing in the time period is performed, during which no primary TS is forwarded.

<The sequence of operations of the multiplexer 60>

The following more accurately describes the sequence of operations of the multiplexer 60 of the present embodiment.

In order to implement the multiplexing described above, the multiplexer 60 imitates, according to the decoder model, the buffer state fixed when the primary TS is played back, and finds a time period for sending each packet of the primary TS and a time period for no transmission of the primary TS. After finding these time periods, the multiplexer 60 converts each PES packet constituting the secondary TS into TS packets, so that each of the PES packets is sent during the time period when the packet is sent, whose type is the same as in the primary TS, or during the time period when the primary TS is not forwarded, and attaches the ATS to each TS packet. Because the ATS attached by this method indicates the period of time when a packet is sent whose type is the same as that in the primary TS, or the period of time when the primary TS is not forwarded, each PES packet constituting the secondary TS is sent to the decoder during the period the time when a packet is being forwarded whose type is the same as in the primary TS, or the time period when the primary TS is not transmitted, as shown in FIG.

<Delivery according to DVD>

In the case where the elementary streams supplied from the local storage device are not created in the transport stream format, but in the program stream format, the multiplexer 60 converts the PES packets constituting the elementary streams into packets, and the SCR (system time stamp) is attached to the TS header of each packet . An SCR attached in this way also indicates, like ATS, the period of time when a packet is forwarded whose type is the same as in the primary TS, or the period of time when the primary TS is not forwarded. Therefore, each PES packet constituting the secondary PS (program stream supplied from the local storage device) is sent to the decoder for a period of time when the packet whose type is the same as that of the primary PS (program stream supplied with the BD-ROM) , or a period of time when the primary TS is not forwarded, as shown in FIG. In the case when elementary streams delivered from a local storage device are created in the program stream format, the time period when a packet is forwarded whose type is the same as in the primary TS, or the time period when the primary TS is not forwarded are expressed in large units time, pack (PES package). Therefore, the overhead when the authoring is performed is significantly less, which helps the application of over-compaction. This is an advantage when the over-compaction application is implemented in the DVD player.

Thus, the present embodiment performs multiplexing by selecting, as input periods for the packets constituting the secondary TS, the time period when the packet is forwarded, whose type is the same as in the primary TS, or the time period when the primary TS is not forwarded. This helps to limit the number of bits shown satisfactory in Embodiment 1. The implementation of such multiplexing in the authoring system according to option 2 implementation facilitates the formulation of the movie with the application of super-compaction. By this, the guarantee of no overflow during playback can be easily implemented when the authoring is performed.

Option 5 implementation

In the present embodiment, an audio mixing application is explained in detail. This appendix includes an exception to the supercompression rule for selecting only one elementary flow for each type. That is, the audio mixing application simultaneously selects the audio stream for the primary TS and the audio stream for the secondary TS, and simultaneously decodes two audio, audio of the primary TS and audio of the secondary TS.

40 shows a method by which a primary TS and a secondary TS constituting an audio mixing application are supplied to a decoder in a BD-ROM reproducing apparatus. In the figure, among the internal structural components of the BD-ROM playback device, the BD-ROM drive 1a, the local storage device 200, and the network module 25 are shown on the left side, while the respective decoders are shown on the right side. A PID filter that performs stream demultiplexing is shown in the center. Primary TS (video 1, audio 1 (English), audio 2 (Spanish), PG 1 (English subtitles), IG 1 (English menu)) and secondary TS (audio 3 (narrator), PG 2 (subtitles in Japanese), PG 3 (subtitles in Korean), IG 4 (subtitles in Chinese), IG 2 (menu in Japanese) in the figure are the transport streams supplied with the BD-ROM and local storage respectively. Since only English (audio 1) and Spanish (audio 2) are recorded on the disc, the announcer's text of the movie director cannot be selected on the disc. However, by downloading to the local storage device a secondary TS, which includes audio 3 provided by the content provider, English audio (audio 1), audio 3 (narration text) can be sent to the decoder. The decoder then mixes the audio in English (audio 1) and audio 3 (narrator text) and outputs a result that allows the user to play together with the video (video 1) audio in English, to which the narrator text is attached.

Here, the only difference from the overcompression application is the decoding of two audio streams simultaneously. With the primary TS, a case may arise where, for example, it is desirable to add the audio of the announcer's text of the director after the release of the disc. Accordingly, a limitation on the bit rate of the primary TS is not preferable, and therefore, a restriction on the secondary TS stream is introduced, as in the case of supercompression. Since audio mixing needs to decode audio in addition to each elementary stream (video, audio, subtitle and menu), two nomenclatures of audio decoder are needed.

<Structures of primary and secondary audio streams>

When implementing an audio mixing application, an audio stream that will belong to the primary TS is referred to as a primary audio stream, while an audio stream that will belong to a secondary TS is referred to as a secondary audio stream. The following describes such primary and secondary audio streams.

There are 32 primary audio streams, each of which has a different PID, from 0x1100 to 0x111F. On the other hand, like primary streams, there are 32 secondary audio streams, each of which has a different PID from a number from 0x1A00 to 0x1A1F.

The difference between the secondary audio streams and the primary audio streams is that the audio frames of the secondary audio streams include metadata composed of “downmix information” and “gain control information”.

“Downmix Information” is information for downmixing. Downmix is a conversion that turns the number of audio playback channels into less than the number of encoded channels. The downmix information defines a matrix of transform coefficients for the downmix and thereby causes the playback device to downmix. Playing a 5.1-channel audio stream after converting it to a 2-channel audio stream is one example of down-mix.

“Gain control information” is information for increasing or decreasing the gain of an audio output of a primary audio stream; however, here gain control information should only reduce gain. Thus, the metadata of the secondary audio stream is able to reduce in real time the output power of the primary audio stream, which is played simultaneously with the secondary audio stream. In the case of superimposing secondary audio on primary audio, since the pair of primary audio and secondary audio to be mixed is known in advance, there is a need to adjust the gain of the two audio in real time. In this case, mixing (overlay) can only be done well by reducing the gain of the primary audio while keeping the gain of the secondary audio unchanged. By providing such metadata, it is possible to avoid an increase in the output sound level of the reproduction of the primary audio stream and the output sound level of the reproduction of the secondary audio stream and, thus, damage to the speakers. This is the audio streams of the present embodiment. The following describes the enhancements to the playlist information of the present embodiment.

<STN table (STN_table) for implementing an audio mixing application>

Elementary streams of the same type must be decoded by the decoder at the same time, and therefore, regarding the playlist information of the present embodiment, the multiple primary audio streams and the multiple secondary audio streams allowed for playback are shown in the STN (STN_table) table of each playitem.

The following describes the STN table (STN_table) of the present embodiment. To implement the audio mixing application, the stream input pairs (Stream_entry) and stream attribute (Stream_attribute) in the secondary audio streams are present in the STN table (STN_table) in addition to the stream input pair (Stream_entry) and stream attribute (Stream_attribute) in the primary audio streams. Each pair of stream input (Stream_entry) and stream attribute (Stream_attribute) in the secondary audio streams is associated with information of combining secondary audio and primary audio (Comb_info_Secondary_audio_Primary_audio).

(Information combining secondary audio and primary audio (Comb_info_Secondary_audio_Primary_audio))

Information combining the secondary audio and primary audio (Comb_info_Secondary_audio_Primary_audio) uniquely defines one or more primary audio streams with which the playback of the secondary audio streams can be mixed. This involves when authoring is performed, creating a mix-up setting according to the audio attribute, so that, for example, the secondary audio stream is not mixed when the primary audio stream having a predetermined attribute is to be played, while the secondary audio stream is mixed when the primary audio stream having the attribute is to be played other than a predefined attribute.

(SP connection information (sp_connection_condition Information))

In the playlist information, the same value as the connection condition information (connection_condition) of the playitem information is set for the connection condition information SP (sp_connection_condition) of the auxiliary playitem. Therefore, when the connection item information (connection_condition) of the playitem information is “= 5”, the connection condition information SP (sp_connection_condition) of the auxiliary playitem information is also set to “SP_CC = 5”. In addition, the information about the start time (In_Time) and the end time (Out_Time) of the sub play item shows the same time points as the information about the start time (In_Time) and the end time (Out_Time) of the play item.

This is an improvement in the recording medium of the present embodiment. The following describes the internal structure of the reproducing apparatus of the present embodiment.

<Internal structure of the playback device>

Fig. 41 shows an internal structure of a reproducing apparatus according to Embodiment 5. TB 6, EB 7 and audio decoder 8 are replaced by an audio mixing processor (covered by dashed lines), as shown in the figure. The audio mixing processor introduces two audio streams from the primary TS and secondary TS, decodes and mixes them at the same time. The remainder of the internal structure is the same as that for the implementation of the superconsolidation application. The following describes the mixing processor. The mixing processor consists of: transport buffers 6a and 6b; EB 7a and 7b; preload buffer 7c; audio decoders 8a and 8b and mixers 9a and 9b.

Transport buffer 6a stores TS packets having PIDs of audio streams and PIDs derived from PID filter 3b in a first-in-first out manner, and sends TS packets to audio decoder 8a.

The transport buffer 6b stores TS packets having PIDs of audio streams and output from the 3d PID filter in a “first in, first out” manner, and sends TS packets to the audio decoder 8b.

EB 7a is a buffer that stores PES packets received by converting TS packets stored in buffer 6a.

EB 7b is a buffer that stores PES packets received by converting TS packets stored in buffer 6a.

The preload buffer 7c is a memory for preloading the sound.bdmv file read from the BD-ROM / local storage device. The sound.bdmv file is a file that stores audio data in it, which should be displayed in response to an operation performed on a menu.

The audio decoder 8a decodes the PES packets constituting the primary TS to thereby obtain uncompressed audio data in the LPCM state, and outputs the received audio data. It successfully performs digital audio stream output.

The audio decoder 8b decodes the PES packets constituting the secondary TS to thereby obtain uncompressed audio data in the LPCM state, and outputs the received audio data. It successfully performs digital audio stream output.

The mixer 9a mixes the digital audio in the LPCM state outputted from the audio decoder 8a and the digital audio in the LPCM state outputted from the audio decoder 8b.

The mixer 9b mixes the digital audio in the LPCM state output from the mixer 9a and the acoustic data stored in the buffer 7c. This mixing operation by the sound mixer 9b is carried out in that the controller 22 decodes the navigation command with the intention of emitting a click sound of a button.

This is a description of the reproducing apparatus of the present embodiment.

<Testing by audio mixing application>

Since the audio mixing application consists of primary audio streams and secondary audio streams, as described above, the verification, which is shown in Embodiment 2, is carried out under the assumption that the primary audio stream and the secondary audio stream have been read at the same time. More specifically, the window shifts one packet at a time on the ATC timeline referenced by the main clip and sub clip. The shift procedure is the same as that shown in the flowchart of the method of FIG. 35. At each coordinate of the ATC time axis indicated by the ATS, the stream having the highest calculated bit rate is selected for each type of video stream, multiple primary audio streams, multiple secondary audio streams, multiple PG streams and multiple IG streams. The highest bit rate of the video stream, the highest bit rate of the primary audio stream, the highest bit rate of the secondary audio stream, the highest bit rate of the PG stream and the highest bit rate of the IG stream are added up and it is judged whether the total is 48 Mbps or less. If the total amount exceeds 48 Mbit, it is determined that there is a violation of the standardization of BD-ROM.

Thus, according to the present embodiment, it is guaranteed that the number of bits per second does not exceed a predetermined upper limit, even when the primary and secondary audio streams are simultaneously read from both the BD-ROM and the local storage device and supplied to decoders for the primary and secondary audio streams. With such a guarantee, an audio mixing application can be created rationally. This enables a delivery system that loads additional content into the local storage device that implements the audio mixing application and delivers it to the decoder from the local storage device. Therefore, the delivery layout can be easily implemented, for example, to add narration after shipment of the BD-ROM.

Option 6 implementation

In Embodiment 1, the connection points between the playitems and between the auxiliary playitems are matched by adjusting the start time points (In_Time) and the end time points (Out_Time) of the playitems and the start time points (In_Time) and the end time points (Out_Time) of the auxiliary playitems. On the other hand, the present embodiment does not require the connection points to be consistent and allows some degree of simultaneity in order to mix the audio.

In the case of providing the possibility of simultaneity requires another restriction. The aforementioned STC change sequence of operations is performed on seamless connections between playitems and between sub-playitems, and this change process is executed when the decoder is in a state without external synchronization. Here, in an uninterrupted connection, the decoder cannot switch to synchronous control until the STC is returned, and therefore an uninterrupted connection involving an STC change cannot often be allowed due to implementation problems. Accordingly, the connection points CC = 5 remaining both in the playitems and the auxiliary playitems must be restrained from appearing at a predetermined interval (for example, three seconds or so) from each other.

Fig. 42 shows a relationship between playitems and sub-playitems defined by a playlist indicating audio mixing. Level 1 of FIG. 42 shows three playitems (playitem information # 1, playitem information # 2 and playitem information # 3) in the playlist information. These three playitems define the primary TS, and the connection between the playitem information # 1 and # 2 is set to CC = 1, while the connection between the playitem information # 2 and # 3 is set to CC = 5. The level 2 of FIG. 42 shows three sub-play items (sub-play item # 1, sub-play item # 2 and sub-play item # 3) in the playlist information. These three sub-play elements define the secondary TS, and the connection between the sub-play items # 1 and # 2 is set to CC = 1, while the connection between the sub-play items # 2 and # 3 is set to CC = 5. The level 3 of FIG. 42 shows nine sub-play items (sub-play item # 1, sub-play item # 2, from sub-play item # 3 to sub-play item # 9) in the progressive playlist information. These nine sub-play elements define the secondary TS. Here, the connection between the sub-play items # 3 and # 4 is set to CC = 1, the connection between the sub-play items # 4 and # 5 is set to CC = 5, and the remaining connections are set to CC = 6.

In the figure, the start of the level 2 sub-play element # 3 is 3 seconds before the start of the level 1 play-back item # 3. Similarly, the start moment of the level 3 sub-play item # 5 is 3 seconds before the start of the level 1 play-back element # 3.

The time period for changing the time axes of the STC playitems and auxiliary play items is 3 seconds, and therefore, the change of the time axes of the STC does not occur too often.

The timing CC = 1 for playitems is set in accordance with SP_CC = 1. This is intended to prevent the reproduction of playback elements and auxiliary playback elements from exiting synchronization in the case where playback continues only of auxiliary playback elements when the connection is non-interrupted at CC = 1.

The connection mode for connecting auxiliary playback elements with SP_CC = 5 in the middle of the playback elements becomes useful when both the stage version and the director’s cut are stored on a single disc.

Level 1 of FIG. 43 shows an example of playlist information constituting both a stage version and a directorial cut. Within the playlist information, the directorial cut consists of a playitem # 1, a playitem # 2 and a playitem # 4, while a stage version consists of a playitem # 1, a playitem # 3, and a playitem # 4. Thus, since the playitem # 1 and the playitem # 4 are shared in two ways, chapters can be created rationally. Since part of the video in each version different from the other is shorter than the total duration of the video, the amount of data recorded on the disc can be effectively reduced. The level 2 of FIG. 43 shows an example in which the speaker texts corresponding to the playback element # 1, the playback element # 2 and the level 1 playback element # 4 are defined as one auxiliary playback element, and the speaker texts corresponding to the playback element # 1, playitem # 3 and playitem # 4 are defined as another sub-playitem. In this case, the narration texts corresponding to the playback element # 1 and the playback element # 4 should be prepared for each of the two auxiliary playback elements, which is unfavorable in terms of data volume.

The level 3 of FIG. 43 shows an example in which sub-play items (sub-play item # 1, sub-play item # 2, sub-play item # 3, and sub-play item # 4) are defined, each of which corresponds to a play item # 1, playitem # 2, playitem # 3, and playitem # 4. Assume that the connections of the auxiliary playback element # 1 to the auxiliary playback element # 2 and to the auxiliary playback element # 3, as well as the connections of the auxiliary playback element # 2 to the auxiliary playback element # 3 and to the auxiliary playback element # 4 are in the state CC = 5 . These connection points appear at time points away from the connection points of the playitems. That is, on the announcer's text side, a branch to the sub-play element # 2 or the sub-play element # 3 is called up 3 seconds before the playback element # 1 ends using CC = 5 (or CC = 6).

In addition, branching to sub-play item # 4 is called out 3 seconds after the play item # 2 and play item # 3 are completed using CC = 5 (or CC = 6). The beginning of the auxiliary playback element # 2 and the auxiliary playback element # 3 and the beginning of the auxiliary playback element # 4 are respectively 3 seconds away from the beginning of the playback element # 2 and the playback element # 3 and the beginning of the playback element # 4. By providing such time periods, changing the time axes of the STC does not occur too often.

In a certain sense, CC = 5 is required solely to cause a return from sub-play element # 3 to sub-play element # 4 (a seamless connection in which the ATC / STC time axes are reset), and CC = 6 can be used instead of CC = 5 for the remaining branches.

Thus, according to the present embodiment, since the start time (In_Time) and the end time (Out_Time) of the playitems do not correspond to the start time (In_Time) and the end time (Out_Time) of the sub play elements, synchronization of the ATC counters 2a and 2c, as well as counters 3a and 3c, the STC is optional, which increases the freedom of design of the playback devices.

Option 7 implementation

In Embodiment 6, primary and secondary audio streams are targets for limiting the number of bits when they are simultaneously read from the BD-ROM and the local storage device and supplied to the decoder. The present embodiment explains the bit limit imposed when a multi-window rendering (PiP) rendering application is implemented.

PiP playback, when the main clips constituting the moving pictures are specified by the main path information from the playlist information and the sub clips constituting another set of moving pictures are defined by the information about the sub-items from the playlist information is a technology for displaying the first moving pictures (primary video) and the latest moving images (secondary video) on the same screen. Here, the primary video consists of HD images, while the secondary video consists of SD images. HD images have a resolution of 1920 × 1080 with a frame synchronization cycle of 3750 (alternatively, 3753 or 3754), similar to film material. SD images have a resolution of 720x480 with a display synchronization cycle of 1501, similar to NTSC material (standard of the National Committee for Television Systems) or a frame synchronization cycle of 1800, similar to PAL material (standard with progressive phase change).

SD images have about 1/4 of the resolution of HD images, and therefore, if the primary video, which are HD images, and the secondary video are displayed on the same screen, the dimensions of the secondary video are about 1/4 relative to the primary video.

Here we assume that the secondary video is moving images in which the director and / or cast appear and give an idea, for example, highlighting the main points in the video content of the primary video. In this case, by combining the video content of the secondary video with the video content of the primary video, it is possible to realize an entertaining screen effect, where the director and the cast give an announcer's text while highlighting the main points of the content in the video playing the movie.

<Playlist information of the present embodiment>

The video stream for the secondary video (secondary video stream) is defined by numerous pieces of information about the sub-play item in the sub-path information from the play list information. To such information about the sub play item, the PiP position information (PiP_Position) and the PiP size information (PiP_Size) are again added.

“PiP_Position” indicates the position at which the secondary video playback video should be located using the X and Y coordinates on the screen plane used to play the primary video.

"PiP_Size" ("PiP size") indicates the height and width of the video playback of the secondary video.

Further, the connection condition information SP of sub play elements in the present embodiment is set to “= 5”. This means guaranteeing an uninterrupted connection between the secondary video stream multiplexed into auxiliary clips of the current auxiliary playback element and the secondary video stream multiplexed into the auxiliary clip of the previous auxiliary playback element. The connection condition information SP (sp_connection_condition) of such sub-play items is set to the same value as the connection condition information (connection_condition) from the play item information. Therefore, if the connection condition information (connection_condition) from the information about the playitem is “= 5”, the information about the connection condition SP (sp_connection_condition) from the information about the playitem should also be set to “= 5”. That is, if the primary video stream on the side of the playitem is connected seamlessly, the secondary video stream on the side of the playitem should be seamlessly connected. In addition, the information about the start time (In_Time) and the end time (Out_Time) of the sub play item should indicate the same time points as the information about the start time (In_Time) and the end time (Out_Time) of the play item.

This is the description of the recording medium of the present embodiment.

<Improvement of the playback device of the present embodiment>

The following explains improvements to the playback device. In order to perform decoding processing of the secondary video streams, the hardware of the reproducing apparatus of the present embodiment includes another set of structural elements used to decode the video streams. Here, the structural elements used to decode the video streams are: transport buffer; multiplexed buffer; elementary buffer; a decoder and a video card, and decode the secondary video streams. In addition, the reproducing apparatus of the present embodiment includes a scaling unit and a synthesis unit described below.

The scaling module increases or decreases the size of the video playback in the plane of the secondary video in height and width indicated by the size PiP (PiP_Size) of the information about the sub-play element.

The synthesis module reproduces PiP by synthesizing video playback, the size of which has been increased by the scaling module, and video playback received by the video decoder. The synthesis of video playback of the primary video and video playback of the secondary video is performed in accordance with the position PiP (PiP_Position) specified by the information about the element of the auxiliary video. By this, the synthesized video that is created by synthesizing video playing the primary video and video playing the secondary video can be played. The synthesis unit is capable of performing color PiP projection synthesis, layer synthesis, and the like, and, for example, performs a sequence of operations to remove the background of a secondary video, extract a human image, and synthesize a human image from a primary video playback video. This is a description of the reproducing apparatus of the present embodiment.

<Verification by PiP application>

In the case where the video stream, which is the primary TS (primary video stream), and the video stream, which is the secondary video stream (secondary video stream), are read at the same time and fed to the decoder in order to play PiP, the primary and secondary video streams are targets for checking for bit limit.

More specifically, while the window is shifted along the ATC time axis, the stream having the highest calculated bit rate is selected on each coordinate of the ATC time axis indicated by the ATS, for each type of primary video stream, secondary video stream, multiple primary audio streams, multiple secondary audio streams , multiple PG streams and multiple IG streams. The highest bit rate of the primary video stream, the highest bit rate of the secondary video stream, the highest bit rate of the primary audio stream, the highest bit rate of the secondary audio stream, the highest bit rate of the PG stream and the highest bit rate of the IG stream are summed, and it is judged whether the total amount is 48 Mbps or less.

Thus, according to the present embodiment, it is guaranteed that the number of bits per second does not exceed a predetermined upper limit, even when primary and secondary video streams are simultaneously read from both the BD-ROM and the local storage device and supplied to respective decoders. With such a guarantee, a PiP application can be created rationally.

(Additional notes)

The best option for carrying out the invention has been described, as far as the applicant knows, at the time of filling out this application. However, further improvements or modifications may be made to the present invention based on the following technical issues. It should be noted here that to make or not such improvements or modifications is optional and depends on the means of implementation of the invention.

(Start time (In_Time), end time (Out_Time))

27, the last video presentation block from TS1 is selected for the end time point (Out_Time) of the previous playitem, while the first video presentation block from TS2 is selected for the start time (In_Times) of the previous playitem and the previous sub-play element. Instead, however, the intermediate video presentation unit in TS1 can be selected for the end time point (Out_Time) of the previous playitem, while the intermediate video presentation unit in TS2 can be selected for the start time (In_Time) times of the current playitem and the current sub play item . In this case, seamless connections cannot be made for the current playitem and the current sub-playitem, and they must be connected using CC = 1 and SP_CC = 1.

(All playlist information)

When it is required to connect two playitems at CC = 5, all information about playitems and all information about auxiliary playitems that belongs to one piece of playlist information must be connected at CC = 5.

(The amount of data supplied to the decoder)

As for over-compaction, the amount of data supplied to the decoder does not always become large. For example, let’s say the case where the primary audio stream is the main clip and consists of DD (Dolby Digital format) for CBR and MLP for VBR, and MLP is replaced with DD from CBR supplied from the local storage device. In this case, the amount of data supplied to the decoder is actually reduced. If the case is obvious, the verification process may be omitted.

(Playback time difference)

In order to implement CC = 5 and SP_CC = 5, it would be desirable if the difference in the playback time of video and audio streams in one playback element was small. The acceptable difference may be: a period of time equivalent to one video frame (from 1/60 to 1/25 of a second); one second or less; a period of time corresponding to a certain fraction of the total playback period (for example, 1% or less); or a combination of two of these. This is also true for the time difference between the playback of video and audio streams in one sub-play element.

In the case where two elementary streams are stored in the same PID, it is desirable that the difference in the playback time of the two streams stored at the same PID is the same or less than the minimum playback unit (1 frame) of the stream having a shorter playback time . This condition can be met by storing the Dolby Digital (AC-3) and MLP (meridian-free packing format) formats in a single elementary stream, and then writing the elementary stream to a BD-ROM.

(Processing additional content)

It is desirable to perform the initial setup of the playback device so that additional content downloaded to the local storage device 200 will be automatically deleted when several months or several years after loading have elapsed.

(Replacing PID)

When an audio mixing application is implemented, PIDs are used to distinguish between primary and secondary audio streams; however, when using MPEG2-PG, it is desirable to make the stream id (stream_id) of the PES packet headers different from each other.

In addition, the primary and secondary audio streams should only differ at the system stream level, so that two audio streams can be distinguished by one demultiplexer. Alternatively, before multiplexing two streams, the PIDs of one of the streams may be replaced by other PIDs.

(Preload)

Preferably, preloading the audio data (“sound.bdmv” file) for the sound of the click of a button is performed when the BD-ROM is loading or when the chapter is switched. This is because if reading the sound.bdmv file is attempted during the playback of the AV clip, the search operation of the optical pickup head for reading a file other than the AV clip is called. On the other hand, when a BD-ROM is loaded or when a chapter is switched, it rarely happens that the AV clip continues to play. Therefore, by reading the sound.bdmv file with such a time reference, it is possible to improve the sensitivity of the device and make it difficult to interrupt the playback of the AV clip.

(Java ™ Platform)

The Java ™ platform can be structured with a full installation on the playback device for each embodiment of the personal 2Micro_Edition (J2ME) Java ™ profile (PBP 1.0) profile and the globally executable MHP specification (GEM1.0.2) for batch audio-visual targets, and then the playback device can be prompted run the BD-J application. To run the application, the reproducing device may be prompted to execute a super-densification infrastructure.

(Chapter)

It is preferable to create a “module manager” in the playback device, which selects a chapter according to the mounting of the BD-ROM, user operation or device status. The decoder in the BD-ROM playback device plays the AV clip based on the information of the playlist according to the selection of the "module manager" chapter.

When the “module manager” selects a chapter, the application manager makes an alarm using the application management table (AMT) corresponding to the previous chapter and AMT corresponding to the current chapter. The alarm takes control, which terminates the application described in AMT of the previous chapter, but not described in AMT of the current one, while it starts the application that is not described in AMT of the previous chapter, but described in AMT of the current chapter.

(Directory structure in local storage)

The individual areas in the local storage described in each embodiment are preferably created under a directory corresponding to the root certificate of the BD-ROM.

A root certificate is a root certificate that is distributed by the root certificate service and assigned to the BD-ROM by the creator of the BD-ROM. The root certificate of the disk is encoded, for example, in X.509 (data encryption standard). The X.509 specifications were issued by the International Telegraphy and Telephony Advisory Committee (CCITT) and are described in the CCITT Recommendation X.509 (1988), “The Directory - Authentification Framework” (X.509 CCITT Recommendations (1988), “Catalog - authentication infrastructure ").

In addition, it is preferable that the contents recorded on the BD-ROM and the local storage device are encoded using an Advanced Content Access System (AACS), the signature information is attached to them, and the authorization of use is specified in the authorization file.

(Software package to be installed)

When the BD-ROM playback device is implemented as a Java ™ platform, it is desirable to install the following BD-J extension on the playback device. The BD-J extension includes various software packages specialized for providing functions beyond GEM [1.0.2] for the Java ™ platform. The software packages included in the BD-J extension are shown below.

- org.bluray.media

This software package provides special features that must be added to the Java ™ audio-visual infrastructure. Controls for selecting angle, audio and subtitle are added to the software package.

- org.bluray.ti

This software package includes: API (application programming interface) for mapping GEM "services" [1.0.2] to a "chapter"; a mechanism for inquiring about chapter information from the BD-ROM; and a mechanism for selecting a new chapter.

- org.bluray.application

This software package includes an API for managing active periods of the application. In addition, the software package includes an API for inquiring about the information required to signal when the application is running.

- org.bluray.ui

This software package includes classes that define the constants used for key events specialized for the BD-ROM, and synchronize with video playback.

- org.bluray.vfs

This software package provides a mechanism (linking scheme) for linking contents recorded on a BD-ROM (contents on a disk) and contents on a local storage device (contents off the disk) that are not recorded on a BD-ROM in order to reproduce data without interruption in a way that doesn't care where the data is written.

The linking scheme associates the contents on the BD-ROM (AV clip, subtitle and BD-J application) with related contents in the local storage device. The linking scheme implements non-stop playback indifferent to where the contents are recorded.

(Virtual software package)

The BD-ROM playback device may be prompted to perform a series of operations creating a virtual software package. This is done by the fact that the playback device creates information about the virtual software package. Virtual package information is information obtained by expanding volume management information on a BD-ROM. Here, the volume management information is information that defines the directory and file structure existing on the recording medium and consists of directory management information related to directories and file management information related to files. Information about the virtual software package is intended to expand the directory structure and BD-ROM files by adding new file management information to the volume management information showing the directory structure and BD-ROM files.

(Implementation of the control procedure)

Both the control procedures explained in the above embodiments using flowcharts and the control procedures explained in the above embodiments satisfy the requirements of the “invention of the program” since the above control procedures are implemented using the specific use of hardware resources and are the creation of technical ideas using natural law.

- Production program of the present invention

The program of the present invention is an object program that can be executed on a computer. An object program consists of one or more control programs that cause a computer to perform each step in a flowchart of a method or each procedure of functional components. There are various types of control programs, such as native processor code and JAVA ™ byte code. There are also various forms of implementation of the stages of control programs. For example, when each step can be implemented by using an external function, call operators to call external functions are used as control programs. Control programs that perform one step may belong to different object programs. In a RISC processor (with a reduced instruction set) in which instruction types are limited, each step in the flowcharts of the sequence of operations of the methods can be carried out by combining arithmetic instructions, logical instructions, branch instructions, and the like.

The program of the present invention can be produced as follows. First, a software developer writes an initial program using a programming language that successfully executes each flowchart of a method and a functional component. In this writing, the software developer uses the class structure, variables, variables of the "array" type, calls to external functions, and so on, which obey the rules of the sentence structure of the programming language that he / she uses.

The written source program is sent to the compiler as files. The compiler translates the source program and generates an object program.

Translation performed by the compiler includes a series of operations, such as analysis of the structure of sentences, optimization, resource allocation, and code generation. When analyzing the structure of sentences, symbols and phrases, the structure of sentences and the meaning of the original program are analyzed, and the original program is converted into an intermediate program. During optimization, the middleware undergoes such sequences of operations as tuning basic elements, analyzing the control flow, and analyzing the data flow. When allocating resources to adapt to the instruction sets of the target processor, variables in the middleware are allocated to the registers or memory of the target processor. When generating the code, each intermediate instruction in the intermediate program is converted into a control program and an object program is obtained.

After the object program is formed, the programmer activates the linker. The linker allocates memory spaces to object programs and related library programs, and links them together to form a boot module. The generated boot module is based on the assumption that it is read by the computer and prompts the computer to perform the procedures indicated in the flowcharts of the sequence of operations of the methods and the procedures of the functional components. The program of the present invention can be created in this way.

The program of the present invention can be used as described below. When the program of the present invention is used as an embedded program, the boot module as a program is written to the instruction ROM together with the basic input / output system (BIOS) program and various pieces of middleware (operating systems). The program of the present invention is used as a control program of the playback device 300, while the instruction ROM is integrated in the control module, and is executed by the CPU.

When the playback device is a model of the boot program, the basic input / output system (BIOS) program is built into the ROM of the instructions, and various parts of the middleware (operating systems) are pre-installed on the hard drive. In addition, a boot ROM to activate the system from the hard disk is provided in the playback device. In this case, only the boot module is supplied to the playback device through a portable recording medium and / or network and is installed on the hard disk as a single application. This enables the playback device to boot through the boot ROM to activate the operating system, and then prompts the CPU to execute the installed boot module as a single application, so that the program of this application can be used.

As described above, when the playback device is a model of a hard disk, the program of the present invention can be used as a single application. Accordingly, the program of the present invention can be transferred, borrowed on time or delivered separately through the network.

(Controller 22)

The controller 22 may be implemented as a single system LSI.

System LSI is obtained by implementing a packageless integrated circuit on a high-density substrate and mounting them in a housing. A system LSI is also obtained by implementing a number of open-circuit integrated circuits on a high-density substrate and mounting them in a housing, so that many open-circuit integrated circuits have the appearance of a single LSI (such a system LSI is called a multi-chip module).

System LSI has type QFP (square flat case) and type PGA (case with a matrix arrangement of pin terminals). In a QFP type system LSI, pins are attached to four sides of the chassis. In a PGA type system LSI, all of the pins are attached to a hollow bottom.

These contacts act as a link to other circuits. System LSI, which is connected to other circuits through such contacts as a connecting link, plays a role as the core of the playback device.

Each of the open-circuit integrated circuits packaged in the system LSI consists of: an input stage module; output stage module and digital processing module. The input stage module digitizes the analog signal, while the output stage module converts the received data into analog form and outputs it.

Each structural element shown in the internal structure diagram in the above embodiment is installed in the digital processing unit.

As described above by “used as firmware”, the boot module as a program, the basic input / output system (BIOS) program, and various pieces of middleware (operating systems) are written in the instruction ROM. The most important improvement of the embodiments is achieved by the boot module as a program. Therefore, it is possible to manufacture the system LSI of the present invention by mounting instructions in the ROM case as open-circuit integrated circuits, in which the boot module is stored as a program, in the form of a open-circuit integrated circuit.

It is advisable to use the SoC implementation or the SiP implementation for the actual implementation. The implementation of SoC (systems on a chip) is a technology that burns through multiple circuits in a single integrated circuit. Implementing SiP (systems in a package) is a technology that places multiple integrated circuits in a single package using polymer. By the above procedure, the system LSI of the present invention can be made based on the internal structure diagrams of the reproducing apparatus described in each embodiment.

It should be noted that, although the term LSI is used here, it can be called IC (integrated circuit, IC), LSI, VLSI (ultra-large integrated circuit, super LSI), UBIS (ultra-large integrated circuit, ultra LSI) or the like, depending on degree of integration.

In addition, part or all of the components of each playback device can be made in the form of a single crystal. The integrated circuit is not limited to the implementation of SoC or the implementation of SiP, but can be performed using a specialized circuit or general-purpose processor. It is also possible to carry out an integrated circuit by using an FPGA (Field Programmable Gate Array), which can be reprogrammed after its manufacture, or a reconfigurable processor that can reconfigure the connection and settings of the circuit cells inside the LSI. Moreover, the technology for the integrated circuit, which replaces the LSI, may appear in the near future, as the semiconductor technology is improved or branched into other technologies. In this case, the new technology may be included in the integration of the functional blocks constituting the present invention, as described above. Such possible technologies include biotechnology.

Industrial applicability

The recording medium and playback device of the present invention can be produced in series based on their internal structures shown in the embodiments above. As such, the recording medium and reproducing apparatus of the present invention have industrial applicability.

Claims (5)

1. A playback device for reproducing in accordance with the information of the playlist of the main stream in which the primary playback section is determined, and the auxiliary stream in which the secondary playback section is determined, wherein
playlist information defines, with respect to each of the plurality of digital streams, a playback section and includes main path information and auxiliary path information,
information about the main path sets from among the number of digital streams one digital stream as the main stream and determines the primary playback section in the main stream,
the auxiliary path information sets, from among the remaining part of the plurality of digital streams, one digital stream as an auxiliary stream and determines in the auxiliary stream a secondary reproduction section to be synchronized with the primary reproduction section,
information about the main path further includes a table of streams showing at least one pair of elementary streams that are allowed to play simultaneously, moreover, this pair of elementary streams is composed of at least one of a plurality of elementary streams multiplexed into the main stream, and at least one of the many elementary streams multiplexed into the auxiliary stream, each of the main stream and the auxiliary stream is sequence of transport stream packets (TS), in which a plurality of packets TS, each of which has attached thereto a receipt timestamp,
wherein the playback device contains
a first reader module, configured to read among the TS packets in the main stream recorded on the first recording medium, TS packets constituting a section corresponding to the primary playback section according to the main path information,
a second reader module, configured to read among the TS packets in the auxiliary stream recorded on the second recording medium, TS packets constituting a section corresponding to the secondary reproduction section according to the auxiliary path information,
the first and second filters, configured to select from among the TS packets read by the first and second reading modules, the TS packets that are allowed to play in the stream number table included in the playlist information, and
the first and second source depacketting means, configured to issue TS packets from the first and second readout modules to the first and second filters at a time indicated by the timestamp of the receipt attached to each of their TS packets,
wherein the total data volume of said at least one pair of elementary streams, which are TS packets and which must be reproduced simultaneously per unit time, is less than or equal to a predetermined value and
said total amount of data per unit time is calculated in a window, which is a check frame on the time axis, acting as a reference for timestamps of receipt, and this total amount of data is less than or equal to said predetermined value regardless what point on the time axis is the window mentioned. 2. The playback device according to claim 1, in which
the first recording medium is a read-only optical disc, and the second recording medium is a rewritable recording medium,
the auxiliary stream and playlist information are recorded in a predetermined area on a rewritable recording medium, and this predetermined area is connected to a read-only optical disk in order to create a virtual data container in combination with it,
wherein the first and second read modules simultaneously read the TS packets from a read-only optical disc and the TS packets from a rewritable recording medium. 3. The playback device according to claim 1, in which at least one of said plurality of TS packets constituting an auxiliary stream overlaps on the time axis with at least one of said plurality of TS packets constituting the main stream,
moreover, TS packets from the first and second source de-packaging means are simultaneously issued to the first and second filters at the moment corresponding to the coordinate of the time axis indicated by each of the arrival timestamps. 4. The playback device according to claim 1, in which each of the main stream and the auxiliary stream is recorded after its bit rate is checked at the coordinates indicated by the aforementioned set of time stamps of receipt,
the initial state of this check corresponds to where the starting point of the said window is located at the time indicated by the arrival time stamp attached to the first TS packet from the plurality of TS packets constituting the main stream and the auxiliary stream,
the final state of this check corresponds to where the end point of the said window has reached the point in time indicated by the arrival time stamp attached to the last TS packet from the plurality of TS packets constituting the main stream and the auxiliary stream,
wherein said verification is a process of verifying that the total size of TS packets included in said window is less than or equal to said predetermined value when shifting the coordinates of said starting point and said ending point at coordinates indicated by said plurality of arrival time marks between said initial state and said final state. 5. The playback device according to claim 1, in which the total size of the TS packets is calculated by summing the highest bit rates of the TS packets, each of these highest bit rates being selected among the bit rates of the TS packets constituting each of these elementary streams that are allowed to play simultaneously in the stream number table.

Images