JP4968562B2 - REPRODUCTION DEVICE, REPRODUCTION METHOD, AND RECORDING METHOD - Google Patents

Description

The present technology relates to a playback device, a playback method , and a recording method, and more particularly to a playback device, a playback method , and a recording method that enable selection of various data to be played back along with a main AV stream.

  In the DVD (Digital Versatile Disc) video standard, when AV (Audio Visual) content such as a movie recorded on a recording medium is played, an interactive operation such as audio switching or subtitle switching is provided to the user. (For example, refer nonpatent literature 1). Specifically, for the AV content displayed on the display device 1 in FIG. 1, the user operates the audio switching button 11 or the subtitle switching button 12 of the remote controller 2 to switch the audio or subtitle. Switch. For example, when the voice 1 is set as the initial state and the user operates the voice switching button 11 of the remote controller 2, the voice 1 is switched to the voice 2 as shown in FIG.

AV content on a DVD video is recorded in MPEG (Moving Picture Experts Group) 2 program stream format. As shown in FIG. 3, the MPEG2 program stream includes a video stream (video in FIG. 3), a plurality of audio streams (audios 1, 2, and 3 in FIG. 3), and a plurality of sub-picture streams (in FIG. 3). The sub-pictures 1, 2, 3) are multiplexed so as to be reproduced in AV synchronization with the video stream.
The sub-picture stream (sub-picture 1, 2, 3) is a stream in which a bitmap image is run-length encoded, and is mainly used for subtitles.

  In general, a plurality of audio streams are used for recording audio in different languages, and a plurality of sub-picture streams are used for recording subtitles in different languages. When the video is being played back, the user can interactively select audio or subtitles in a desired language using the remote controller 2.

  Also, the DVD video includes audio numbers and subtitles provided to the user for a plurality of audio streams (audio 1, 2, 3) and a plurality of sub-picture streams (sub-pictures 1, 2, 3) in the program stream. Defines a table structure that represents the number relationship.

  FIG. 4 is a diagram for explaining a stream number table representing the relationship between audio signals and caption signals provided to the user. Here, the audio number is referred to as A_SN (Audio Stream Number), and the subtitle number is referred to as S_SN (SubPicture Stream Number). In FIG. 4, A_SN is given to each of the plurality of audio streams of the MPEG2 program stream, and S_SN is given to each of the plurality of sub-picture streams of the MPEG2 program stream. A_SN = 1: Audio 2; A_SN = 2: Audio 1; A_SN = 3: Audio 3 Also, S_SN = 1: sub-picture 3, S_SN = 2: sub-picture 1, and S_SN = 3: sub-picture 2. Here, the smaller the A_SN or S_SN number, the higher the priority as the audio signal provided to the user. That is, A_SN = 1 is an audio stream reproduced by default, and S_SN = 1 is a sub-picture stream reproduced by default.

  Specifically, the sound 1 reproduced in the initial state of FIG. 1 corresponds to the audio 2 (FIG. 4) with A_SN = 1, and the sound 2 reproduced in FIG. 2 after the sound is switched. Corresponds to audio 1 (FIG. 4) with A_SN = 2.

  In addition, there is a technique in which a Sub path is used to reproduce additionally recorded sound (for example, so-called after-recording sound) (for example, Patent Document 1).

JP 2000-158972 A

DVD Specifications for Read-Only Disc Part 3; Version1.1

  However, in the case of DVD video, when a program stream video is being played back, the user's operations such as audio switching and subtitle switching are performed from among the audio stream and sub-picture stream multiplexed in the program stream being played back. However, I could not choose. In other words, when the MPEG2 program stream as shown in FIG. 3 is being reproduced, when switching the audio, the choice is one of the audio 1 to 3.

  For this reason, when an audio stream and subtitles are prepared in a different stream from the program stream being played back, the user cannot select an audio switching operation from the other stream, so there is no scalability. There was a problem.

  Furthermore, in the future, it is assumed that various types of content such as video as well as audio will be targeted as the content to be switched. Therefore, in the future, since these various types of content switching operations cannot be selected from other streams, the problem will not be scalable.

  Moreover, in the technique described in Patent Document 1, a sub path is used to reproduce additionally recorded sound (for example, so-called after-recording sound). There is no disclosure of a configuration for synthesizing and reproducing different types of streams, that is, two audio streams and two video streams.

  The present technology has been made in view of such a situation, and makes it possible to select various data to be reproduced together with a main AV stream.

Reproducing apparatus of an embodiment of the present technology is referred to by the main playback path includes information indicating a playback section on the time axis of the first stream comprising a first video stream and the first audio stream, the main and information PlayItem constituting the playback paths are referred to by the sub playback paths, including information indicating a playback section on the time axis of the second stream comprising a second video stream and the second audio stream, wherein SubPlayItem information constituting a secondary playback path, and table information included in the PlayItem information and used for selecting a stream to be played back, which can be played back, and the second video stream preparative acquires the said table information including combination information indicating a combination of an audio stream, the playback management information including the loaded optical disk Decoding a part, a reading unit for reading the sound data from the optical disk is a data of the first stream and the second stream and sound effects, the first video stream included in the first stream A first video decoder; a second video decoder for decoding the second video stream included in the second stream; and a first for decoding the first audio stream included in the first stream. And a second decoder that decodes the second audio stream that is included in the second stream and that is indicated by the combination information that can be reproduced in combination with the second video stream. An audio decoder and a decoding by the first audio decoder Synthesizing a first combining unit for combining the audio data obtained by decoding, the audio data and the sound data after synthesis by the first synthesis unit according to the audio data and the second audio decoder obtained by A second combining unit .

  The table information may further include identification information of the second audio stream that can be reproduced in combination with the second video stream.

In one aspect of the present technology, is referenced by the main playback path includes information indicating a playback section on the time axis of the first stream comprising a first video stream and the first audio stream, playback of the main and information PlayItem constituting the path, is referred to by the sub playback path includes information indicating a playback section on the time axis of the second stream comprising a second video stream and the second audio stream, the said sub SubPlayItem information constituting a playback path, and table information included in the PlayItem information and used for selecting a stream to be played back, which can be played back and the second audio stream reproduction management information including the table information including combination information indicating a combination of a stream, a is obtained from the optical disk mounted, before And sound data are data of the first stream a second stream and sound effects are read from the optical disc. In addition, the first video stream included in the first stream is decoded by a first video decoder, the second video stream included in the second stream is decoded by a second video decoder, The first audio stream included in the first stream is decoded by a first audio decoder, included in the second stream, and can be reproduced in combination with the second video stream. The second audio stream indicated by the combination information is decoded by a second audio decoder, obtained by decoding by the first audio decoder and by decoding by the second audio decoder Ode Odeta is synthesized. Also, the synthesized audio data and the sound data are synthesized.

  According to the present technology, it is possible to select various data to be reproduced together with the main AV stream.

It is a figure explaining the conventional voice switching. It is a figure explaining the conventional voice switching. It is a figure explaining the structure of an MPEG2 program stream. It is a figure explaining the stream number table showing the relationship between the audio | voice signal provided to a user, and a caption signal. It is a figure which shows the example of the application format on the recording medium with which the reproducing | regenerating apparatus to which this invention is applied is mounted | worn. It is a figure explaining the structure of a main path and a sub path. It is a figure explaining the example of a main path and a sub path. It is a figure explaining another example of a main path and a sub path. It is a figure which shows the syntax of PlayList (). It is a figure which shows the syntax of SubPath (). It is a figure explaining SubPath_type. It is a figure which shows the syntax of SubPlayItem (i). It is a figure which shows the syntax of PlayItem (). It is a figure which shows the syntax of 1st STN_table (). [Fig. 11] Fig. 11 is a diagram illustrating a first example of the syntax of stream_entry (). It is a figure which shows the example of a syntax of 2nd STN_table (). It is a figure which shows the 2nd example of the syntax of stream_entry (). It is a figure which shows the syntax of stream_attribute (). It is a figure explaining stream_cording_type. It is a figure explaining video_format. It is a figure explaining frame_rate. It is a figure explaining aspect_ratio. It is a figure explaining audio_presentation_type. It is a figure explaining sampling_frequency. It is a figure explaining Character code. It is a figure explaining the example of the stream number table showing the relationship between the audio | voice signal provided to a user, and a caption signal. It is a block diagram which shows the structural example of the reproducing | regenerating apparatus to which this invention is applied. It is a block diagram which shows the structural example of the reproducing | regenerating apparatus to which this invention is applied. It is a flowchart explaining the reproduction | regeneration processing in the reproducing | regenerating apparatus of FIG. It is a flowchart explaining the reproduction | regeneration processing in the reproducing | regenerating apparatus of FIG. It is a flowchart explaining the reproduction | regeneration processing in the reproducing | regenerating apparatus of FIG. It is a flowchart explaining a process in case switching of a voice is instruct | indicated by the user. It is a flowchart explaining the example of the detailed process of step S59 of FIG. It is a figure explaining the example in the case of carrying out mixing reproduction of two audios. It is a figure which shows the syntax of 3rd STN_table (). It is a block diagram which shows the structural example of the reproducing | regenerating apparatus to which this invention is applied. It is a figure which shows the example of the syntax of 4th STN_table () which defines the combination of audio stream # 1 and audio stream # 2. It is a figure which shows the example of the bitmap syntax of Combination_of_Primary_and_Secondary. It is a figure explaining the other example of the stream number table showing the relationship between the audio | voice signal provided to a user, and a caption signal. It is a figure explaining the other example which mixes and reproduces two audios. It is a figure which shows the example of the syntax of 5th STN_table () which defines the combination of audio stream # 1 and audio stream # 2. It is a figure which shows the example of the syntax of 6th STN_table () which defines the combination of audio stream # 1 and audio stream # 2. It is a figure explaining a picture in picture technique. It is a figure which shows the example of the syntax of 6th STN_table () in the case of defining the combination of the secondary video stream, the primary audio stream, the secondary audio stream, and the subtitle stream reproduced | regenerated simultaneously in combination with a primary video stream. It is a figure which shows the example of the syntax of 7th STN_table () in the case of defining the combination of the secondary video stream, the primary audio stream, the secondary audio stream, and the subtitle stream reproduced | regenerated simultaneously in combination with a primary video stream. It is a figure which shows the example of the stream number table showing the relationship which can combine the audio | voice signal provided to a user, the video signal as a secondary video stream, and a caption signal. It is a figure explaining the selection operation of the user for selecting the combination of a secondary video stream, a primary audio stream, a secondary audio stream, and a subtitle stream reproduced | regenerated simultaneously in combination with a primary video stream. In the case of realizing the picture-in-picture method, an example of a PlayList generated using the SubPath_type in the example of FIG. 11 is shown. It is a figure explaining the example different from FIG. 11 of SubPath_type. It is a figure which shows the 3rd example of the syntax of stream_entry (). An example of a PlayList generated using the SubPath_type in the example of FIG. 49 when the picture-in-picture method is realized is shown. When the picture-in-picture technique is realized, another example of the PlayList generated using the SubPath_type in the example of FIG. 49 is shown. FIG. 37 is a block diagram showing an example of the configuration of a playback apparatus to which the present invention is applied, which is different from FIG. FIG. 54 is a flowchart for describing secondary video switching processing in the playback apparatus of FIG. 53. FIG. FIG. 54 is a flowchart for describing secondary video switching processing in the playback apparatus of FIG. 53. FIG. It is a figure explaining the example of the kind of application_type. It is a figure for demonstrating a sub path | pass restriction | limiting technique, Comprising: It is a figure which shows the relationship of the combination of SubPath which the PlayList can have with respect to the kind of PlayList. It is a figure for demonstrating a sub path | pass restriction | limiting technique, Comprising: It is a figure which shows the relationship of the combination of the number of PlayItem and SubPath in PlayList. FIG. 60 is a diagram illustrating a specific example of a PlayList created by the subpath restriction method described with reference to FIGS. 57 to 59. FIG. 60 is a diagram illustrating a specific example of a PlayList created by the subpath restriction method described with reference to FIGS. 57 to 59. FIG. 60 is a diagram illustrating a specific example of a PlayList created by the subpath restriction method described with reference to FIGS. 57 to 59. FIG. 60 is a diagram illustrating a specific example of a PlayList created by the subpath restriction method described with reference to FIGS. 57 to 59. 54 is a flowchart for describing PlayList playback processing of application_type = 3 in the playback device of FIG. 53. 54 is a flowchart for describing PlayList playback processing of application_type = 3 in the playback device of FIG. 53. 54 is a flowchart for describing PlayList playback processing with application_type = 1or2 in the playback device of FIG. 53. 54 is a flowchart for describing PlayList playback processing with application_type = 1or2 in the playback device of FIG. 53. 54 is a flowchart for describing PlayList playback processing with application_type = 1or2 in the playback device of FIG. 53. 54 is a flowchart for describing PlayList playback processing with application_type = 1or2 in the playback device of FIG. 53. The eighth defines the combination of the audio stream # 1 and the audio stream # 2, the combination of the video stream # 2 and the audio stream # 1, and the combination of the video stream # 2 and the text subtitle stream for the video stream # 2. It is a figure which shows the example of the syntax of STN_table (). The eighth defines the combination of the audio stream # 1 and the audio stream # 2, the combination of the video stream # 2 and the audio stream # 1, and the combination of the video stream # 2 and the text subtitle stream for the video stream # 2. It is a figure which shows the example of the syntax of STN_table (). It is a figure for demonstrating a register. It is a figure for demonstrating the example in case a primary PG TextST stream is reproduced | regenerated at the time of execution of a picture in picture display. It is a figure for demonstrating the example in case the PG TextST stream for picture in picture applications is reproduced | regenerated at the time of execution of a picture in picture display. It is a flowchart for demonstrating the PG TextST reproduction | regeneration processing of the application containing PiP (picture in picture). It is a flowchart for demonstrating PiP PG TextST display processing. It is a figure which shows the example of the syntax of 9th STN_table () which defines the combination of still more streams. It is a figure which shows the example of the syntax of 9th STN_table () which defines the combination of still more streams. It is a figure which shows the example of the syntax of 9th STN_table () which defines the combination of still more streams. It is a figure for demonstrating a register. It is a figure for demonstrating the example in case primary audio and secondary audio are mixed and reproduced | regenerated at the time of execution of a picture in picture display. It is a figure for demonstrating the example in case the primary audio for picture in picture applications is reproduced | regenerated at the time of execution of a picture in picture display. It is a flowchart for demonstrating the reproduction process 1 of the audio stream of the application containing PiP. It is a figure which shows the example of the syntax of 10th STN_table () which defines the combination of still more streams. It is a figure which shows the example of the syntax of 10th STN_table () which defines the combination of still more streams. It is a figure which shows the example of the syntax of 10th STN_table () which defines the combination of still more streams. It is a figure for demonstrating the example in the case of performing the picture in picture display, the primary audio for picture in picture applications, and the secondary audio are mixed and reproduced | regenerated. It is a flowchart for demonstrating the reproduction process 2 of the audio stream of the application containing PiP. It is a figure for demonstrating manufacture of the recording medium which recorded the data which can be reproduce | regenerated with a reproducing | regenerating apparatus. It is a figure for demonstrating manufacture of the recording medium which recorded the data which can be reproduce | regenerated with a reproducing | regenerating apparatus. It is a figure which shows the structure of a personal computer.

  FIG. 5 shows an application format on a local storage (for example, a hard disk or a memory) of the playback apparatus 1 (described later with reference to FIGS. 27 and 28) to which the present invention is applied or on a recording medium to be mounted. It is a figure which shows the example of. The recording medium may be a magnetic disk or a semiconductor memory in addition to the optical disk described later.

The application format has two layers, PlayList and Clip, for managing AV (Audio Visual) streams. Here, a pair of one AV stream and Clip information as information accompanying the AV stream is considered as one object, and these are collectively referred to as a Clip. Hereinafter, the AV stream is also referred to as an AV stream file.
In addition to audio data and video data, the AV stream file may include various stream files that are reproduced together with these data. Clip information is also referred to as a Clip information file.

  In general, a file used in a computer or the like is handled as a byte string, but the content of an AV stream file is expanded on the time axis, and an access point of a clip is mainly designated by a PlayList with a time stamp. That is, PlayList and Clip are layers for AV stream management.

  When the access point in the Clip is indicated by the PlayList with a time stamp, the Clip Information file is used to find address information to start decoding in the AV stream file from the time stamp.

  The PlayList is a collection of AV stream playback sections. One playback section in a certain AV stream is called PlayItem, which is represented by a pair of an IN point (playback start point) and an OUT point (playback end point) of the playback section on the time axis. Therefore, the PlayList is composed of one or a plurality of PlayItems as shown in FIG.

  In FIG. 5, the first PlayList from the left is composed of two PlayItems, and the two PlayItems refer to the first half and the second half of the AV stream included in the left Clip, respectively. Also, the second PlayList from the left is composed of one PlayItem, whereby the entire AV stream included in the right Clip is referenced. Further, the third PlayList from the left is composed of two PlayItems, and the two PlayItems refer to a portion of the AV stream included in the left Clip and a portion of the AV stream included in the right Clip, respectively. ing.

  For example, when the disc navigation program of FIG. 5 specifies the left PlayItem included in the first PlayList from the left as the information indicating the playback position at that time, it is included in the left Clip referred to by the PlayItem. The first half of the AV stream is played back. Thus, the PlayList is used as playback management information for managing playback of AV stream files.

  The disc navigation program has a function of controlling the playback order of the PlayList and interactive playback of the PlayList. The disc navigation program also has a function of displaying a menu screen for the user to instruct execution of various reproductions. This disc navigation program is described in a programming language such as Java (registered trademark), and is prepared on a recording medium.

In this embodiment, a playback path created by a sequence of one or more PlayItems (by successive PlayItems) in the PlayList is called a main path (Main Path), and in the PlayList, parallel to the Main Path (in parallel) ) And a playback path created by a sequence of one or more sub paths (which may be discontinuous or may be continuous) is referred to as a sub path.
In other words, the application format on the local storage in the playback apparatus 1 (described later with reference to FIGS. 27 and 28) or on the recording medium to be mounted is played back in association with the main path. Sub Path is included in the PlayList.

  FIG. 6 is a diagram for explaining the structure of the main path and the sub path. The PlayList can have one main path and one or more sub paths. One main path is created by a sequence of one or more PlayItems, and one sub-path is created by a sequence of one or more SubPlayItems.

  In the case of the example in FIG. 6, the PlayList has one main path made up of a sequence of three PlayItems and three sub paths. Each PlayItem constituting the main path is assigned ID (Identification) in order from the top. Specifically, the main path is made up of PlayItems with PlayItem_id = 0, PlayItem_id = 1, and PlayItem_id = 2. Subpaths are also assigned IDs in order from the top, Subpath_id = 0, Subpath_id = 1, and Subpath_id = 2. A subpath of Subpath_id = 0 includes one SubPlayItem, a subpath of Subpath_id = 1 includes two SubPlayItems, and a subpath of Subpath_id = 2 includes one SubPlayItem.

  The stream referred to by the SubPlayItem included in the subpath of Subpath_id = 0 is assumed to be, for example, a Japanese dubbing sound of a movie, and may be played in place of the audio stream of the AV stream file referenced by Mainpath. Also, the stream referred to by SubPlayItem included in the subpath of Subpath_id = 1 is assumed to be a director's cut of a movie, for example, and a comment such as a movie director is included only in a predetermined part of the AV stream file referred to by Main Path. The case is considered.

  A Clip AV stream file referred to by one PlayItem includes at least video stream data (main image data). In addition, the Clip AV stream file may or may include one or more audio streams that are played back at the same timing (synchronously) as the video stream (main image data) included in the Clip AV stream file. It does not have to be. Furthermore, the Clip AV stream file may or may not include one or more bitmap subtitle streams (Presentation Graphic streams) that are played back at the same timing as the video stream included in the Clip AV stream file. Good. The Clip AV stream file may or may not include one or more interactive graphics streams that are played back at the same timing as the video stream included in the Clip AV stream file. The video stream included in the Clip AV stream file and the audio stream, bitmap subtitle stream file, or interactive graphics stream reproduced at the same timing as the video stream are multiplexed. That is, a Clip AV stream file referred to by one PlayItem includes video stream data, zero or more audio streams to be played in accordance with the video stream, zero or more bitmap subtitle stream data, and zero or more. The interactive graphics stream data is multiplexed.

  That is, a Clip AV stream file referred to by one PlayItem includes a plurality of types of streams such as a video stream, an audio stream, a bitmap subtitle stream file, or an interactive graphics stream.

  One SubPlayItem refers to audio stream data and subtitle data of a stream (different stream) different from the Clip AV stream file referred to by the PlayItem.

  When playing a PlayList that has only the main path, the user can select audio and subtitles only from the audio stream and sub-picture stream multiplexed in the clip referenced by the main path. Can not do it. On the other hand, when playing a PlayList having a main path and a sub path, in addition to the audio stream and sub picture stream multiplexed in the Clip AV stream file referenced by the main path, the clip audio stream referenced by the SubPlayItem It is also possible to refer to the sub-picture stream.

  In this way, since a plurality of SubPaths are included in one PlayList and each SubPath refers to each SubPlayItem, an AV stream with high expandability and high flexibility can be realized. In other words, in addition to the Clip AV stream referred to by MainPath, a SubPlayItem can be added later.

  FIG. 7 is a diagram illustrating an example of a main path and a sub path. In FIG. 7, an audio playback path that is played back at the same timing as the main path (synchronized with AV) is represented using sub-paths.

  The PlayList in FIG. 7 includes one PlayItem with PlayItem_id = 0 as a main path and one SubPlayItem as a sub path. One PlayItem () with PlayItem_id = 0 in the main path refers to the main Clip AV stream. SubPlayItem () includes the following data. First, SubPlayItem () includes Clip_Information_file_name for designating a Clip referred to by a Sub Path in the PlayList. In the case of the example in FIG. 7, an auxiliary audio stream (audio stream) with SubClip_entry_id = 0 is referred to by SubPlayItem. Also, SubPlayItem () includes SubPlayItem_IN_time and SubPlayItem_OUT_time for designating the playback section of the Sub Path in the Clip (Auxiliary audio stream here). Further, SubPlayItem () includes sync_PlayItem_id and sync_start_PTS_of_PlayItem for designating the time at which the Sub Path starts to be reproduced on the time axis of the Main path. In the example of FIG. 7, sync_PlayItem_id = 0 and sync_start_PTS_of_PlayItem = t1. As a result, it is possible to specify the time t1 at which the Sub Path starts playback on the time axis of PlayItem_id = 0 of the main path. That is, in the case of the example in FIG. 7, the reproduction start time t1 of the main path and the start time t1 of the sub path are the same time.

  Here, the Clip AV stream of the audio referred to by the Sub Path must not include STC discontinuities (system time base discontinuities). The audio sample clock of Clip used for the sub path is locked to the audio sample clock of the main path.

  In other words, in SubPlayItem (), information specifying the Clip referred to by the Sub Path, information specifying the playback section of the Sub Path, and information specifying the time when the Sub Path starts playback on the time axis of the Main path It is included. Since the Clip AV stream used in Sub Path does not include STC, information included in SubPlayItem () (information specifying Clip referred to by Sub Path, information specifying Sub Path playback section, and time axis of Main path) Based on the above (information specifying the time at which the Sub Path starts playback), refer to an audio stream of a Clip AV stream different from the Clip AV stream (mainly played main AV stream) referred to by the main path. Can play.

  In this way, PlayItem and SubPlayItem manage Clip AV stream files, respectively. Here, the Clip AV stream file (main AV stream) managed by PlayItem and the Clip AV stream file managed by SubPlayItem are different files. sell.

  Similar to the example of FIG. 7, the subtitle stream playback path that is played back at the same timing as the main path can also be expressed using the sub path.

  FIG. 8 is a diagram for explaining another example of the main path and the sub path. In FIG. 8, a playback path of audio that is played back at the same timing as the main path (synchronized with AV) is represented using sub paths. Here, the main Clip AV stream file referred to by PlayItem in the main path is the same as in FIG.

  For example, a clip AV stream referenced by the main path is set as one movie content (AV content), an auxiliary audio stream (Clip) referenced by the sub-path audio path is set as a director's comment for the movie, and is referenced by the main path. Such an arrangement is used in the case where an auxiliary audio stream referred to by the audio path of the sub path is mixed (reproduced) with the audio stream of the clip AV stream. In other words, when the user inputs a command to listen to the director's comment together with the movie while watching the movie, in other words, the audio of the Clip AV stream referenced in the main path and the sub path This is used when the auxiliary audio stream referred to in the audio path is mixed and reproduced.

  In FIG. 8, three PlayItems are arranged in the main path as PlayItem_id = 0, 1, 2, and two SubPlayItems are arranged in the subpath (Subpath_id = 0). SubPlayItem (described later in FIG. 12) called by SubPath with Subpath_id = 0 (described later in FIG. 10) is an Auxiliary audio stream (English Auxiliary audio stream clip with SubClip_entry_id = 0 and SubClip_entry_id = 1 Japanese) SubPlayItem_IN_time and SubPlayItem_out_time for specifying the subpath playback section of (Auxiliary audio stream clip).

  Comparing FIG. 8 and FIG. 7, in FIG. 8, Auxiliary audio stream (English or Japanese audio stream) of SubClip_entry_id = 0, 1 can be referred to by SubPlayItem. In other words, a SubPlayItem is used to refer to a plurality of audio stream files, and when the SubPlayItem is reproduced, the audio stream file is selected from the plurality of audio stream files and reproduced. In the case of the example in FIG. 8, one audio stream file is selected and reproduced from the English audio stream file and the Japanese audio stream file. Specifically, one of subClip_entry_id = 0 and 1 is selected (for example, based on a user instruction), and an auxiliary audio stream referred to by the ID is reproduced. In addition to this, when playback in combination with an audio stream referenced by the main path is further selected, the 1st audio stream referenced by the main path and the 2nd audio stream file referenced by the audio path of the sub path are selected. Are mixed and played back. Note that two audio streams referenced in the two sub-paths may be mixed and played back, which will be described later.

  Next, a data structure (syntax) for specifically realizing the main path and sub path structures described with reference to FIGS. 6 to 8 will be described.

  FIG. 9 is a diagram illustrating the syntax of PlayList ().

  length is a 32-bit unsigned integer indicating the number of bytes from immediately after this length field to the end of PlayList (). That is, this field indicates the number of bytes from reserved_for_future_use to the end of the PlayList. After this length, 16-bit reserved_for_future_use is prepared. number_of_PlayItems is a 16-bit field indicating the number of PlayItems in the PlayList. For example, in the example of FIG. 6, the number of PlayItems is three. The value of PlayItem_id is assigned from 0 in the order in which PlayItem () appears in the PlayList. For example, as shown in FIGS. 6 and 8, PlayItem_id = 0, 1, and 2 are allocated.

  number_of_SubPaths is a 16-bit field indicating the number of SubPaths (number of entries) in the PlayList. For example, in the example of FIG. 6, the number of Sub Paths is three. The value of SubPath_id is assigned from 0 in the order in which SubPath () appears in the PlayList. For example, as shown in FIG. 6, Subpath_id = 0, 1, and 2 are allocated. In the subsequent for statement, PlayItems are referenced by the number of PlayItems, and Sub Paths are referenced by the number of Sub Paths.

  FIG. 10 is a diagram illustrating the syntax of SubPath ().

  length is a 32-bit unsigned integer indicating the number of bytes from immediately after this length field to the end of Sub Path (). That is, this field indicates the number of bytes from reserved_for_future_use to the end of the PlayList. After this length, 16-bit reserved_for_future_use is prepared. SubPath_type is an 8-bit field indicating the application type of SubPath. SubPath_type is used, for example, when the type indicates whether the Sub Path is audio, bitmap subtitle, text subtitle, or the like. A first example of this SubPath_type will be described later with reference to FIG. 15-bit reserved_for_future_use is prepared after SubPath_type. is_repeat_SubPath is a 1-bit field that designates the SubPath playback method, and indicates whether the SubPath playback is repeated during the main path playback or the SubPath playback is performed only once. For example, when the playback timing of the main AV stream and the stream included in the clip specified by the sub path are different (when the main path is a still image slide show and the sub path audio path is used as the main path BGM (background music), etc.) ). 8-bit reserved_for_future_use is prepared after Is_repeat_SubPath. number_of_SubPlayItems is an 8-bit field indicating the number of SubPlayItems (number of entries) in one SubPath. For example, number_of_SubPlayItems has one SubPlayItem with SubPath_id = 0 in FIG. 6 and two SubPlayItems with SubPath_id = 1. In subsequent for statements, SubPlayItems are referenced by the number of SubPlayItems.

  FIG. 11 is a diagram illustrating a first example of SubPath_type (subpath type). That is, the type of SubPath can be defined as shown in FIG. 11, for example.

  In FIG. 11, SubPath_type = 0, 1 is reserved. SubPath_type = 2 is an Audio presentation path of the Browsable slideshow. For example, SubPath_type = 2 indicates that in the playlist, the audio presentation path referred to by the sub path and the main path referred to by the play item are asynchronous.

  SubPath_type = 3 is an interactive graphics presentation menu (interactive graphics presentation menu). For example, SubPath_type = 3 indicates that in the playlist, the interactive graphics menu referenced by the sub path and the main path referenced by the play item are asynchronous.

  SubPath_type = 4 is a Text subtitle presentation path (a text subtitle presentation path). For example, SubPath_type = 4 indicates that in the playlist, the text subtitle presentation path referenced by the sub path and the main path referenced by the play item are synchronized.

  SubPath_type = 5 is 2nd Audio Presentation path (second audio presentation path) (path for referring to the 2nd audio stream). Specifically, SubPath_type = 5 indicates that in the playlist, the second audio presentation path referred to by the sub path and the main path referred to by the play item are synchronized. For example, the (second) audio stream referenced in this subpath is a director's comment (voice) on the movie. In the case of the Subpath indicated by Subpath_id = 0 in FIG. 8, the SubPath_type in FIG. 10 is SubPath_type = 5.

  SubPath_type = 6 is 2nd Video Presentation path (second video presentation path) (path for referring to the 2nd video stream). Specifically, SubPath_type = 6 indicates that in the playlist, the second video presentation path referred to by the sub path and the main path referred to by the play item are synchronized. For example, the (second) video stream referred to in this sub-path is a director's comment (moving image) for the movie. In the description of the processing of the playback apparatus 1 described with reference to FIGS. 27 and 28, only audio synthesis is described in detail, but moving image synthesis can also be realized with the same configuration. It will be described later.

  SubPath_type = 7 to 255 is reserved.

  FIG. 12 is a diagram illustrating the syntax of SubPlayItem (i).

  The length is a 16-bit unsigned integer indicating the number of bytes from immediately after this length field to the end of Sub playItem ().

  In FIG. 12, it is divided into a case where SubPlayItem refers to one Clip and a case where it refers to a plurality of Clips.

  First, a case where SubPlayItem refers to one Clip will be described.

  SubPlayItem includes Clip_Information_file_name [0] for specifying Clip. Also, Clip_codec_identifier [0] that specifies the codec system of Clip, reserved_for_future_use, is_multi_Clip_entries that indicates the presence / absence of multi-clip registration, and ref_to_STC_id [0] that is information about STC discontinuity points (system time base discontinuity points) including. When the is_multi_Clip_entries flag is set, the syntax when SubPlayItem refers to a plurality of Clips is referenced. In addition, SubPlayItem_IN_time and SubPlayItem_OUT_time for specifying the playback section of the Sub Path in the Clip are included. Furthermore, sync_PlayItem_id and sync_start_PTS_of_PlayItem are included to specify the time when the Sub Path starts to be played on the time axis of the main path. As described above, the sync_PlayItem_id and sync_start_PTS_of_PlayItem are used in the case of FIGS. 7 and 8 (when the playback timing of the stream included in the file indicated by the main AV stream and the sub path is the same), and the main AV stream and the sub path are used. When the playback timings of the streams included in the file indicated by are different (for example, when the still image referenced by the main path and the audio referenced by the sub path are not synchronized, such as BGM of a slide show composed of still images) ) Is not used. Also, SubPlayItem_IN_time, SubPlayItem_OUT_time, sync_PlayItem_id, and sync_start_PTS_of_PlayItem are used in common in the Clip referred to by SubPlayItem.

Next, a case where SubPlayItem refers to a plurality of Clips (if (is_multi_Clip_entries == 1b), that is, multi-clip registration is performed) will be described.
Specifically, as shown in FIG. 8, a case where SubPlayItem refers to a plurality of Clips is shown.

  num_of_Clip_entries indicates the number of Clips, and Clip_Information_file_name [SubClip_entry_id] specifies Clips excluding Clip_Information_file_name [0]. That is, Clips such as Clip_Information_file_name [1] and Clip_Information_file_name [2] are specified excluding Clip_Information_file_name [0]. SubPlayItem also includes Clip_codec_identifier [SubClip_entry_id] that specifies the codec format of Clip, ref_to_STC_id [SubClip_entry_id] that is information related to STC discontinuity points (system time base discontinuity points), and reserved_for_future_use.

  Note that SubPlayItem_IN_time, SubPlayItem_OUT_time, sync_PlayItem_id, and sync_start_PTS_of_PlayItem are used in common among a plurality of clips. In the example of FIG. 8, SubPlayItem_IN_time, SubPlayItem_OUT_time, sync_PlayItem_id, and sync_start_PTS_of_PlayItem are used in common between SubClip_entry_id = 0 and SubClip_entry_id = 1, and the Text based subtitle for the selected SubClip_entry_id is _ Playback is based on SubPlayItem_OUT_time, sync_PlayItem_id, and sync_start_PTS_of_PlayItem.

  Here, the value of SubClip_entry_id is assigned from 1 in the order in which Clip_Information_file_name [SubClip_entry_id] in SubPlayItem appears. Also, SubClip_entry_id of Clip_Information_file_name [0] is 0.

  FIG. 13 is a diagram illustrating the syntax of PlayItem ().

  length is a 16-bit unsigned integer indicating the number of bytes from immediately after this length field to the end of PlayItem (). Clip_Information_file_name [0] is a field for designating a Clip referred to by PlayItem. In the example of FIG. 7, the main Clip AV stream file is referred to by Clip_Information_file_name [0]. In addition, PlayItem () includes Clip_codec_identifier [0] that specifies the codec method of Clip, reserved_for_future_use, is_multi_angle, connection_condition, and ref_to_STC_id [0] that is information on STC discontinuity points (system time base discontinuity points). . Furthermore, PlayItem () includes IN_time and OUT_time for designating the playback section of the PlayItem in the Clip. In the example of FIG. 7, the playback range of the main ClipAV stream file is represented by IN_time and OUT_time. Also, PlayItem () includes UO_mask_table (), PlayItem_random_access_mode, and still_mode. Note that the case where there are a plurality of is_multi_angles is not directly related to the present invention, and thus the description thereof is omitted.

  STN_table () in PlayItem () refers to the PlayItem's operations such as audio switching and subtitle switching when the target PlayItem and one or more SubPaths to be played back are prepared. It provides a mechanism that allows the user to select from Clips and Clips referenced by one or more of these SubPaths.

  FIG. 14 is a diagram illustrating a first example (first STN_table ()) of syntax of STN_table (). The first STN_table () is set as an attribute of PlayItem. Further, the first STN_table () of FIG. 14 provides a mechanism that allows selection of playback of a combination of a clip referenced by PlayItem and a Clip referenced by SubPath.

  length is a 16-bit unsigned integer indicating the number of bytes from immediately after this length field to the end of STN_table (). 16-bit reserved_for_future_use is prepared after the length. number_of_video_stream_entries indicates the number of streams to which video_stream_id entered (registered) in STN_table () is given. video_stream_id is information for identifying a video stream, and video_stream_number is a video stream number that is used for video switching and is visible to the user. number_of_audio_stream_entries indicates the number of streams to which audio_stream_id entered in STN_table () is given. The audio_stream_id is information for identifying the audio stream, and the audio_stream_number is an audio stream number that can be seen by the user used for audio switching.

  number_of_PG_txtST_stream_entries indicates the number of streams to which PG_txtST_stream_id entered in STN_table () is given. In this, a stream (PG, Presentation Graphics stream) obtained by run-length encoding bitmap subtitles such as a DVD sub-picture and a text subtitle file (txtST) are entered. PG_txtST_stream_id is information for identifying a subtitle stream, and PG_txtST_stream_number is a subtitle stream number (text subtitle stream number) that can be seen by the user used for subtitle switching.

  number_of_IG_stream_entries indicates the number of streams provided with IG_stream_id entered in STN_table (). In this, an interactive graphics stream is entered. IG_stream_id is information for identifying an interactive graphics stream, and IG_stream_number is a graphics stream number that is visible to the user used for graphics switching.

  Here, the syntax of stream_entry () will be described with reference to FIG.

  length is an 8-bit unsigned integer indicating the number of bytes from immediately after this length field to the end of stream_entry (). type is an 8-bit field indicating the type of information necessary to uniquely specify the stream to which the above-described stream number is given.

  In type = 1 or type = 3, a 16-bit packet is used to identify one elementary stream from among a plurality of elementary streams multiplexed in a Clip (Main Clip) referenced by PlayItem. An ID (PID) is specified. ref_to_stream_PID_of_mainClip indicates this PID. That is, with type = 1 and type = 3, the stream is determined simply by specifying the PID in the main ClipAV stream file.

  In type = 2 or type = 3, when SubPath refers to a plurality of Clips at a time and each Clip multiplexes a plurality of elementary streams, a plurality of one Clip (SubClip) referenced by SubPath SubPath_id, Clip id, and packet ID (PID) of the SubPath are specified in order to identify one elementary stream from among the elementary streams. ref_to_SubPath_id indicates this SubPath_id, ref_to_SubClip_entry_id indicates this Clip id, and ref_to_stream_PID_of_SubClip indicates this PID. This is used when a plurality of Clips are referred to in SubPlayItem and a plurality of elementary streams are referred to this Clip.

  That is, when type = 3, a packet ID (PID) for specifying one elementary stream from among a plurality of elementary streams multiplexed in a Clip (Main Clip) referred to by PlayItem When SubPath refers to a plurality of Clips at a time and each Clip multiplexes a plurality of elementary streams, one of a plurality of elementary streams of one Clip (SubClip) referenced by SubPath Two streams can be entered with SubPath_id, Clip id, and Packet ID (PID) of SubPath for specifying one elementary stream. For example, two audio streams, an audio stream referenced by the main path and an audio stream referenced by the sub path, can be entered. In the following, the audio stream referred to by the main path is also referred to as a 1st audio stream, and the audio stream referred to by the sub path is also referred to as a 2nd audio stream.

  As described above, when a type (three types of 1 to 3) is used and a PlayItem and one or more SubPaths to be played back are prepared, one Clip and one reference to this PlayItem are provided. One elementary stream can be identified from the Clip referred to by the above SubPath. Note that type = 1 indicates a clip (main clip) referred to by the main path, and types = 2 and 3 indicate clips (sub clips) referred to by the sub path.

When type = 3, the audio stream (1st audio stream) of the main Clip AV stream file referenced in the main path and the audio stream (2nd audio stream) referenced in the sub path can be specified and entered. .
For example, the 1st audio stream and the 2nd audio stream can be combined and reproduced.

  Returning to the description of the first STN_table () in FIG. 14, video_stream_id from 0 to video_stream_id is assigned to one video elementary stream specified for each stream_entry () in order in the for loop of the video stream ID (video_stream_id). Given. Note that a video stream number (video_stream_number) may be used instead of the video stream ID (video_stream_id). In this case, video_stream_number is given from 1 instead of 0. That is, video_stream_number is obtained by adding 1 to the value of video_stream_id. The video stream number is defined from 1 since it is a video stream number that is used for video switching and is visible to the user.

  Similarly, in the for loop of the audio stream ID (audio_stream_id), audio_stream_id is given from 0 to one audio elementary stream that is specified for each stream_entry () in order. As in the case of the video stream, an audio stream number (audio_stream_number) may be used instead of the audio stream ID (audio_stream_id). In this case, audio_stream_number is given from 1 instead of 0. That is, audio_stream_number is obtained by adding 1 to the value of audio_stream_id. The audio stream number is an audio stream number that can be seen by the user and is used for audio switching.

  Similarly, PG_txtST_stream_id is assigned from 0 to one bitmap subtitle elementary stream or text subtitle identified in order for each stream_entry () in the for loop of the subtitle stream ID (PG_txtST_stream_id). As in the case of the video stream, a subtitle stream number (PG_txtST_stream_number) may be used instead of the subtitle stream ID (PG_txtST_stream_id). In this case, PG_txtST_stream_number is given from 1 instead of 0. That is, PG_txtST_stream_number is obtained by adding 1 to the value of PG_txtST_stream_id. The subtitle stream number is a subtitle stream number (text subtitle stream number) that can be seen by the user used for subtitle switching, and is defined from 1.

  Similarly, in the for loop of the graphics stream ID (IG_stream_id), IG_stream_id is assigned from 0 to one interactive graphics elementary stream that is identified for each stream_entry () in order. As in the case of the video stream, a graphics stream number (IG_stream_number) may be used instead of the graphics stream ID (IG_stream_id). In this case, IG_stream_number is given from 1 instead of 0. That is, IG_stream_number is obtained by adding 1 to the value of IG_stream_id. Since the graphics stream number is a graphics stream number that can be seen by the user used for switching graphics, it is defined from 1.

  Next, stream_attribute () of the first STN_table () in FIG. 14 will be described. In this stream_attribute, attribute information of the stream specified by stream_entry () is recorded.

  The stream_attribute () in the for loop of the video stream ID (video_stream_id) gives stream attribute information of one video elementary stream specified for each stream_entry (). That is, in this stream_attribute (), stream attribute information of one video elementary stream specified for each stream_entry () is described.

  The stream_attribute () in the for loop of the audio stream ID (audio_stream_id) gives stream attribute information of at least one or more audio elementary streams specified for each stream_entry (). That is, in this stream_attribute (), stream attribute information of at least one or more audio elementary streams specified for each stream_entry () is described. Specifically, when type = 1 or type = 2 in stream_entry () in FIG. 15, there is one audio elementary stream specified by this stream_entry (), so stream_attribute () Provides stream attribute information of the audio elementary stream. Here, when type = 3 in stream_entry () in FIG. 15, there are two audio elementary streams specified by stream_entry () (1st audio stream referenced in the main path and 2nd audio referenced in the sub path). Stream_attribute () gives stream attribute information of two audio elementary streams.

  As described above, when type = 3, that is, when referring to the audio stream in both the main path and the sub path, two stream_attributes can be inserted. That is, in the audio stream of the for statement, when stream = entry () is type = 3, two stream_attributes () are entered in the subsequent stream_attributes ().

  That is, in the first STN_table () of FIG. 14, stream_attributes () is defined for each stream_entry () of the audio stream ID (audio_stream_id). When type = 3, for stream_entry (), Two stream_attributes () are given. As a result, two stream_attributes () referenced by the main path and the sub path are entered for one audio_stream_id, so that two audio streams to be reproduced simultaneously can be entered.

  The stream_attribute () in the for loop of the subtitle stream ID (PG_txtST_stream_id) gives stream attribute information of one bitmap subtitle elementary stream or text subtitle elementary stream specified for each stream_entry (). That is, in this stream_attribute (), stream attribute information of one bitmap subtitle elementary stream specified for each stream_entry () is described.

  Similarly, stream_attribute () in the for loop of the graphics stream ID (IG_stream_id) gives stream attribute information of one interactive graphics elementary stream specified for each stream_entry (). That is, stream attribute information of one interactive graphics elementary stream specified for each stream_entry () is described in the stream_attribute ().

  In the example of FIGS. 14 and 15, the case where two audio streams of the audio stream referred to by the main path and the audio stream referred to by the sub path are entered has been described. However, reference is made to the two sub paths. FIGS. 16 and 17 show examples of the syntax of the second STN_table () and the syntax of the second Stream_entry () that allow entry of two audio streams. In the figure, portions corresponding to those in FIG. 14 and FIG. 15 are repeated, and the description thereof is omitted.

  FIG. 16 is a diagram illustrating the syntax of the second STN_table (). The second STN_table () is set as an attribute of PlayItem. FIG. 17 is a diagram illustrating the syntax of the second Stream_entry ().

  In type = 1 in FIG. 17, a 16-bit packet ID is used to identify one elementary stream from among a plurality of elementary streams multiplexed in a Clip (Main Clip) referred to by PlayItem. (PID) is specified. ref_to_stream_PID_of_mainClip indicates this PID. That is, with type = 1, the stream is determined simply by specifying the PID in the main ClipAV stream file.

  In type = 2, when SubPath refers to multiple Clips at a time and each Clip multiplexes multiple elementary streams, multiple elementary streams of one Clip (SubClip) referenced by SubPath SubPath_id, Clip id, and packet ID (PID) of the SubPath are specified in order to identify one elementary stream from the list. ref_to_SubPath_id indicates this SubPath_id, ref_to_SubClip_entry_id indicates this Clip id, and ref_to_stream_PID_of_SubClip indicates this PID. This is used when a plurality of Clips are referred to in SubPlayItem and a plurality of elementary streams are referred to this Clip.

  As described above, when a type (two types of type 1 and type 2) is used, when a PlayItem and one or more SubPaths to be played back are prepared, one Clip and a reference to this PlayItem are provided. One elementary stream can be identified from the Clip referred to by the above SubPath. Note that type = 1 indicates a clip (main clip) referred to by the main path, and type = 2 indicates a clip (sub clip) referred to by the sub path.

  Returning to the description of the second STN_table () in FIG. 16, audio_stream_id is given from 0 to the audio elementary stream specified for each stream_entry () in turn in the for loop of the audio stream ID (audio_stream_id). . Here, unlike FIG. 14, the second audio after the Stream_entry () and Stream_attributes () for the first audio stream (decoded by the first audio decoder 75-1 of the playback apparatus in FIG. 28 described later). A secondary_audio_present_flag indicating whether or not to define a stream is provided. When defining the second audio stream (Secondary Audio Stream), the value of this secondary_audio_present_flag is set to 1, and the second audio stream is specified by Stream_entry () and Stream_attributes () in the if statement described thereafter. A stream is defined. When Secondary_audio_present_flag is 1, since two stream_entry () are defined in the for loop of the audio stream ID (audio_stream_id), the two audio elementary streams specified by each of the two stream_entry () , Audio_stream_id is given from 0. As in the case of the video stream, an audio stream number (audio_stream_number) may be used instead of the audio stream ID (audio_stream_id). In this case, audio_stream_number is given from 1 instead of 0. That is, audio_stream_number is obtained by adding 1 to the value of audio_stream_id. The audio stream number is an audio stream number that can be seen by the user and is used for audio switching.

  In some cases, the Secondary Audio Stream is not entered (that is, the second stream_entry () is not entered). In this case, the secondary_audio_present_flag is set to 0, and only the first audio stream (Main Audio Stream) is stored. Entered. That is, when there is no second audio stream, the second Stream_entry () is not entered. In this case, mixing reproduction of the two audios is not performed.

  In the for loop of the audio stream ID (audio_stream_id), two combinations of stream_entry () and stream_attribute () are described. The first combination of stream_entry () and stream_attribute () is a definition for an audio stream that is decoded by the first audio decoder 75-1 of the playback apparatus 1 shown in FIG. 28 to be described later, and is an entry when secondary_audio_present_flag is 1. The combination of the second stream_entry () and stream_attribute () is a definition for an audio stream that is decoded by the 2nd audio decoder 75-2 of the playback device 1 shown in FIG. 28 described later. That is, two audio streams can be defined.

  The stream_attribute () in the for loop of the audio stream ID (audio_stream_id) gives stream attribute information of at least one or more audio elementary streams specified for each stream_entry (). That is, in this stream_attribute (), stream attribute information of at least one or more audio elementary streams specified for each stream_entry () is described. Specifically, when secondary_audio_present_flag in FIG. 16 is 1, since there are two audio elementary streams specified by two stream_entry (), stream_attribute () is a stream attribute of the corresponding audio elementary stream. Give information. That is, an audio stream (Main Audio Stream) to be decoded by the 1st audio decoder 75-1 of the playback apparatus 1 shown in FIG. 28 described later is defined by the first stream_entry () in the for loop of the audio stream ID (audio_stream_id). The stream attribute information of the audio elementary stream specified by this is given by the first stream_attribute () in the for loop of the audio stream ID (audio_stream_id). Similarly, when the secondary_audio_present_flag is 1, the 2nd audio decoder 75-2 of the playback apparatus 1 in FIG. 28 to be described later is used by the next (second) stream_entry () in the for loop of the audio stream ID (audio_stream_id). The audio stream (Secondary Audio Stream) to be decoded is defined, and the stream attribute information of the audio elementary stream specified by this is given by the second stream_attribute () in the for loop of the audio stream ID (audio_stream_id) It is done.

  For example, the audio stream defined by the first stream_entry () in the for loop of the audio stream ID (audio_stream_id) is, for example, a Japanese dubbing voice or English voice of a movie, and the audio stream ID (audio_stream_id) When the secondary_audio_present_flag in the for loop is 1, the audio stream defined by the second stream_entry () is assumed to be, for example, the audio for the director's cut of a movie, and is the AV stream file referenced by the Main Path It may be used when a comment such as a movie director is included only in a predetermined part.

  That is, when two audio streams are synthesized, 1 is set in secondary_audio_present_flag in FIG. 16 which is a flag indicating that there is an audio stream to be synthesized with the primary (1st) audio stream, and the audio to be synthesized. The stream_entry () and stream_attribute () of the stream are referred to, and the audio stream is synthesized.

  Specifically, one audio stream to be synthesized is an audio stream referred to by the main path (Main Audio Stream decoded by the first audio decoder 75-1 of the playback apparatus 1 in FIG. 28 described later), and the other is referred to by the sub path. 16 (secondary audio stream decoded by the 2nd audio decoder 75-2 of the playback apparatus 1 in FIG. 28 described later), the type value of the first stream_entry () in FIG. The stream attribute information of the audio elementary stream specified thereby is given by stream_attribute (), secondary_audio_present_flag in FIG. 16 is set to 1, and the value of type of the second stream_entry () is set to type = 2, The stream attribute information of the audio elementary stream specified by the stream_attribute () Given.

  Also, if both of the audio streams to be synthesized are audio streams that are referenced in the sub-path, the type value of the first stream_entry () in the for loop of the audio stream ID (audio_stream_id) in FIG. = 2 and stream attribute information of the audio elementary stream specified thereby is given by stream_attribute (), secondary_audio_present_flag in the for loop of the audio stream ID (audio_stream_id) in Fig. 16 is set to 1, and the second The value of type of stream_entry () is set to type = 2, and stream attribute information of the audio elementary stream specified thereby is given by stream_attribute (). That is, when two audio streams to be combined are audio streams that are referred to by sub-paths, the type of stream_entry () in FIG.

  As described above, since the audio stream (or video stream) is added in the form specified by the sub-path, the audio stream (or video stream) on the disc and the audio obtained by downloading or the local storage are stored. Streams (or video streams) can be played back in synchronization with each other, and audio streams (or video streams) acquired together by downloading or the like are synchronized with a video stream (or audio stream) on a disk or acquired. Can be played.

If the PlayList is also acquired, the acquired video stream (or audio stream) specified by the main path and the two acquired audio streams (or video streams) specified by the sub path are played back in synchronization. You can also

  When playing back only the audio referenced in the main path or playing back only the audio specified in one sub-path, 0 is set to secondary_audio_present_flag in FIG. 16, and the first stream_entry () is used in the main path. When reference is made, type = 1 is set, and when only the audio specified by the sub path is reproduced, type = 2 is set, and stream attribute information of the audio elementary stream specified thereby is given by stream_attribute (). That is, it is possible to reproduce only one audio stream without mixing.

  As described above, in FIG. 16, it is assumed that stream_entry () and stream_attributes can be inserted for two audio elementary streams. That is, in the audio stream of the for statement, corresponding stream_attributes () can be entered using two stream_entry ().

  That is, in the second STN_table () of FIG. 16, corresponding stream_attributes () is defined by two stream_entry () of the audio stream ID (audio_stream_id). As a result, a combination of two stream_entry () and stream_attributes () that are referenced by the main path and the sub path, or referenced by the two sub paths, is entered for one audio_stream_id, so that playback is performed simultaneously. Two audio streams can be entered.

  Note that the video stream ID (video_stream_id), subtitle stream ID (PG_txtST_stream_id), and graphics stream ID (IG_stream_id) in FIG. 16 are the same as those in FIG.

  Next, the syntax of stream_attribute () will be described with reference to FIG.

  The length is a 16-bit unsigned integer indicating the number of bytes from immediately after this length field to the end of stream_attribute ().

  The stream_coding_type indicates the encoding type of the elementary stream as shown in FIG. As encoding types of elementary streams, MPEG-2 video stream, HDMV LPCM audio, Dolby AC-3 audio, dts audio, Presentation graphics stream, Interactive graphics stream, and Text subtitle stream are described. Note that this stream_coding_type may be extended so that a video stream compressed by another codec such as MPEG-4 AVC or VC-1 can be handled.

  video_format indicates the video format of the video elementary stream as shown in FIG. 480i, 576i, 480p, 1080i, 720p, and 1080p are described as the video format of the video elementary stream.

  frame_rate indicates the frame rate of the video elementary stream, as shown in FIG. As frame rates of the video elementary stream, 24000/1001, 24, 25, 30000/1001, 50, and 60000/1001 are described.

  The aspect_ratio indicates the aspect ratio information of the video elementary stream as shown in FIG. As aspect ratio information of the video elementary stream, 4: 3 display aspect ratio and 16: 9 display aspect ratio are described.

  audio_presentation_type indicates presentation type information of an audio elementary stream as shown in FIG. As the presentation type information of the audio elementary stream, single mono channel, dual mono channel, stereo (2-channel), and multi-channel are described.

  Sampling_frequency indicates the sampling frequency of the audio elementary stream as shown in FIG. As sampling frequencies of audio elementary streams, 48 kHz and 96 kHz are described.

  audio_language_code indicates the language code of the audio elementary stream (Japanese, Korean, Chinese, etc.).

  PG_language_code indicates the language code (Japanese, Korean, Chinese, etc.) of the bitmap subtitle elementary stream.

  IG_language_code, which indicates the language code (Japanese, Korean, Chinese, etc.) of the interactive graphics elementary stream.

  textST_language_code indicates the language code (Japanese, Korean, Chinese, etc.) of the text subtitle elementary stream.

  Character_code indicates the character code of the text subtitle elementary stream as shown in FIG. The character code of the text subtitle elementary stream includes Unicode V1.1 (ISO 10646-1), Shift JIS (Japanese), KSC 5601-1987 including KSC 5653 for Roman character (Korean), GB 18030-2000 (Chinese), GB2312 (Chinese) and BIG5 (Chinese) are described.

  A specific example of the syntax of stream_attribute () in FIG. 18 will be described below with reference to FIGS. 18 and 19 to 25.

  When the encoding type of the elementary stream (stream_coding_type in FIG. 18) is an MPEG-2 video stream (FIG. 19), the stream_attribute () includes the video format (FIG. 20) and frame rate (FIG. 21) of the elementary stream. ), And aspect ratio information (FIG. 22).

  When the encoding type of the elementary stream (stream_coding_type in FIG. 18) is HDMV LPCM audio, Dolby AC-3 audio, or dts audio (FIG. 19), the stream_attribute () includes the presentation type information of the audio elementary stream. (FIG. 23), sampling frequency (FIG. 24), and language code are included.

  When the encoding type of the elementary stream (stream_coding_type in FIG. 18) is a Presentation graphics stream (FIG. 19), stream_attribute () includes the language code of the bitmap subtitle elementary stream.

  When the encoding type of the elementary stream (stream_coding_type in FIG. 18) is Interactive graphics stream (FIG. 19), stream_attribute () includes the language code of the interactive graphics elementary stream.

  When the encoding type of the elementary stream (stream_coding_type in FIG. 18) is a Text subtitle stream (FIG. 19), stream_attribute () includes the character code (FIG. 25) and language code of the text subtitle elementary stream. .

  Note that the attribute information is not limited to this.

  As described above, when a PlayItem and one or more SubPaths to be played back are prepared, a stream_entry () is used to select a Clip referred to by the PlayItem and a Clip referred to by one or more SubPaths. The attribute information of the identified elementary stream (one if type = 1 and type = 2, two if type = 3) can be known by stream_attribute ().

By examining this attribute information (stream_attribute ()), the playback device can check whether or not it has the function of playing back the elementary stream.
Further, the playback device can select the elementary stream corresponding to the initial information of the language setting of the playback device by examining the attribute information.

  For example, it is assumed that the playback apparatus has only a playback function for a bitmap subtitle elementary stream and does not have a playback function for a text subtitle elementary stream. When the user instructs the playback apparatus to switch the language, the playback apparatus sequentially selects and reproduces only the bitmap subtitle elementary stream from the for loop of the subtitle stream ID (PG_txtST_stream_id).

  Also, for example, assume that the initial information of the language setting of the playback device is Japanese. When the user instructs the playback device to switch to the secondary audio (ie, secondary audio) or to mix and play the main audio and the secondary audio, the playback device uses the audio stream ID ( Only audio elementary streams whose language code is Japanese are sequentially selected from the for loop of (Audio stream id) and played.

  Further, for example, when an AV stream (movie) composed of a video stream and an audio stream, which is referred to by the main path, is played back, the user instructs the playback apparatus to switch the audio, and the audio stream referenced by the sub path When mixing playback with (comment by a director or a performer) is instructed, the playback device further mixes (superimposes) the 2nd audio stream referenced by the sub path on the 1st audio stream referenced by the main path. Play with the video stream.

  Also, for example, when the user commands switching of audio and mixing playback of two audio streams referenced by two subpaths is commanded to the playback device (the syntaxes of FIGS. 16 and 17 are used, FIG. 16) when secondary audio reproduction is instructed to reproduce an audio stream corresponding to an audio stream ID whose type is 2 and the type is 2 in FIG. The audio stream is mixed (superposed) and reproduced together with the video stream.

  In this way, by providing STN_table () in PlayItem (), when PlayItem and one or more SubPaths to be played back are prepared, operations such as audio switching and subtitle switching by the user Provides a mechanism for selecting a stream to be played back from the Clip referenced by this PlayItem and the Clip referenced by one or more SubPaths, so that a stream or data file different from the AV stream to be played back can be selected. Even interactive operations can be performed.

  Also, since a plurality of SubPaths are used in one PlayList and each SubPath refers to a SubPlayItem, an AV stream with high expandability and high flexibility can be realized. That is, a configuration in which SubPlayItems can be added later in the content played back by this PlayList. For example, when there is a ClipAV stream file referenced by the Main Path and a PlayList associated with the Clip AV stream file, and the PlayList is rewritten to a PlayList with a new Sub Path added, the Main Path refers to the PlayList based on the new PlayList. Along with the ClipAV stream file, playback can be performed with reference to a ClipAV stream file different from the ClipAV stream file referred to by the Main Path. Thus, it can be set as the structure which has an expandability.

  Furthermore, STN_table () in PlayItem () mixes the audio stream of the Clip referenced by the two SubPaths when this PlayItem and one or more SubPaths to be played back are prepared. Since a mechanism capable of playing back is provided (FIGS. 16 and 17), an audio stream (for example, Japanese dubbing sound of a movie) decoded by the 1st audio decoder 75-1 and a 2nd audio decoder 75- An interactive operation can also be performed for superimposed reproduction with an audio stream (for example, a director's director's cut) decoded in step 2.

  In addition, when STN_table () is provided in PlayItem (), when the PlayItem and one or more SubPaths to be played back are prepared, the first audio stream of the Clip referred to by this PlayItem and Since a mechanism capable of mixing and playing back the second audio stream of the clip referred to by SubPath is provided, the user can select an audio stream different from the main clip AV stream (a second audio stream, for example, a director's comment stream). ) Can also be interactively operated.

  A specific example will be described with reference to FIG. FIG. 26 is a diagram illustrating an example of a stream number table representing a relationship between an audio signal and a caption signal provided to a user. FIG. 26 shows an example corresponding to the second STN_table and the second Stream_entry () shown in FIGS. 16 and 17.

  In FIG. 26, the audio number is referred to as A_SN (Audio Stream Number), and the subtitle number is referred to as S_SN (SubPicture Stream Number). In FIG. 26, A_SN is assigned to each of a plurality of audio streams of the main ClipAV stream referred to by the Main Path of the PlayList and an audio stream of the sub ClipAV stream referred to by the Sub Path. That is, A_SN = 1 is given an audio 2 that is an audio stream referenced by the Main Path, A_SN = 2 is given an audio 1 that is an audio stream referenced by the Sub Path, and A_SN = 3 is referenced by the Main Path. Audio 3 which is an audio stream to be played is given.

  Also, A_SN is given so that the audio stream referenced by the SubPath of the PlayList can be mixed and reproduced with each of the audio streams given by A_SN = 1 to 3. That is, audio 4 that is an audio stream referenced by audio 2 + Sub Path is given to A_SN = 4, and audio 4 that is an audio stream referenced by Audio_1 + Sub Path is referenced by A_SN = 5. Is given. That is, when A_SN = 4, A_SN for mixing reproduction of the audio stream referenced by the main path and the audio stream referenced by the sub path is given, and when A_SN = 5, two audios referenced by the two sub paths are given. A_SN is provided for mixing reproduction of the stream.

  In addition, S_SN is assigned to each of a plurality of sub-picture streams of the main Clip AV stream referred to by the main path of the PlayList. That is, sub-picture 3 is given to S_SN = 1, sub-picture 1 is given to S_SN = 2, and sub-picture 2 is given to S_SN = 3. Here, the smaller the A_SN or S_SN number, the higher the priority of the audio signal or sub-picture provided to the user. That is, A_SN = 1 is an audio stream reproduced by default, and S_SN = 1 is a sub-picture stream reproduced by default.

  Specifically, the audio reproduced based on the initial information of the language setting of the reproduction apparatus corresponds to audio 2 (FIG. 26) with A_SN = 1, and the audio reproduced after the audio is switched. Corresponds to audio 1 (FIG. 26) with A_SN = 2.

  In order to provide such a stream number table, first, in the STN table () in PlayItem () referenced by PlayList (), “Audio 1 or Audio 3” and “Sub” in the main path of PlayItem “Picture 1, sub-picture 2 or sub-picture 3” can be referred to, and further, “audio 2 or audio 4” is reproduced in a sub-path (SubPath () referenced in PlayList) associated with PlayItem. A configuration that can be referred to. Then, audio 2 (type = 1) is given by audio_stream_id = 0 (A_SN = 1) in STN table () (FIG. 16), audio 1 (type = 2) is given by audio_stream_id = 1 (A_SN = 2), and audio_stream_id = 2 (A_SN = 3) gives audio 3 (type = 1), audio_stream_id = 3 (A_SN = 4) gives audio 2 and audio 4 (type = 1 and type = 2), and audio_stream_id = 4 (A_SN = 5) gives audio 1 and audio 4 (type = 2 and type = 2).

  That is, in the case of audio_stream_id = 3 (A_SN = 4), the audio stream (type = 1) referred to by the main path is registered by the first stream_entry () and stream_attributes () corresponding to the audio stream ID in FIG. The subsequent secondary_audio_present_flag is set to 1, and the audio stream (type = 2) referred to by the subpath is registered by the second stream_entry () and stream_attributes (). At this time, the audio_stream_id can be used to define a combination of an audio stream referenced by the main path and an audio stream referenced by the sub path. That is, since it is not necessary to define inconsistent combinations, the user can always select a combination of audio streams that can be reproduced. In other words, in the example of FIG. 26, only the selectable items can be provided to the user by not defining the audio 2 + audio 3 that is a combination of contradictory audio streams.

  In this way, in the stream_entry () (FIG. 17) of the STN table () (FIG. 16) in PlayItem (), the 1st audio stream referenced in the main path and the 2nd audio stream referenced in the sub path are entered. (Registration) is configured (secondary_audio_present_flag is provided), and further, the audio_stream_id is used to provide a combination of the 1st audio stream and the 2nd audio stream, so the two audio streams are mixed and played back can do. That is, streams of the same type (in this example, an audio stream) can be mixed (superimposed or combined) from a plurality of types of streams and played back simultaneously.

When audio_stream_id = 4 (A_SN = 5), the first stream_entry () and stream_attributes () and secondary_audio_present_flag corresponding to the audio stream ID in FIG. 16 and the second stream_entry () and stream_attributes () Two audio streams (both type = 2) referenced by one sub-path are registered.
At this time, a combination of two audio streams referred to by the two subpaths can be defined using audio_stream_id. That is, since it is not necessary to define inconsistent combinations, the user can always select a combination of audio streams that can be reproduced.

  In this way, it is possible to enter (register) two audio streams referenced by two sub-paths with stream_entry () (FIG. 17) of STN table () (FIG. 16) in PlayItem (). In addition, since the audio_stream_id is used to provide a combination of two audio streams, the two audio streams can be mixed and reproduced. That is, streams of the same type (in this example, an audio stream) can be mixed (superimposed or combined) from a plurality of types of streams and played back simultaneously.

  Further, it is not necessary to individually select two audio streams to be mixed, and an audio stream to be reproduced can be selected from a combination of two audio streams.

  In the syntaxes of FIGS. 14 and 15, the combination (audio_stream_id = 3 (A_SN = 4)) of the 1st audio stream referenced in the main path and the 2nd audio stream referenced in the sub path in FIG. 26 can be defined. is there. In this case, the audio stream referred to by the main path and the audio stream referred to by the sub path are registered in stream_entry () in FIG. 14 and two stream_attributes () are also inserted accordingly. In this way, in the stream_entry () (FIG. 15) of the STN table () (FIG. 14) in PlayItem (), the 1st audio stream referenced in the main path and the 2nd audio stream referenced in the sub path are entered. (Registration) is configured (type = 3 is provided), and the audio_stream_id is used to provide a combination of the 1st audio stream and the 2nd audio stream, so that the two audio streams are mixed. Can be played. That is, streams of the same type (in this example, an audio stream) can be mixed (superimposed or combined) from a plurality of types of streams and played back simultaneously.

  Next, a first configuration example of a playback apparatus to which the present invention is applied will be described. FIG. 27 is a block diagram showing a configuration example of the playback apparatus 1 to which the present invention is applied. This playback apparatus 1 is a playback apparatus 1 that plays back a PlayList having the main path and sub path described above.

  The controller 21 executes a control program prepared in advance or controls the storage drive 22 so as to be recorded on the recording medium 11 such as an optical disk or recorded on the local storage 24. By reading out the navigation program, developing it in the memory 23 and executing it, the overall operation of the playback device 1 is controlled. For example, when the recording medium 11 is loaded, the controller 21 can display a predetermined menu screen on an external display device.

  The storage drive 22 reads data from the recording medium 11 according to control by the controller 21, and outputs the read data to the controller 21, the memory 23, or the AV decoder unit 26. When the information read from the recording medium 11 is a navigation program or a PlayList, the information read by the storage drive 22 is output to the controller 21 or the memory 23. When the information read from the recording medium 11 is an AV stream or text data, the information read by the storage drive 22 is output to the AV decoder unit 26.

  The memory 23 appropriately stores data necessary for the controller 21 to execute various processes. Furthermore, various registers in which information necessary for the reproduction process of the reproduction apparatus 1 is set are provided in a part of the storage area of the memory 23. The local storage 24 is composed of, for example, an HDD (Hard Disk Drive).

  The Internet interface 25 is connected to the network 2 by wire or wirelessly, communicates with the server 3 via the network 2 according to control from the controller 21, and downloads data downloaded from the server 3 to the local storage. 24. From the server 3, for example, data for updating the data described with reference to FIG. 5 recorded in the recording medium 11 mounted in the playback device 1 at that time is downloaded as content. The local storage 24 can record content downloaded from the server 3 via the network 2.

  The AV decoder unit 26 decodes the AV stream or text data supplied from the storage drive 22 or the local storage 24, and outputs the obtained video signal and audio signal to an external display device. In the display device, based on the signal decoded by the AV decoder unit 26, for example, the content recorded on the recording medium 11 (video display, audio output) is output.

  The operation input unit 29 includes, for example, an input device such as a button, a key, a touch panel, a jog dial, and a mouse, and a receiving unit that receives a signal such as an infrared ray transmitted from a predetermined remote commander, and acquires a user operation input. To the controller 21.

  In addition, a drive 27 is connected to the controller 21 as necessary. For example, a magnetic disk (including a flexible disk), an optical disk (CD-ROM (Compact Disk-Read Only Memory), DVD) A removable medium 28 including a magneto-optical disk (including MD (registered trademark) (Mini-Disk)), a semiconductor memory, or the like.

  FIG. 28 is a block diagram illustrating a configuration example of the AV decoder unit 26 of the playback device 1 of FIG.

  In the case of the example of FIG. 28, first, the controller 21 reads a PlayList file from the storage drive 22 or the local storage 24, and the HDD, Blu-ray Disc (trademark) via the storage drive 22 based on the information of the PlayList file. Or the AV stream or AV data is read from the recording medium 11 such as a DVD or from the local storage 24. The user can use the operation input unit 29 to instruct the controller 21 to switch between audio and subtitles. Further, the controller 21 reads various setting information including initial information of language setting of the playback device 1 registered in the register of the memory 23.

  The PlayList file includes STN_table () in addition to Main Path and Sub Path information. The controller 21 records the main Clip AV stream file referenced by the PlayItem included in the PlayList file, the sub Clip AV stream file referenced by the SubPlayItem, and the text subtitle data referenced by the SubPlayItem via the storage drive 22 or the local Read from the storage 24. Here, the main Clip AV stream file referred to by PlayItem and the sub Clip AV stream file referred to by SubPlayItem may be recorded on different recording media. For example, the main Clip AV stream file is recorded on the recording medium 11, and the corresponding sub Clip AV stream file is supplied via the network 2 (not shown) and stored in the local storage 24 such as an HDD. Also good. In addition, the controller 21 refers to various setting information registered in the register of the memory 23, selects the elementary stream corresponding to the playback function of itself (playback apparatus 1), and controls to play back, The elementary stream corresponding to the initial information of the language setting of the playback device 1 is selected and controlled to be played back.

  The AV decoder unit 26 includes buffers 51 to 54, PID filter 55, PID filter 56, switches 57 to 59, background decoder 71, MPEG (Moving Picture Experts Group) 2 video decoder 72, presentation graphics decoder 73, interactive graphics. Decoder 74, 1st audio decoder 75-1, 2nd audio decoder 75-2, Text-ST composition 76, switch 77, background plane generator 91, video plane generator 92, presentation graphics plane generator 93, interactive A graphics plane generation unit 94, a buffer 95, a video data processing unit 96, a mixing processing unit 97, and a mixing processing unit 98 are provided. Both the 1st audio decoder 75-1 and the 2nd audio decoder 75-2 decode the audio stream. The 1st audio decoder 75-1 is an audio decoder used preferentially over the 2nd audio decoder 75-2. That is, when there is only one audio stream to be decoded, the first audio decoder is used regardless of whether the audio stream is an audio stream included in the main Clip AV stream file or an audio stream included in the sub Clip AV stream file. Decoded at 75-1. That is, the playback device 1 has two audio decoders (1st audio decoder 75-1 and 2nd audio decoder 75-2) in order to decode two audio streams. In the following, the 1st audio decoder 75-1 and the 2nd audio decoder 75-2 are referred to as audio decoders 75 when not distinguished from each other.

  The file data read by the controller 21 is demodulated by a demodulation and ECC decoding unit (not shown), and error correction is performed on the demodulated multiplexed stream. The switch 32 selects the demodulated and error-corrected data for each stream type based on the control from the controller 21 and supplies the selected data to the corresponding buffers 51 to 54. Specifically, the switch 32 supplies background image data to the buffer 51, supplies data of the main Clip AV stream file to the buffer 52, and sub-clip data to the buffer 53 based on control from the controller 21. The switch 32 is switched so that the Text-ST data is supplied to the buffer 54. The buffer 51 buffers the background image data, the buffer 52 buffers the data of the main Clip AV stream file, the buffer 53 buffers the data of the sub Clip AV stream file, and the buffer 54 -Buffer ST data.

  The main Clip AV stream file is a stream (for example, a transport stream) obtained by multiplexing one or more streams in addition to video among video, audio, bitmap subtitle (Presentation Graphics stream), and interactive graphics. The sub Clip AV stream file is a stream obtained by multiplexing one or more streams of audio, bitmap subtitle (Presentation Graphics stream), interactive graphics, and audio. Note that the data of the text subtitle data file (Text-ST) may or may not be in the form of a multiplexed stream such as a transport stream.

  Further, when the main Clip AV stream file, the sub Clip AV stream file, and the text subtitle data are read from the storage drive 22 (recording medium 11), the respective files may be alternately read in a time division manner, or the sub Before the clip AV stream file and text subtitle data are read from the main clip, all may be preloaded into the buffer (buffer 53 or buffer 54).

  The playback device 1 reads the data of these files from the recording medium 11 via the storage drive 22 and plays back video, bitmap subtitles, interactive graphics, and audio.

  Specifically, the stream data read from the buffer 52, which is the main Clip AV stream file read buffer, is output to the PID (packet ID) filter 55 in the subsequent stage at a predetermined timing. The PID filter 55 distributes the input main Clip AV stream file to the decoders of the elementary streams in the subsequent stage according to the PID (packet ID) and outputs the result. That is, the PID filter 55 supplies the video stream to the video decoder 72, supplies the presentation graphics stream to the switch 57 serving as the supply source to the presentation graphics decoder 73, and supplies the interactive graphics stream to the interactive graphics decoder 74. The audio stream is supplied to the switch 59 serving as the supply source to the 1st audio decoder 75-1.

  The presentation graphics stream is, for example, bitmap subtitle data, and the text subtitle data is, for example, text subtitle data.

  The stream data read from the buffer 53, which is the sub-Clip AV stream file read buffer, is output to the PID (packet ID) filter 56 at the subsequent stage at a predetermined timing. The PID filter 56 distributes the input sub Clip AV stream file to the decoders of the elementary streams in the subsequent stage according to the PID (packet ID) and outputs the result. That is, the PID filter 56 supplies the presentation graphics stream to the switch 57 serving as the supply source to the presentation graphics decoder 73 and supplies the interactive graphics stream to the switch 58 serving as the supply source to the interactive graphics decoder 74. Then, the audio stream of the main clip and the audio stream of the sub clip AV stream file are supplied to the switch 59 serving as the supply source to the first audio decoder 75-1 and the second audio decoder 75-2.

  Data read from the buffer 51 that buffers background image data is supplied to the background decoder 71 at a predetermined timing. The background decoder 71 decodes the background image data and supplies the decoded background image data to the background plane generation unit 91.

  The video stream distributed by the PID filter 55 is supplied to the video decoder 72 at the subsequent stage. The video decoder 72 decodes the video stream and outputs the decoded video data to the video plane generation unit 92.

  The switch 57 selects and selects one of the presentation graphics stream included in the main Clip AV stream file supplied from the PID filter 55 and the presentation graphics stream included in the sub Clip AV stream file. The presentation graphics stream is supplied to the presentation graphics decoder 73 at the subsequent stage. The presentation graphics decoder 73 decodes the presentation graphics stream, and supplies the decoded presentation graphics stream data to the switch 77 serving as a supply source to the presentation graphics plane generation unit 93.

  In addition, the switch 58 selects one of the interactive graphics stream included in the main Clip AV stream file supplied from the PID filter 55 and the interactive graphics stream included in the sub Clip AV stream file, The selected interactive graphics stream is supplied to the subsequent interactive graphics stream decoder 74. That is, the interactive graphics stream input to the interactive graphics decoder 74 at the same time is a stream separated from either the main Clip AV stream file or the sub Clip AV stream file. The interactive graphics decoder 74 decodes the interactive graphics stream and supplies the decoded interactive graphics stream data to the interactive graphics plane generating unit 94.

  Further, the switch 59 selects one of the audio stream included in the main Clip AV stream file supplied from the PID filter 55 and the audio stream included in the sub Clip AV stream file, and selects the selected audio stream. Are supplied to the 1st audio decoder 75-1 or the 2nd audio decoder 75-2 in the subsequent stage. Here, the audio stream simultaneously input to the 1st audio decoder 75-1 is a stream separated from either the main Clip AV stream file or the sub Clip AV stream file. The audio stream that is simultaneously input to the 2nd audio decoder 75-2 is a stream separated from the sub clip.

  Specifically, the switch 59 switches the selection according to the value of the type of stream_entry () in FIG. 15 or FIG. For example, when type = 1, the switch 59 switches the selection so that the audio stream of the main Clip AV stream file supplied via the PID filter 55 is supplied to the 1st audio decoder 75-1. For example, when type = 2, the switch 59 supplies the audio stream of the sub Clip AV stream file supplied through the PID filter 56 to the 1st audio decoder 75-1 or the 2nd audio decoder 75-2. Switch the selection.

  For example, in the case where mixing playback of the audio stream of the main Clip AV stream file and the audio stream of the sub Clip AV stream file is instructed (that is, type = in Stream_entry () of the first STN_table described using FIG. 3. In Stream_entry () of the second STN_table described with reference to FIG. 16, secondary_audio_present_flag is 1, type = 1 in the first Stream_entry (), and type = 2 in the second Stream_entry (). When playback is instructed), the switch 59 switches the selection so that the audio stream of the main Clip AV stream file supplied via the PID filter 55 is supplied to the 1st audio decoder 75-1, and also via the PID filter 56. 2nd audio stream of the sub clip AV stream file supplied Toggle selection to supply the Odekoda 75-2.

  Also, for example, when the reproduction reproduction of the audio stream of two sub Clip AV stream files is instructed (secondary_audio_present_flag in FIG. 16 is 1, type = 2 in the first Stream_entry (), type = in the second Stream_entry () 2), the audio stream of the sub Clip AV stream file supplied via the PID filter 56 is an audio stream ID (audio_stream_id) of the stream_entry () in FIG. The sub clip AV stream file supplied via the PID filter 56 is switched to be supplied to the first audio decoder 75-1 when corresponding to the audio stream defined by the first stream_entry () in the for loop. Is an audio stream of stream_entry () in FIG. If corresponding to the audio stream defined by the over arm ID (audio_stream_id) of for 2 th stream_entry in the loop (), it switches the selection to supply the 2nd audio decoder 75-2.

  That is, the 1st audio decoder 75-1 receives the audio stream read from the buffer 52, which is the main Clip AV stream file read buffer (type = 1 or type = 3 in the example of FIG. 15, or in the example of FIG. type = 1), or an audio stream read from the buffer 53 which is a sub clip AV stream file read buffer (type = 2 or type = 3 in the example of FIG. 15, or in the example of FIG. 17) (if type = 2) is supplied, and only the audio stream read from the buffer 53 which is the sub-clip AV stream file read buffer is supplied to the 2nd audio decoder 75-2.

  The 1st audio decoder 75-1 decodes the audio stream and supplies the decoded audio stream data to the mixing processing unit 101. The 2nd audio decoder 75-2 decodes the audio stream and supplies the decoded audio stream data to the mixing processing unit 101.

  Here, when two audio streams are superimposed and reproduced, the audio stream decoded by the 1st audio decoder 75-1 and the audio stream decoded by the 2nd audio decoder 75-2 are mixed into the mixing processing unit 101. To be supplied.

  For example, when the 1st audio stream referred to in the main path and the 2nd audio stream referred to in the sub path are superimposed and reproduced (type = 3 in the example of FIG. 15, secondary_audio_present_flag is 1, 1 in FIG. 16). The first Stream_entry () type = 1, the second Stream_entry () type = 2), the audio stream of the main Clip AV stream file decoded by the 1st audio decoder 75-1 and the 2nd audio decoder 75- The audio stream of the sub-Clip AV stream file decoded in step 2 is supplied to the mixing processing unit 101.

  Also, for example, when two audio streams referenced in the sub path are superimposed and reproduced (secondary_audio_present_flag in FIG. 16 is 1, and type is 2 in both the first and second Stream_entry () In some cases, the audio stream of the sub Clip AV stream file (type = 2, the audio stream defined by the first stream entry () in FIG. 16) decoded by the 1st audio decoder 75-1 and the 2nd audio decoder 75- 2, the audio stream of the sub-Clip AV stream file (type = 2, when the secondary_audio_present_flag in FIG. 16 is 1) is the mixing processing unit 101. To be supplied.

  The mixing processing unit 101 mixes (superimposes) the audio data from the 1st audio decoder 75-1 and the audio data from the 2nd audio decoder 75-2, and outputs the mixed data to the subsequent mixing processing unit 97. In the present embodiment, mixing (superimposing) the audio data output from the 1st audio decoder 75-1 and the audio data output from the 2nd audio decoder 75-2 is also referred to as synthesis. In other words, synthesis means to mix two audio data.

  The sound data selected by the switch 32 is supplied to the buffer 95 and buffered. The buffer 95 supplies the sound data to the mixing processing unit 97 at a predetermined timing. The sound data is, for example, sound effect data by menu selection or the like. The mixing processing unit 97 is the audio data mixed by the mixing processing unit 101 (audio data obtained by mixing the audio data output from the 1st audio decoder 75-1 and the audio data output from the 2nd audio decoder 75-2). The sound data supplied from the buffer 95 is mixed (superimposed or synthesized) and output as an audio signal.

  The data read from the buffer 54 that is a text subtitle read buffer is output to the text subtitle composition (decoder) 76 at the subsequent stage at a predetermined timing. The text subtitle composition 76 decodes the Text-ST data and supplies it to the switch 77.

  The switch 77 selects either the presentation graphics stream decoded by the presentation graphics decoder 73 or Text-ST (text subtitle data), and supplies the selected data to the presentation graphics plane generating unit 93. . That is, the subtitle image simultaneously supplied to the presentation graphics plane 93 is an output of either the presentation graphics decoder 73 or the text subtitle (Text-ST) composition 76. In addition, the presentation graphics stream that is simultaneously input to the presentation graphics decoder 73 is a stream separated from either the main Clip AV stream file or the sub Clip AV stream file (selected by the switch 57). Therefore, the subtitle image that is simultaneously output to the presentation graphics plane 93 is a decoding output of the presentation graphics stream from the main Clip AV stream file, or the presentation graphics stream from the sub Clip AV stream file, or text subtitle data. It is.

Based on the background image data supplied from the background decoder 71, for example, the background plane generation unit 91 generates a background plane that becomes a wallpaper image when the video image is displayed in a reduced size, and this is converted into the video data. This is supplied to the processing unit 96. The video plane generation unit 92 generates a video plane based on the video data supplied from the video decoder 72 and supplies this to the video data processing unit 96.
The presentation graphics plane generation unit 93 generates, for example, a presentation graphics plane which is a rendering image based on the data (presentation graphics stream or text subtitle data) selected by the switch 77 and supplied, and this is converted into a video. The data is supplied to the data processing unit 96. The interactive graphics plane generating unit 94 generates an interactive graphics plane based on the data of the interactive graphics stream supplied from the interactive graphics decoder 74, and supplies this to the video data processing unit 96.

  The video data processing unit 96 includes a background plane from the background plane generation unit 91, a video plane from the video plane generation unit 92, a presentation graphics plane from the presentation graphics plane generation unit 93, and an interactive graphics plane generation unit. The interactive graphics plane from 94 is synthesized and output as a video signal.

  The switches 57 to 59 and the switch 77 are switched based on the selection from the user via the operation input unit 29 or to the side where the file including the target data is supplied. For example, when the audio stream is included only in the sub-Clip AV stream file, the switch 59 switches the switch to the side that supplies the sub-Clip AV stream file (the output of the PID filter 56).

  Next, playback processing in the playback apparatus 1 of FIG. 28 will be described with reference to the flowcharts of FIGS. Note that this process is started when a reproduction of a predetermined AV stream is instructed by the user via the operation input unit 29.

  In step S <b> 11, the controller 21 reads out the PlayList file recorded in the local storage 24 such as the recording medium 11 or an HDD (Hard Disk Drive) via the storage drive 22. For example, the PlayList file described with reference to FIG. 9 is read.

  In step S12, the controller 21 reads out the main Clip AV stream file, the sub Clip AV stream file, and the text subtitle data (Text-ST data). Specifically, the controller 21 reads out the corresponding main Clip AV stream file from the main Clip based on the PlayItem included in the PlayList described with reference to FIG. Further, the controller 21 reads the sub Clip AV stream file and the text subtitle data based on the SubPlayItem described with reference to FIGS. 10 to 12 that is referred to by the SubPath included in the PlayList.

  In step S <b> 13, the controller 21 controls the switch 32 to supply the read data (main Clip AV stream file, sub Clip AV stream file, and text subtitle data) to the corresponding buffers 51 to 54. Specifically, the controller 21 supplies background image data to the buffer 51, supplies data of the main Clip AV stream file to the buffer 52, supplies data of the sub Clip AV stream file to the buffer 53, and Text- The switch 32 is switched to supply ST data to the buffer 54.

  In step S <b> 14, the switch 32 switches the switch 32 based on the control from the controller 21. Thereby, the background image data is supplied to the buffer 51, the data of the main Clip AV stream file is supplied to the buffer 52, the data of the sub Clip AV stream file is supplied to the buffer 53, and the text subtitle data is supplied to the buffer 54. Is done.

  In step S15, each of the buffers 51 to 54 buffers the supplied data. Specifically, the buffer 51 buffers background image data, the buffer 52 buffers data of the main Clip AV stream file, the buffer 53 buffers data of the sub Clip AV stream file, The buffer 54 buffers Text-ST data.

  In step S <b> 16, the buffer 51 outputs the background image data to the background decoder 71.

  In step S 17, the buffer 52 outputs the stream data of the main Clip AV stream file to the PID filter 55.

  In step S18, the PID filter 55 distributes to the decoder of each elementary stream based on the PID attached to the TS (transport stream) packet constituting the main Clip AV stream file. Specifically, the PID filter 55 supplies the video stream to the video decoder 72, supplies the presentation graphics stream to the switch 57 serving as the supply source to the presentation graphics decoder 73, and converts the interactive graphics stream to the interactive graphics. The audio stream is supplied to the switch 58 serving as the supply source to the decoder 74, and the audio stream is supplied to the switch 59 serving as the supply source to the 1st audio decoder 75-1. That is, different PIDs are attached to the video stream, presentation graphics stream, interactive graphics stream, and audio stream, respectively.

  In step S <b> 19, the buffer 53 outputs the stream data of the sub Clip AV stream file to the PID filter 56.

  In step S20, the PID filter 56 distributes each elementary stream to a decoder corresponding to each elementary stream based on the PID. Specifically, the PID filter 56 supplies the presentation graphics stream to the switch 57 serving as the supply source to the presentation graphics decoder 73, and the switch 58 serving as the supply source to the interactive graphics decoder 74. And the audio stream is supplied to the switch 59 serving as a supply source to the 1st audio decoder 75-1 and the 2nd audio decoder 75-2.

  In step S21, the switches 57 to 59 subsequent to the PID filter 55 and the PID filter 56 select either the main Clip AV stream file or the sub Clip AV stream file based on control from the controller 21. Specifically, the switch 57 selects the presentation graphics stream of the main Clip AV stream file or the sub Clip AV stream file supplied from the PID filter 55 and supplies the presentation graphics stream to the subsequent presentation graphics decoder 73. Further, the switch 58 selects the interactive graphics stream of the main Clip AV stream file or the sub Clip AV stream file supplied from the PID filter 55 and supplies the interactive graphics stream to the subsequent interactive graphics stream decoder 74. Further, the switch 59 selects the audio stream of the main Clip AV stream file supplied from the PID filter 55 or the sub Clip AV stream file supplied from the PID filter 56, and supplies the selected audio stream to the first audio decoder 75-1. Note that when the user issues a command to switch the audio, the switch 59 may supply the audio stream of the sub Clip AV stream file to the 2nd audio decoder 75-2. Since the process before switching is described, the description is omitted.

  In step S <b> 22, the buffer 54 outputs the text subtitle data to the text subtitle composition 76.

  In step S <b> 23, the background decoder 71 decodes the background image data and outputs it to the background plane generation unit 91.

  In step S <b> 24, the video decoder 72 decodes the video stream and outputs it to the video plane generation unit 92.

  In step S25, the presentation graphics decoder 73 decodes the presentation graphics stream selected and supplied by the switch 57, and outputs it to the subsequent switch 77.

  In step S <b> 26, the interactive graphics decoder 74 decodes the interactive graphics stream selected and supplied by the switch 58, and outputs this to the subsequent interactive graphics plane generating unit 94.

  In step S27, the 1st audio decoder 75-1 decodes the supplied audio data selected by the switch 59 and outputs it to the mixing processing unit 101 at the subsequent stage. In the reproduction process (reproduction process in FIGS. 29 to 32) in a state where the user has not instructed to switch the audio, the audio data is not output from the 2nd audio decoder 75-2. The audio data output from the 1st audio decoder 75-1 is supplied to the subsequent mixing processing unit 97 as it is.

  In step S28, the Text-ST composition 76 decodes the text subtitle data and outputs it to the switch 77 at the subsequent stage.

  In step S29, the switch 77 selects either the presentation graphics decoder 73 or the data from the Text-ST composition 76. Specifically, the switch 77 selects any one of the presentation graphics stream decoded by the presentation graphics decoder 73 and Text-ST (text subtitle data), and the selected data is displayed in the presentation graphics sprayer. Supplied to the generator 93.

  In step S <b> 30, the background plane generation unit 91 generates a background plane based on the background image data supplied from the background decoder 71.

  In step S31, the video plane generating unit 92 generates a video plane based on the video data supplied from the video decoder 72.

  In step S32, the presentation graphics plane generating unit 93 selects the presentation based on the data from the presentation graphics decoder 73 or the data from the Text-ST composition 76 selected and supplied by the switch 77 in the process of step S29. Generate a graphics plane.

  In step S <b> 33, the interactive graphics plane generating unit 94 generates an interactive graphics plane based on the interactive graphics stream data supplied from the interactive graphics decoder 74.

  In step S34, the buffer 95 buffers the sound data selected and supplied in step S14 and supplies the sound data to the mixing processing unit 97 at a predetermined timing.

  In step S35, the video data processing unit 96 combines and outputs the data of each plane. Specifically, the data from the background plane generation unit 91, the video plane generation unit 92, the presentation graphics plane generation unit 93, and the interactive graphics plane generation unit 94 are combined and output as video data.

  In step S36, the mixing processing unit 97 mixes (synthesizes) audio data (audio data output from the mixing processing unit 101) and sound data, and outputs them, and the processing is completed.

  29 to 31, the main Clip AV stream file, the sub Clip AV stream file, and the text subtitle data associated with the main path and sub path included in the PlayList are referred to and reproduced. Since the main path and sub path are provided in the PlayList, and the clip AV stream file different from the clip AV stream file specified by the main path can be specified by the sub path, it is different from the main clip indicated by the PlayItem of the main path. The sub clip AV stream file data included in the sub clip that is a clip and the main clip AV stream file data included in the main clip can be reproduced together (at the same timing).

  In FIG. 29 to FIG. 31, the processing of step S16 and step S17 may be performed in reverse order or in parallel. In addition, the processes in step S18 and step S20 may be performed in reverse order or in parallel. Further, the order of steps S23 to S28 may be reversed or may be executed in parallel. In addition, the processes in steps S30 to S33 may be executed in reverse order or in parallel. Furthermore, the order of steps S35 and S36 may be reversed, or may be executed in parallel. That is, in FIG. 28, the processing of buffers 51 to 54 in the same vertical layer, the processing of switches 57 to 59, the processing of decoders 71 to 76, the processing of plane generation units 91 to 94, the video data processing unit 96, and the mixing processing unit The processes 97 may be executed in parallel, and the order thereof is not limited.

  Next, processing in the playback device 1 when an instruction to switch audio is given will be described with reference to the flowchart of FIG. This process is, for example, a process executed during the execution of the reproduction process of FIGS.

  In step S51, the controller 21 acquires an order list of audio stream numbers (may be IDs). Specifically, the controller 21 refers to the STN_table () of the PlayItem described with reference to FIG. 13, and further, the audio stream number (ID) entered in the STN_table () described with reference to FIG. 14 or FIG. Get a list of the order. This process is a process executed when the reproduction process of FIGS. 29 to 31 is started.

When a voice switching command is issued by the user via the operation input unit 29, the controller 21 receives a voice switching command from the user in step S52.
At this time, the controller 21 may control an external display device (not shown) to display a stream number table as shown in FIG. 26 on the external display device. In this way, the controller 21 displays the audio signal and caption signal options to be provided to the user, and accepts an audio switching (audio selection) command from the user. In FIG. 32, the process of step S51 is a process that has been executed in advance, and the process after step S52 is performed when a voice switching command is issued by the user.

  In step S53, the controller 21 acquires the audio stream number next to the audio stream number being reproduced. For example, when an audio stream (Auxiliary audio stream) of SubClip_entry_id = 0 in FIG. 8 is being reproduced, the number of the audio stream file corresponding to the next SubClip_entry_id = 1 is acquired. For example, when an audio stream file corresponding to the audio 3 with A_SN = 3 in FIG. 26 is being reproduced, the next is audio 2 + audio 4 with A_SN = 4, so the controller 21 is referred to in the main path. The number of the audio stream corresponding to the audio 2 and the number of the audio stream corresponding to the audio 4 referred to by the sub path are acquired.

  In step S54, the controller 21 determines whether or not it has a function of reproducing an audio stream corresponding to the acquired number. Specifically, the controller 21 obtains, for example, by referring to various settings registered in the memory 23 as necessary, based on the contents described in the stream_attribute () (FIG. 18). It is determined whether or not the own (reproducing apparatus 1) has a function of reproducing the audio stream corresponding to the number. For example, in the case of type = 1 in the Stream_entry () of FIG. 15 or FIG. 17, the controller 21 determines whether or not it has a function of reproducing the audio stream of the main Clip AV stream file corresponding to the acquired number. . Further, for example, in the case of type = 2 in Stream_entry () of FIG. 15 or FIG. 17, the controller 21 determines whether or not it has a function of playing back the audio stream of the sub clip corresponding to the acquired number.

Further, for example, when the audio stream of the main Clip AV stream file and the audio stream of the sub Clip AV stream file are mixed and reproduced (in the first STN_table of FIG. 14, type = 3 in the stream_entry () of FIG. In the second STN_table in FIG. 16, secondary_audio_present_flag is 1, type = 1 in the first Stream_entry (), and type = 2 in the second Stream_entry ()), the controller 21 Determines whether or not it has a function of playing back an audio stream corresponding to the acquired number for only the audio stream of the main Clip AV stream file. Here, in the case of mixing and reproducing the audio stream of the main Clip AV stream file and the audio stream of the sub Clip, determination as to whether or not it has a function of reproducing the audio stream of the sub Clip AV stream file may be omitted.
This is because the main Clip AV stream file may include an extended stream or a multi-channel stream, but a sub Clip AV stream file is likely to have some restrictions. This is because it is not particularly necessary to determine whether or not the stream included in the sub Clip AV stream file can be reproduced. Of course, the controller 21 may determine whether or not the stream included in the sub Clip AV stream file can be reproduced.

  Also, for example, when audio streams of two sub-Clip AV stream files are mixed and played back (in the second STN_table of FIG. 16, secondary_audio_present_flag is 1 and the first Stream_entry () is type = 2. When type = 2 of the second Stream_entry () is 2), the controller 21 decodes the first audio decoder 75-1 (first in the for loop of the audio stream ID (audio_stream_id) in FIG. 16). It is determined whether or not the stream has a function of playing back an audio stream corresponding to the acquired number. Here, in the stream entry () of FIG. 16, whether or not it has a function of reproducing the audio stream of the sub-Clip AV stream file that is entered second after secondary_audio_present_flag is set to 1 may be omitted. Good. This is because the first audio stream in the for loop of the audio stream ID (audio_stream_id) in FIG. 16 is assumed to be a sound such as Japanese dubbing of a movie, and is entered second in the stream entry () in FIG. Since it is likely that the audio quality is higher than that of the director's cut of a movie assumed as an audio stream of the sub-Clip AV stream file, the second in the stream entry () in FIG. This is because it is not necessary to determine whether or not it has a function of reproducing the audio stream of the entered sub-Clip AV stream file. Of course, the controller 21 may determine whether or not the audio streams of the two sub Clip AV stream files referenced by the two sub paths can be reproduced.

  If it is determined in step S54 that the audio stream corresponding to the acquired number is not reproduced, the controller 21 acquires the next stream number after the current stream number in step S55. That is, when the audio stream having the current stream number is not reproduced, the stream number is skipped (not subject to reproduction), and the next stream number is acquired. And after the process of step S55, a process returns to step S54 and the process after it is repeated. That is, the process is repeated until the controller 21 obtains the number of an audio stream having a function of reproducing itself.

  When it is determined in step S54 that the audio stream corresponding to the acquired number is played back, in step S56, the controller 21 performs the audio stream of the Clip AV stream file included in the main Clip corresponding to the acquired number. And the audio stream of the Clip AV stream file included in the sub clip. For example, in the example of FIG. 8, the acquired SubClip_entry_id = 1 is referred to by the Sub Path, so the controller 21 confirms that the audio stream corresponding to the acquired number is in the sub Clip AV stream file. Can do. Also, for example, in the case of type = 1 in Stream_entry () of FIG. 15 or FIG. 17, the controller 21 confirms that the audio stream corresponding to the acquired number is in the main Clip AV stream file. Further, for example, in the case of type = 2 in Stream_entry () of FIG. 15 or FIG. 17, the controller 21 confirms that the audio stream corresponding to the acquired number is in the sub Clip AV stream file. Also, for example, when audio streams of the main Clip AV stream file and the sub Clip AV stream file are mixed and reproduced (in the first STN_table of FIG. 14, when stream = entry () of FIG. 15 is type = 3, or FIG. In the second STN_table, secondary_audio_present_flag is 1, type = 1 in the first Stream_entry (), and type = 2 in the second Stream_entry ()), the controller 21 corresponds to the acquired number. Confirm that the audio stream to be recorded is in both the main Clip AV stream file and the sub Clip AV stream file. Further, for example, when audio streams of two sub-Clip AV stream files are mixed and played back (secondary_audio_present_flag in FIG. 16 is 1, type = 2 in the first Stream_entry (), and in the second Stream_entry () When type = 2), the controller 21 confirms that the audio stream corresponding to the acquired number exists only in the sub Clip AV stream file.

  In step S57, the controller 21 specifies a desired audio stream. Specifically, a desired audio stream in the main Clip AV stream file or the sub Clip AV stream file to which the stream corresponding to the acquired number belongs is specified. For example, in FIG. 15 or FIG. 17, the controller 21 specifies a predetermined audio stream in the main Clip AV stream file when type = 1, and if type = 2, 15 is specified and type = 3 in FIG. 15, or in FIG. 17, secondary_audio_present_flag is 1, first Stream_entry () is type = 1, and second Stream_entry () is type = 2. In this case, a predetermined audio stream in the main Clip AV stream file and a predetermined audio stream in the sub Clip are specified. In FIG. 17, when secondary_audio_present_flag is 1, the first Stream_entry () is type = 2, and the second Stream_entry () is type = 2, the controller 21 uses two of the two sub Clip AV stream files. Identify a given audio stream.

  In step S <b> 58, the controller 21 determines whether a clip (a main clip AV stream file only, a sub clip AV stream file only, a main clip AV stream file and a sub clip AV stream file, or two in which a desired audio stream is multiplexed. The storage drive 22 is instructed to read the specified audio stream from the sub clip AV stream file), or the desired audio stream recorded in the local storage 24 is multiplexed. The specified audio stream is read from the clip that has been made into a clip. When receiving an instruction from the controller 21, the storage drive 22 reads the target audio stream based on the instruction. Then, the controller 21 supplies the read audio stream to the AV decoder unit 26 and instructs the AV decoder unit 26 to play it back. For example, in the first STN_table in FIG. 14, the type_3 in the stream_entry () in FIG. 15, or in the second STN_table in FIG. 16, the secondary_audio_present_flag is 1 and the first Stream_entry When () is type = 1 and the second Stream_entry () is type = 2, the AV decoder unit 26 is instructed to play both the main clip audio stream and the sub-clip AV stream file audio stream. To do.

  In step S59, the AV decoder unit 26 decodes the audio stream and outputs the audio. That is, the audio data decoded by the audio decoder 75 and the sound data output from the buffer 95 are mixed by the mixing processor 97 and output as an audio signal. Specifically, when type = 1 in FIG. 15 or FIG. 17, the 1st audio decoder 75-1 decodes a predetermined audio stream in the main Clip AV stream file and outputs it to the mixing processing unit 101. To the mixing processing unit 97. In addition, when type = 2 in FIG. 15 or FIG. 17, either the 1st audio decoder 75-1 or the 2nd audio decoder 75-2 (determined by the description of STN_table) is included in the sub Clip AV stream file. A predetermined audio stream is decoded and supplied to the mixing processing unit 97 via the mixing processing unit 101.

  At this time, when the audio streams of the main Clip AV stream file and the sub Clip AV stream file are mixed and reproduced (when type = 3 in FIG. 15 or secondary_audio_present_flag in FIG. 16 is 1, the type of the first Stream_entry () = 1, 2nd Stream_entry () where type = 2) The first audio decoder 75-1 decodes a predetermined audio stream in the main Clip AV stream file and supplies this to the mixing processing unit 101 Then, the 2nd audio decoder 75-2 decodes a predetermined audio stream in the sub Clip AV stream file and supplies the decoded audio stream to the mixing processing unit 101. Also, when mixing and reproducing the audio streams of two sub-Clip AV stream files (when secondary_audio_present_flag in FIG. 16 is 1, type = 2 of the first Stream_entry (), and type = 2 of the second Stream_entry ()) ) The 1st audio decoder 75-1 decodes a predetermined audio stream in the sub-Clip AV stream file and supplies it to the mixing processing unit 101. The 2nd audio decoder 75-2 receives the sub-Clip AV stream file. The predetermined audio stream is decoded and supplied to the mixing processing unit 101. The mixing processing unit 101 mixes (superimposes and synthesizes) the audio data supplied from the 1st audio decoder 75-1 and the audio data supplied from the 2nd audio decoder 75-2, and supplies this to the mixing processing unit 97. To do. The mixing processing unit 97 mixes (superimposes and synthesizes) the audio data supplied from the mixing processing unit 101 and the sound data supplied from the buffer 95, and outputs this as an audio signal.

  A detailed example of the process in step S59 in FIG. 32 will be described with reference to the flowchart in FIG. FIG. 33 is a flowchart illustrating an example of processing for outputting an audio stream in the AV decoder unit 26.

  In step S <b> 81, the AV decoder unit 26 determines whether or not the audio stream to be reproduced is only the audio stream of the main Clip AV stream file based on the control from the controller 21. For example, the AV decoder unit 26 determines whether the audio stream entered in FIG. 14 or 16 is only type = 1 in FIG. 15 or FIG.

  If it is determined in step S81 that the audio stream to be reproduced is only the audio stream of the main Clip AV stream file, the audio stream of the main Clip AV stream file selected and supplied by the PID filter 55 is Since it is selected and supplied to the 1st audio decoder 75-1, the 1st audio decoder 75-1 decodes the audio stream of the main Clip AV stream file in step S82. The 1st audio decoder 75-1 supplies the decoded audio stream of the main Clip AV stream file to the mixing processing unit 101 at the subsequent stage. The mixing processing unit 101 outputs the supplied audio data as it is to the subsequent mixing processing unit 97.

  If it is determined in step S81 that the audio stream to be reproduced is not only the audio stream of the main Clip AV stream file (that is, if the audio stream to be reproduced is type = 2 or type = 3 in FIG. 15, or FIG. In step S83, the AV decoder unit 26 determines whether or not the audio stream to be reproduced is only the audio stream of the sub Clip AV stream file based on the control from the controller 21 in step S83. Determine. That is, the AV decoder unit 26 determines whether the audio stream entered in FIG. 14 or 16 is only type = 2 in FIG. 15 or FIG.

  If it is determined in step S83 that the audio stream to be reproduced is not only the audio stream of the sub Clip AV stream file, the audio stream to be reproduced is a combination of the audio stream of the main Clip AV stream file and the audio stream of the sub Clip AV stream file. That is, the audio stream to be reproduced is type = 3 in FIG. 15, or in FIG. 16, secondary_audio_present_flag is set to 1, and the first Stream_entry () is type = 1 and the second Since Stream_entry () is a combination in which type = 2, the audio stream of the main Clip AV stream file selected and supplied by the PID filter 55 is selected by the switch 59, and the 1st audio decoder 75- 1 and the audio stream of the sub Clip AV stream file selected and supplied by the PID filter 56 is selected by the switch 59 and supplied to the 2nd audio decoder 75-2.

  Therefore, in step S84, the 1st audio decoder 75-1 decodes the audio stream of the main Clip AV stream file. The 1st audio decoder 75-1 supplies the decoded audio stream of the main Clip AV stream file to the mixing processing unit 101 at the subsequent stage.

  In step S85, the 2nd audio decoder 75-2 decodes the audio stream of the sub Clip AV stream file. The 2nd audio decoder 75-2 supplies the decoded audio stream of the sub Clip AV stream file to the mixing processing unit 101 at the subsequent stage. Note that the processes of step S84 and step S85 may be executed in parallel or in reverse order.

  In step S86, the mixing processing unit 101 outputs the audio stream of the main Clip AV stream file decoded by the 1st audio decoder 75-1 and the audio stream of the sub Clip AV stream file decoded by the 2nd audio decoder 75-2. Mix. That is, the mixing processing unit 101 mixes (synthesizes and superimposes) the audio data supplied from the 1st audio decoder 75-1 and the audio data supplied from the 2nd audio decoder 75-2. The mixing processing unit 101 supplies the audio data obtained as a result of mixing to the mixing processing unit 97 at the subsequent stage.

  On the other hand, when it is determined in step S83 that the audio stream to be reproduced is only the audio stream of the sub Clip AV stream file (when only type = 2 in FIG. 15 or FIG. 17), the process proceeds to step S87. Based on the control from the controller 21, the AV decoder unit 26 determines whether or not the audio stream to be reproduced is a combination of two audio streams of the sub-Clip AV stream file. That is, secondary_audio_present_flag is set to 1 in the stream entry () of FIG. 16 and two audio streams are entered. Further, in FIG. 17, it is determined whether or not both are set to type = 2. In the case of the syntaxes of FIGS. 14 and 15, the combination of the two audio streams of the sub Clip AV stream file is not defined. Therefore, in step S87, the audio stream to be reproduced is the two audio streams of the sub Clip AV stream file. It is determined that the combination is not.

  If it is determined in step S87 that the audio stream to be reproduced is not a combination of two audio streams of the sub-Clip AV stream file (in FIG. 14 and FIG. 15, if type = 2, or FIG. 16 and FIG. 17, when secondary_audio_present_flag is 0 and type = 2 is set in the first Stream_entry (), that is, when only one audio stream is entered, this audio stream is of course a sub-Clip AV stream file. Therefore, the audio stream of the sub Clip AV stream file selected and supplied by the PID filter 56 is selected by the switch 59 and supplied to the 1st audio decoder 75-1. Therefore, in step S88, the 1st audio decoder 75-1 decodes the audio stream of the sub Clip AV stream file. The 1st audio decoder 75-1 supplies the decoded audio stream of the sub-Cli AV stream file p to the mixing processing unit 101 at the subsequent stage. The mixing processing unit 101 outputs the supplied audio data as it is to the subsequent mixing processing unit 97.

  On the other hand, when it is determined in step S87 that the audio stream to be reproduced is a combination of two audio streams of the sub-Clip AV stream file (in FIGS. 16 and 17, secondary_audio_present_flag is set to 1 and the first Stream_entry ( ) And the second Stream_entry () are both set to type = 2), the sub-clip AV stream file selected and supplied by the PID filter 56 has two audio streams, so the switch 59 is set to 1st The audio stream to be decoded by the audio decoder 75-1, that is, the first entry audio stream in the for loop of the audio stream ID (audio_stream_id) in FIG. 16 is supplied to the 1st audio decoder 75-1. Select 2n The audio stream decoded by the audio decoder 75-2, that is, the second entry audio stream after the secondary_audio_present_flag in the for loop of the audio stream ID (audio_stream_id) in FIG. A switch is selected to be supplied to the decoder 75-2.

  Therefore, in step S89, the 1st audio decoder 75-1 decodes the audio stream of the supplied sub Clip AV stream file. The 1st audio decoder 75-1 supplies the decoded audio stream of the sub Clip AV stream file to the mixing processing unit 101 at the subsequent stage.

  In step S90, the 2nd audio decoder 75-2 decodes the audio stream of the supplied sub Clip AV stream file. The 2nd audio decoder 75-2 supplies the decoded audio stream of the sub Clip AV stream file to the mixing processing unit 101 at the subsequent stage. Note that the processes of step S89 and step S90 may be executed in parallel or in reverse order.

  In step S91, the mixing processing unit 101 performs the audio stream of the sub Clip AV stream file decoded by the 1st audio decoder 75-1 and the audio stream of the sub Clip AV stream file decoded by the 2nd audio decoder 75-2 ( That is, the audio streams of the two sub Clip AV stream files are mixed. In other words, the mixing processing unit 101 mixes (synthesizes and superimposes) the audio data supplied from the 1st audio decoder 75-1 and the audio data supplied from the 2nd audio decoder 75-2. The mixing processing unit 101 supplies the audio data obtained as a result of mixing to the mixing processing unit 97 at the subsequent stage.

  After the process of step S82, after the process of step S86, after the process of step S88, or after the process of step S91, the process proceeds to step S92, and the mixing processing unit 97 combines the supplied audio data with the supplied audio data. The sound data supplied from the buffer 95 is mixed and output. For example, when the process of step S92 is after the process of step S82, the mixing processing unit 97 mixes the audio data obtained by decoding the audio stream of the main Clip AV stream file and the sound data. For example, when the process of step S92 is after the process of step S88, the mixing processing unit 97 mixes the audio data and the sound data obtained by decoding the audio stream of the sub Clip AV stream file. Further, for example, when the process of step S92 is after the process of step S86, the mixing processing unit 97 receives the audio data obtained by decoding the audio stream of the main Clip AV stream file and the audio stream of the sub Clip AV stream file. Sound data is mixed with audio data obtained by mixing the decoded audio data. For example, when the process of step S92 is after the process of step S91, the mixing processing unit 97 performs audio data obtained by mixing audio data obtained by decoding the audio streams of the two sub Clip AV stream files. Next, mix the sound data.

  According to the processing of FIG. 33, it is possible to perform mixing reproduction of the audio stream referenced in the main path and the audio stream referenced in the sub path. Also, it is possible to perform mixing reproduction of two audio streams that are referred to by two sub-paths.

  The selection of the switch 59 in FIG. 28 in step S21 in FIG. 30 is determined in the same manner as described with reference to FIG. That is, when the audio stream reproduced in the processing described with reference to FIG. 33 is only the main Clip AV stream file, the switch 59 converts the audio stream supplied from the main side, that is, the PID filter 55, to the 1st audio decoder 75- 1 is selected (YES in step S81 in FIG. 33), and when the target clip is only one sub clip AV stream file, the switch 59 is supplied from the sub side, that is, the PID filter 56. The selected audio stream is selected to be supplied to the 1st audio decoder 75-1 (NO in step S87 in FIG. 33), and the target Clip AV stream file is both the main Clip AV stream file and the sub Clip AV stream file. The switch 59 is supplied from the PID filter 55. The selected audio stream is supplied to the 1st audio decoder 75-1, and the audio stream supplied from the PID filter 56 is selected to be supplied to the 2nd audio decoder 75-2 (NO in step S83 in FIG. 33). When the clip is a two-sub clip AV stream file, the switch 59 sends the audio stream supplied from the sub-side, that is, the PID filter 56 to the 1st audio decoder 75-1 or the 2nd audio decoder 75-2. Select to supply (YES in step S87 in FIG. 33).

  In this way, the controller 21 can control the switching of audio (audio) based on STN_table () of PlayItem. Further, the controller 21 can control playback switching by selecting only a stream having a playback function by referring to the stream_attribute of the STN_table (). Further, the controller 21 superimposes and reproduces the audio stream included in the main Clip AV stream file and the audio stream included in the sub Clip AV stream file based on the STN_table (), and two sub Clip AVs. The audio stream included in the stream file can be controlled to be superimposed and reproduced.

  Note that when the syntaxes of FIGS. 14 and 15 (first STN_table ()) are used, the combination of two audio streams of the sub Clip AV stream file is not defined, so it is always determined NO in step S87 of FIG. The process after step S88 is performed. That is, in the case of corresponding to the syntaxes of FIGS. 14 and 15, the processes of step S89, step S90, and step S91 of FIG.

  In the processing of FIG. 32, the audio is switched based on the audio stream number, but the audio may be switched based on the audio stream ID (audio_stream_id). In this case, the audio stream ID is obtained by subtracting 1 from the audio stream number.

  In the above example, the audio stream referenced by the main path and the audio stream referenced by the sub path are mixed (synthesized) and reproduced (FIGS. 14 and 15 or FIG. 16 and FIG. 17) and an example (FIGS. 16 and 17) in which audio streams referenced by two sub-paths are mixed (synthesized) and reproduced, but not limited to audio streams, the types of streams are the same. If there is, it can be similarly applied to various stream files. For example, this configuration can also be applied to the case of picture-in-picture display in which two video streams (stream types are video streams) are combined and played back. Thus, according to the present invention, a combination of a predetermined type of stream file included in the main clip and a stream file included in the same type of sub clip is defined, or the same type of stream included in the sub clip is defined. By defining a combination of files, a stream referenced in the main path and a stream referenced in the sub path (two streams of the same type) and a stream referenced in the two sub paths (two streams of the same type) Can be synthesized and played back.

  That is, in the configuration described above, for example, as described later with reference to FIG. 43, the AV stream video stream referred to in the main path is displayed as the main screen, and the sub-path video path is displayed in the main screen. It can also be used for so-called PinP (picture-in-picture) display in which a referenced video stream is synthesized and displayed as a child screen. The details of executing PinP (picture in picture) display by applying the above-described configuration will be described later.

  As described above, when audio or subtitles are prepared in a stream or data file different from the main AV stream, since the PlayList has a main path and sub path, operations such as audio switching and subtitle switching are performed by the user. When performed, a stream to be reproduced can be selected from another stream different from the main Clip AV stream or another data file.

  In addition, since the first and second STN_table () can select a combination of two audio streams, that is, an audio stream included in the main clip and an audio stream included in the sub clip, the user can Audio stream superimposition playback can be selected. Specifically, in STN_table (), two stream_attributes () can be entered as audio streams, and stream_entry () can be entered as a combination of both main and sub clips. Therefore, it is possible to instruct to superimpose and reproduce two audio streams.

  Also, as shown in FIG. 16 and FIG. 17, since the second STN_table () is configured to select a combination of two audio streams included in two sub Clips (since secondary_audio_present_flag is provided), The user can select superimposed playback of two audio streams. Specifically, in the second STN_table (), since a combination of two stream_entry () and stream_attributes () can be entered with one audio stream ID as an audio stream, two audio streams are superimposed. And can be instructed to play.

  In addition, since the STN_Table that defines the type of data that is multiplexed in the AV stream file and the data that is referenced by the Sub Path is provided in the PlayItem of the main path, a stream with higher expandability is provided. Can be realized.

  Furthermore, the playback device 1 can sequentially select and play back only the streams corresponding to the functions of itself by referring to the stream_attribute of the STN_table ().

  The processing of the playback apparatus 1 described above is summarized as follows.

  The playback apparatus 1 includes a main path that is a main playback path indicating the position of a main clip AV stream file (main clip AV stream file including a plurality of types of streams) that is recorded on a recording medium and includes at least one stream. A PlayList as reproduction management information configured by a plurality of sub-paths indicating respective positions of the sub-Clip AV stream file to be reproduced in accordance with the reproduction timing of the main stream file included in the main Clip AV stream file, The sub-clip AV stream file of the same type (for example, the same type of main stream file (for example, the 1st audio stream) included in the main path in FIG. 2nd audio Based on the information indicating the combination of streams) STN_table () (STN_table shown by PlayItem ()), receives the selection of the stream to be reproduced.

  Here, a predetermined type main stream file (for example, a 1st audio stream) included in the main Clip AV stream file and a sub clip AV stream (2nd audio of the same type) that is reproduced in accordance with the reproduction timing of the main stream file. When the selection of the combination with the stream is accepted, the playback device 1 reads out the corresponding sub ClipAV stream file referred to by the subpath together with the selected main ClipAV stream file. The mixing processing unit 101 of the playback apparatus 1 combines the two audio streams (decoded audio streams), and the mixing processing unit 97 (AV decoder unit 26) plays back (outputs) the combined stream. .

  As a result, as shown in FIG. 34, the main clip AV stream file main movie (movie) includes video data (Video) and audio data (Audio), and as a director commentary, When an English commentary and a Japanese commentary are given in the sub-ClipAV stream file, the user has audio_stream_number = 1 main part audio, audio_stream_number = 2 main part audio + English commentary, and audio_stream_number = 3. You can select the audio stream to play from the main audio + commentary in Japanese. When a combination of the main ClipAV stream file and the sub ClipAV stream file (audio_stream_number = 1 and 2) is selected, they can be combined and reproduced.

  Thus, since the PlayList includes the Main Path and the Sub Path and refers to the Clip AV stream file having a different Main Path and Sub Path, the stream can be extended. Further, since a plurality of files can be referred to by one Sub Path (for example, FIG. 8), the user can select from a plurality of different streams.

  Further, in the Play item of the Main Path, the attached data (for example, an audio stream) multiplexed (included) in the AV stream file referenced by the Main Path and the attached data referenced by the Sub Path are defined. Since the first STN_table () described with reference to FIG. 14 or the second STN_table () described with reference to FIG. 16 is provided as the table to be performed, a more highly scalable stream can be realized. it can. Also, the Sub Path can be easily expanded by entering STN_table ().

  In addition, since stream_attribute () of FIG. 18 which is stream attribute information is provided in STN_table (), it is possible to determine whether or not the selected stream can be played back on the playback device 1 side. Furthermore, by referring to stream_attribute (), it is possible to select and reproduce only a stream having a reproduction function.

  Furthermore, in the first or second STN_table () (FIG. 14 or FIG. 16), the stream file referred to by the main path in the stream_entry () (FIG. 15 or FIG. 17) and the same type referred to by the sub path A combination with a stream file (here, an audio stream file) is defined, and a combination of two audio stream files referred to by sub-paths in FIGS. 16 and 17 is defined. As described above, the playback apparatus 1 is provided with two audio decoders (1st audio decoder 75-1 and 2nd audio decoder 75-2) and mixes the audio data decoded by the two audio decoders (mixing). I was going to provide part 101 It can be played simultaneously by combining two of the same type stream.

  For example, when selection of a combination of two sub-clip AV stream files of a predetermined type included in the sub-Clip AV stream file is accepted (that is, the second STN_table () is referred to, the secondary_audio_present_flag is set to 1, and the first When both Stream_entry () and the second Stream_entry () are set to type = 2), the playback device 1 reads the two selected sub ClipAV stream files. The mixing processing unit 101 of the playback apparatus 1 combines the two audio streams (decoded audio streams), and the mixing processing unit 97 (AV decoder unit 26) plays back (outputs) the combined stream. .

  Further, as shown in FIG. 11, the Sub Path includes a SubPath_type that indicates the type of Sub Path (type such as audio or text subtitle), a Clip_Information_file_name in FIG. 12 that indicates the name of the sub Clip referred to by the Sub Path, and the Sub Path. 12 includes the SubPlayItem_IN_time in FIG. 12 indicating the clip In point and the SubPlayItem_OUT_time in FIG. 12 indicating the Out point, so that the data referred to by the Sub Path can be accurately identified.

  Furthermore, in the Sub Path, sync_PlayItem_id in FIG. 12 (for example, sync_PlayItem_id in FIGS. 7 and 8), which is designation information for designating an AV stream file on the Main Path for reproducing the Main Path at the same timing as the Sub Path. And sync_start_PTS_of_PlayItem (for example, sync_start_PTS_of_PlayItem in FIGS. 7 and 8) that is the time on the main path at which the in point of the data referred to by the sub path starts synchronously on the time axis of the main path. 7 and FIG. 8, the data (file) referred to by the Sub Path can be reproduced in synchronization with the main ClipAV stream file referred to by the Main Path.

  The data read by the storage drive 22 in FIG. 28 and supplied to the AV decoder unit 26 is data recorded on a recording medium 11 such as a DVD (Digital Versatile Disc). The data recorded in the local storage 24 composed of a hard disk or the like is also supplied to the AV decoder unit 26 together with the data read by the storage drive 22. The data recorded in the local storage 24 may be data downloaded via the network 2. In other words, the data supplied to the AV decoder unit 26 may be data recorded on the removable recording medium 11, data stored in the playback device 1 in advance, Data supplied from a network via a network or a combination of these may be used. For example, AV stream playback processing is executed based on the PlayList and sub-Clips that are downloaded and recorded in the local storage 24 configured by a hard disk or the like, and the main Clip AV stream file that is recorded on the recording medium 11 such as a DVD. May be. Further, for example, when a PlayList that uses a Clip AV stream file recorded on a recording medium 11 such as a DVD as a sub-Clip and a main Clip are recorded in the local storage 24 constituted by a hard disk or the like, the local storage On the basis of the PlayList recorded in 24, the main Clip AV stream file and the sub Clip AV stream file may be read from the local storage 24 and the recording medium 11 and reproduced.

  Further, for example, when stream data indicated by one main path and one sub path is reproduced, or when stream data indicated by two sub paths is reproduced, the main path and sub path, or two sub paths are reproduced. When the stream data indicated by (2) is acquired as a separate transport stream (TS) by downloading or the like, the audio streams included in the two acquired different TSs are combined. On the other hand, even when the stream data indicated by the two sub-paths is reproduced, if one TS including at least two streams indicated by the two sub-paths is acquired, one TS The two audio streams included in are combined.

Further, when the first STN_table described with reference to FIG. 14 is used and two audio streams are defined in the main path and the sub path by setting type = 3 in the stream_attribute () of FIG. 15, or FIG. 16 is used. When the two STN_tables described above are used and two audio streams are defined in the stream_attribute () of FIG. 17 using only the main path and the sub path or only the sub path using the secondary_audio_present_flag, the 1st audio decoder 75-1 of the playback apparatus 1 is used. Decodes the audio stream of the main Clip AV stream file or the sub Clip AV stream file, and the 2nd audio decoder 75-2 decodes the audio stream of the sub Clip AV stream file, but the codec (encoding) system is the same ( For example, Dolby AC-3 audio in Fig. 19) A constraint may be provided so that a combination of audio streams of two Clips is defined by STN_table (). In other words, a restriction may be provided so that what is decoded simultaneously in the combination defined by STN_table () has the same codec format.
In this case, even when the 1st audio decoder 75-1 and the 2nd audio decoder 75-2 are in the middle of decoding the audio stream (for example, in the state where inverse quantization is performed in MPEG but IDCT is not performed) This is because if the codec method is the same, the data method in the middle of decoding is the same, so that the mixing processing unit 101 can perform mixing.

  In the playback apparatus 1 described above, by referring to the first STN_table () in FIG. 14 or the second STN_table () in FIG. 16, the clip referred to by PlayItem and the clip referenced by SubPath are combined. It was made so that you could choose to recycle things. Next, different examples for providing a mechanism capable of selecting mixing reproduction of two audio streams will be described.

FIG. 35 is a diagram illustrating a third example (third STN_table ()) of syntax of STN_table () in the case of performing mixing reproduction of two audio streams. It goes without saying that the third STN_table () is also set as an attribute of PlayItem. The third STN_table () in FIG. 35 is configured so that the clip to be played can be freely selected instead of selecting the playback of the clip referred to by PlayItem and the clip referenced by SubPath. .
In addition, about the part similar to the case where it demonstrated using FIG. 14, the description is abbreviate | omitted suitably.

  The third STN_table () in FIG. 35 differs from the case described with reference to FIG. 14 in that two audio streams are defined. Length, number_of_video_stream_entries, video_stream_id, and video_stream_number are the same as those described with reference to FIG.

  number_of_audio_stream_entries indicates the number of streams of the first audio stream to which audio_stream_id entered in the third STN_table () is given. The audio_stream_id is information for identifying the audio stream, and the audio_stream_number is an audio stream number that can be seen by the user used for audio switching. number_of_audio_stream2_entries indicates the number of streams of the second audio stream to which audio_stream_id2 entered in the third STN_table () is given. The audio_stream_id2 is information for identifying an audio stream, and the audio_stream_number is an audio stream number that can be seen by the user used for audio switching.

  Specifically, the audio stream of number_of_audio_stream_entries entered in the third STN_table () is an audio stream decoded by the 1st audio decoder 75-1 of the playback device 141 in FIG. 36 or the playback device 401 in FIG. The number_of_audio_stream2_entries audio stream entered in the third STN_table () is an audio stream decoded by the 2nd audio decoder 75-2 of the playback device 141 in FIG. 36 or the playback device 401 in FIG. 53 described later. In this way, in the third STN_table () in FIG. 35, audio streams to be decoded by the two audio decoders can be entered.

  In the following, the audio stream of number_of_audio_stream_entries decoded by the 1st audio decoder 75-1 of the playback device 141 of FIG. 36 or the playback device 401 of FIG. The audio stream of number_of_audio_stream2_entries decoded by the 2nd audio decoder 75-2 of the playback device 141 in FIG. 36 or the playback device 401 in FIG. 53 is sometimes referred to as an audio stream. # 2 (further, the audio stream # 2 may be referred to as a secondary audio stream, as will be described later). Also, it is assumed that the audio stream # 1 is an audio stream that has priority over the audio stream # 2.

  number_of_PG_txtST_stream_entries, PG_txtST_stream_id, and PG_txtST_stream_number are the same as those described with reference to FIG.

  Further, number_of_IG_stream_entries, IG_stream_id, and IG_stream_number are the same as those described with reference to FIG.

  The stream_entry () syntax is the same as that described with reference to FIG.

  Similarly to the for loop of the video stream ID (video_stream_id) and the for loop of the audio stream ID (audio_stream_id) described with reference to FIGS. 14 and 16, the for loop of the audio stream ID2 (audio_stream_id2) Thus, audio_stream_id2 is given from 0 to one audio elementary stream that is specified in order for each stream_entry (). As in the case of the video stream, audio stream number 2 (audio_stream_number2) may be used instead of audio stream ID2 (audio_stream_id2). In this case, audio_stream_number2 is given from 1 instead of 0. That is, audio_stream_number2 is obtained by adding 1 to the value of audio_stream_id2. The audio stream number 2 is an audio stream number 2 that can be seen by the user and used for audio switching.

  That is, in the third STN_table () of FIG. 35, an audio stream of number_of_audio_stream_entries (audio stream # 1) and an audio stream of number_of_audio_stream2_entries (audio stream # 2) are defined. In other words, since the audio stream # 1 and the audio stream # 2 can be entered using the third STN_table (), the user can select two audio streams to be played back synchronously. .

  Then, the for loop of the subtitle stream ID (PG_txtST_stream_id) and the id of the for loop of the graphics stream ID (IG_stream_id) are the same as those in FIG. 14 or FIG.

  Next, stream_attribute () of the third STN_table () in FIG. 35 will be described. Note that the syntax of stream_attribute () is the same as that described with reference to FIGS.

  In the for statement after reserved_for_future_use, the video stream is referenced by the amount of the video stream, the audio stream is referenced by the disc manufacturer who has set the Main Path and Sub Path for the audio stream, and the amount of PG textST stream. Only the PG textST stream is referred to, and the IG stream is referred to by the amount of the IG stream.

  The stream_attribute () in the for loop of the video stream ID (video_stream_id) gives stream attribute information of one video elementary stream specified for each stream_entry (). That is, in this stream_attribute (), stream attribute information of one video elementary stream specified for each stream_entry () is described.

  Similarly, stream_attribute () in the for loop of the audio stream ID (audio_stream_id) gives stream attribute information of one audio elementary stream specified for each stream_entry (). That is, in this stream_attribute (), stream attribute information of one audio elementary stream specified for each stream_entry () is described. For example, since there is one audio elementary stream specified by type = 1 or type = 2 of stream_entry () in FIG. 15, stream_attribute () gives stream attribute information of that one audio elementary stream.

  Similarly, stream_attribute () in the for loop of audio stream ID2 (audio_stream_id2) gives stream attribute information of one audio elementary stream specified for each stream_entry (). That is, in this stream_attribute (), stream attribute information of one audio elementary stream specified for each stream_entry () is described. For example, since there is one audio elementary stream specified by type = 1 or type = 2 of stream_entry () in FIG. 15, stream_attribute () gives stream attribute information of that one audio elementary stream.

  Similarly, stream_attribute () in the for loop of the subtitle stream ID (PG_txtST_stream_id) gives stream attribute information of one bitmap subtitle elementary stream or text subtitle elementary stream specified for each stream_entry (). That is, in this stream_attribute (), stream attribute information of one bitmap subtitle elementary stream specified for each stream_entry () is described.

  Similarly, stream_attribute () in the for loop of the graphics stream ID (IG_stream_id) gives stream attribute information of one interactive graphics elementary stream specified for each stream_entry (). That is, stream attribute information of one interactive graphics elementary stream specified for each stream_entry () is described in the stream_attribute ().

  As described above, even when the third STN_table is used, when a PlayItem and one or more SubPaths to be played back are prepared, the Clip referred to by the PlayItem and one or more SubPaths are referenced. The attribute information of one elementary stream specified by stream_entry () can be known from stream_attribute ().

  Whether or not the playback apparatus has a function of playing back the elementary stream by examining this attribute information (stream_attribute ()) in the same manner as when the first or second STN_table is used. Can be examined. Further, the playback device can select the elementary stream corresponding to the initial information of the language setting of the playback device by examining the attribute information.

  Note that when the third STN_table () of FIG. 35 is applied, both the audio stream # 1 and the audio stream # 2 may be audio streams included in the Clip referred to by the main path. Also, one of the audio stream # 1 and the audio stream # 2 may be an audio stream included in a clip referred to by the main path, and the other may be an audio stream included in the clip referenced by the sub path. As described above, when the third STN_table () of FIG. 35 is applied, two audio streams superimposed on the main Clip AV stream referenced by the main path may be selected, mixed, and reproduced. Is possible.

  In this way, by providing the third STN_table () in PlayItem (), when the PlayItem and one or more SubPaths to be played back in association with it are prepared, the user's voice By the operation of switching or subtitle switching, a mechanism is provided that allows the user to select any combination of streams to be played from the Clip referenced by this PlayItem and the Clip referenced by one or more SubPaths. Interactive operations can be performed on a stream or data file different from the AV stream to be reproduced.

  Similarly to the case of the first and second STN_tables, when the third STN_table is applied, a plurality of SubPaths are used in one PlayList, and each SubPath refers to a SubPlayItem. Therefore, it is possible to realize an AV stream with high expandability and high flexibility. That is, a configuration in which SubPlayItems can be added later in the content played back by this PlayList.

Furthermore, the third STN_table () described with reference to FIG. 35 includes audio stream # 1 decoded by the 1st audio decoder 75-1 of the playback device 141 in FIG. 36 or the playback device 401 in FIG. A mechanism capable of mixing and reproducing the audio stream # 2 decoded by the audio decoder 75-2 is provided. For example, when PlayItem () and one or more SubPaths to be played back are prepared, the audio stream included in the Clip referred to by PlayItem is set as the audio stream # 1, and included in the Clip referred to by SubPath. The audio stream is set to audio stream # 2, and a mechanism for mixing and reproducing these is provided. Also, for example, two audio streams included in a clip (main clip) referred to by PlayItem are set as an audio stream # 1 and an audio stream # 2, respectively, and these can be mixed and played back. did.
Thereby, the main audio stream and an audio stream different from the main audio stream (for example, a director's comment stream) can be superimposed and reproduced. Also, the two audio streams # 1 and # 2 superimposed on the main ClipAV stream can be reproduced by being superimposed (mixed).

  Specifically, for example, as in the case described with reference to FIG. 26, in the stream number table representing the relationship between the audio signal and the caption signal provided to the user, the audio number is A_SN (Audio Stream Number), A_SN2 When the subtitle number is called S_SN (SubPicture Stream Number), the audio stream # 1 (audio stream entered with audio_stream_id) entered in the third STN_table () of the PlayItem constituting the Main Path of the PlayList A_SN is given to each, and the user can select the audio stream # 1 to be reproduced from the audio streams to which A_SN is given. Similarly, by giving A_SN2 to each of the audio streams # 2 (audio streams entered by audio_stream_id2) entered in the third STN_table (), the audio stream mixed with the selected audio stream # 1 is given to the user # 2 can be selected from the audio streams to which A_SN2 is given.

  Here, since A_SN for selecting audio stream # 1 and A_SN2 for selecting audio stream # 2 can be independently selected by the user, switching between the two sounds is independent. ing.

  That is, by defining two audio streams to be reproduced (audio stream # 1 and audio stream # 2) separately, the user can arbitrarily select two audio streams to be reproduced from the defined ones. Can do. In other words, the user can freely select two audio streams to be played (freely defined by the audio stream # 1 and the audio stream # 2). High selection can be made. For example, the user can select a combination of audio 2 + audio 4 (a combination of A_SN = 1 and A_SN2 = 1) or a combination of audio 2 + audio 5 (a combination of A_SN = 1 and A_SN2 = 2).

  As described above, the stream_entry () of the third STN table () (FIG. 35) in PlayItem () is configured so that two audio streams can be entered, so the two audio streams are mixed. Can be played. That is, two streams of the same type (in this example, an audio stream) can be mixed (superimposed or combined) from a plurality of types of streams and simultaneously played back. In addition, the user can instruct mixing reproduction of two desired types of streams.

  Next, as a different example of a playback apparatus to which the present invention is applied, two audio streams are configured using a configuration in which two audio streams can be individually entered, such as a third STN table () (FIG. 35). A playback device 141 that can mix and play a stream will be described.

  The playback device 141 basically has the same configuration as the playback device 1 described with reference to FIG. 27 except that the AV decoder unit 151 is provided instead of the AV decoder unit 26 of the playback device 1. It is. FIG. 36 is a block diagram illustrating a configuration example of the AV decoder unit 151. The playback device 141 including the AV decoder unit 151 can play back an AV stream based on the PlayList in which the third STN_table described with reference to FIG. 35 has the main path and the sub path described above. .

  The AV decoder unit 151 of the playback device 141 is basically the AV decoder of the playback device 1 except that the PID filter 161 is provided instead of the PID filter 55 and the switch 162 is provided instead of the switch 59. The configuration is the same as that of the unit 26. Also, in the third STN_table () of FIG. 35, the 1st audio decoder 75-1 is for decoding the audio stream given by audio_stream_id, and the 2nd audio is for decoding the audio stream given by audio_stream_id2. This is a decoder 75-2.

  The file data read by the controller 21 is demodulated by a demodulation and ECC decoding unit (not shown), and error correction is performed on the demodulated multiplexed stream. The switch 32 selects the demodulated and error-corrected data for each stream type based on the control from the controller 21 and supplies the selected data to the corresponding buffers 51 to 54. Specifically, the switch 32 supplies background image data to the buffer 51 and supplies data read from the main clip (that is, the main clip AV stream) to the buffer 52 based on control from the controller 21. Then, the switch 32 is switched so that the data read from the sub clip (that is, the sub clip AV stream) is supplied to the buffer 53 and the Text-ST data is supplied to the buffer 54. The buffer 51 buffers background image data, the buffer 52 buffers data read from the main clip, the buffer 53 buffers data read from the sub clip, and the buffer 54 Buffer text-ST data.

  The stream data read from the buffer 52, which is the main Clip AV stream read buffer, is output to the subsequent PID (packet ID) filter 161 at a predetermined timing. The PID filter 161 distributes and outputs the input main Clip AV stream to the decoders of the elementary streams in the subsequent stage according to the PID (packet ID). That is, the PID filter 161 supplies the video stream to the video decoder 72, supplies the presentation graphics stream to the switch 57 serving as the supply source to the presentation graphics decoder 73, and supplies the interactive graphics stream to the interactive graphics decoder 74. The audio stream is supplied to the switch 162 serving as the supply source to the 1st audio decoder 75-1 and the 2nd audio decoder 75-2.

  The stream data read from the buffer 53, which is the sub-Clip AV stream read buffer, is output to the PID (packet ID) filter 56 at the subsequent stage at a predetermined timing. The PID filter 56 distributes the input sub Clip AV stream to the decoders of the elementary streams in the subsequent stage according to the PID (packet ID) and outputs the result. That is, the PID filter 56 supplies the presentation graphics stream to the switch 57 serving as the supply source to the presentation graphics decoder 73 and supplies the interactive graphics stream to the switch 58 serving as the supply source to the interactive graphics decoder 74. The audio stream is supplied to the switch 162 which is a supply source to the 1st audio decoder 75-1 and the 2nd audio decoder 75-2.

  The video stream distributed by the PID filter 161 is supplied to the video decoder 72 at the subsequent stage. The video decoder 72 decodes the video stream and outputs the decoded video data to the video plane generation unit 92.

  The switch 57 selects one of the presentation graphics stream included in the main Clip AV stream supplied from the PID filter 161 and the presentation graphics stream included in the sub Clip AV stream, and selects the selected presentation graphic. The stream is supplied to the presentation graphics decoder 73 at the subsequent stage. The presentation graphics decoder 73 decodes the presentation graphics stream, and supplies the decoded presentation graphics stream data to the switch 77 serving as a supply source to the presentation graphics plane generation unit 93.

  In addition, the switch 58 selects and selects one of the interactive graphics stream included in the main Clip AV stream supplied from the PID filter 161 and the interactive graphics stream included in the sub Clip AV stream. The interactive graphics stream is supplied to the subsequent interactive graphics stream decoder 74. That is, the interactive graphics stream that is simultaneously input to the interactive graphics decoder 74 is a stream separated from either the main Clip AV stream or the sub Clip AV stream. The interactive graphics decoder 74 decodes the interactive graphics stream and supplies the decoded interactive graphics stream data to the interactive graphics plane generating unit 94.

  Further, the switch 162 selects one of the audio stream included in the main Clip AV stream supplied from the PID filter 161 and the audio stream included in the sub Clip AV stream, and selects the selected audio stream. The signal is supplied to the 1st audio decoder 75-1 or the 2nd audio decoder 75-2 at the subsequent stage. Here, the audio stream simultaneously input to the 1st audio decoder 75-1 is a stream separated from either the main Clip AV stream or the sub Clip AV stream. Similarly, the audio stream input to the 2nd audio decoder 75-2 at the same time is a stream separated from either the main Clip AV stream or the sub Clip AV stream. For example, when the audio stream # 1 and the audio stream # 2 are included in the main Clip AV stream, the PID filter 161 filters the audio stream # 1 and the audio stream # 2 based on the PID of the audio stream. , Supplied to the switch 162.

  For example, the switch 162 selects the switch so that the audio stream # 1 supplied from the PID filter 161 is supplied to the 1st audio decoder 75-1, and the audio stream # 2 supplied from the PID filter 161 is selected as the 2nd audio decoder. Select the switch to supply to 75-2.

  The 1st audio decoder 75-1 decodes the audio stream and supplies the decoded audio stream data to the mixing processing unit 101. The 2nd audio decoder 75-2 decodes the audio stream and supplies the decoded audio stream data to the mixing processing unit 101.

  Here, when audio stream # 1 and audio stream # 2 are superimposed and reproduced (when two audio streams are selected as audio streams to be reproduced by the user), decoding is performed by the 1st audio decoder 75-1. The audio stream # 1 and the audio stream # 2 decoded by the 2nd audio decoder 75-2 are supplied to the mixing processing unit 101.

  The mixing processing unit 101 mixes (superimposes) the audio data from the 1st audio decoder 75-1 and the audio data from the 2nd audio decoder 75-2, and outputs the mixed data to the subsequent mixing processing unit 97. In the present embodiment, mixing (superimposing) the audio data output from the 1st audio decoder 75-1 and the audio data output from the 2nd audio decoder 75-2 is also referred to as synthesis. In other words, synthesis means to mix two audio data.

  The sound data selected by the switch 32 is supplied to the buffer 95 and buffered. The buffer 95 supplies the sound data to the mixing processing unit 97 at a predetermined timing. In this case, the sound data is sound effect data by menu selection or the like. The mixing processing unit 97 is the audio data mixed by the mixing processing unit 101 (audio data obtained by mixing the audio data output from the 1st audio decoder 75-1 and the audio data output from the 2nd audio decoder 75-2). The sound data supplied from the buffer 95 is mixed (superimposed or synthesized) and output as an audio signal.

  The processing of text subtitle (Text-ST) data, presentation graphics stream, background image data, and video data is basically the same as that described with reference to the AV decoder unit 26 of the playback apparatus 1. It is.

  In addition, the switches 57 to 58, the switch 162, and the switch 77 are based on a selection from the user instructed using the operation input unit 29 or a side to which a file including target data is supplied. Switch the switch. For example, when the audio stream is included only in the sub Clip AV stream file, the switch 162 switches the switch to the side that supplies the sub Clip AV stream file (the output of the PID filter 56).

  Next, playback processing in the playback device 141 of FIG. 36 will be described. When reproduction of a predetermined AV stream is instructed by the user using the operation input unit 29, the controller 21 is a local medium configured from the recording medium 11 via the storage drive 22 or an HDD (Hard Disk Drive) or the like. For example, based on the PlayList file described with reference to FIG. 9 and the PlayItem and SubPlayItem included in the PlayList, the main Clip AV stream, the sub Clip AV stream, and the text subtitle data (Text-ST data) are stored in the storage 24. read out.

  The controller 21 supplies the background image data to the buffer 51, supplies the main Clip AV stream data to the buffer 52, supplies the sub Clip AV stream data to the buffer 53, and supplies the Text-ST data to the buffer 54. Switch 32 is switched to supply.

  Then, the switch 32 is switched, the background image data is supplied to the buffer 51, the data of the main Clip AV stream is supplied to the buffer 52, the data of the sub Clip AV stream is supplied to the buffer 53, and the text subtitle data is The data is supplied to the buffer 54, and each of the buffers 51 to 54 buffers the supplied data.

  Then, the buffer 51 outputs the background image data to the background decoder 71, the buffer 52 outputs the stream data of the main Clip AV stream to the PID filter 161, and the buffer 54 converts the text subtitle data into the text subtitle composition 76. The buffer 53 outputs the stream data of the sub Clip AV stream to the PID filter 56.

  The PID filter 161 supplies the video stream to the video decoder 72, supplies the presentation graphics stream to the switch 57 serving as a supply source to the presentation graphics decoder 73, and supplies the interactive graphics stream to the interactive graphics decoder 74. The audio stream is supplied to the original switch 58, and the audio stream is supplied to the switch 162 which is the supply source to the first audio decoder 75-1 or the second audio decoder 75-2. That is, different PIDs are attached to the video stream, presentation graphics stream, interactive graphics stream, and audio stream, respectively.

  The PID filter 56 supplies the presentation graphics stream to the switch 57 serving as the supply source to the presentation graphics decoder 73, and supplies the interactive graphics stream to the switch 58 serving as the supply source to the interactive graphics decoder 74. The audio stream is supplied to the switch 162 which is a supply source to the 1st audio decoder 75-1 and the 2nd audio decoder 75-2.

Then, the switch 57, the switch 58, and the switch 162 following the PID filter 161 and the PID filter 56 select either the main clip or the sub clip based on the control from the controller 21. That is, the switch 57 selects the presentation graphics stream of the main Clip AV stream or the sub Clip AV stream supplied from the PID filter 161 and supplies it to the presentation graphics decoder 73 at the subsequent stage. In addition, the switch 58 selects the interactive graphics stream of the main Clip AV stream or the sub Clip AV stream supplied from the PID filter 161 and supplies the selected interactive graphics stream to the subsequent interactive graphics stream decoder 74.
Further, the switch 162 is an audio stream of the main Clip AV stream supplied from the PID filter 161 or the sub Clip AV stream supplied from the PID filter 56 (in this case, since audio is not switched, the audio stream # 1) is selected and supplied to the first audio decoder 75-1. When a command to switch the audio is issued by the user, the switch 162 supplies the audio stream of the main Clip AV stream to the 2nd audio decoder 75-2, or the audio stream of the sub Clip AV stream is 1st. Although it is supplied to the audio decoder 75-1 and the 2nd audio decoder 75-2, here, since the reproduction process before the sound is switched is described, the description thereof is omitted.

  The background decoder 71 decodes the background image data and outputs it to the background plane generation unit 91. The video decoder 72 decodes the video stream and outputs it to the video plane generation unit 92. The presentation graphics decoder 73 decodes the presentation graphics stream selected and supplied by the switch 57, and outputs this to the subsequent switch 77. The interactive graphics decoder 74 is selected and supplied by the switch 58. The interactive graphics stream is decoded and output to the subsequent interactive graphics plane generation unit 94. The Text-ST composition 76 decodes the text subtitle data. Code this and output it to the switch 77 at the subsequent stage.

  Then, the 1st audio decoder 75-1 decodes the supplied audio stream (audio stream # 1) selected by the switch 162, and outputs this to the mixing processing unit 101 at the subsequent stage. In the reproduction process in a state where the user has not instructed to switch the audio, the audio data is not output from the 2nd audio decoder 75-2, and therefore the mixing processing unit 101 is output from the 1st audio decoder 75-1. The audio data is supplied to the subsequent mixing processing unit 97 as it is.

  The switch 77 selects either the data from the presentation graphics decoder 73 or the Text-ST composition 76 and supplies the selected data to the presentation graphics plane generating unit 93. The background plane generating unit 91 generates a background plane based on the background image data supplied from the background decoder 71, and the video plane generating unit 92 is based on the video data supplied from the video decoder 72. The presentation graphics plane generating unit 93 generates a video plane based on the data from the presentation graphics decoder 73 or the text-ST composition 76 selected and supplied by the switch 77. The interactive graphics plane generating unit 94 generates an interactive graphics stream data supplied from the interactive graphics decoder 74. An interactive graphics plane is generated based on the data. The buffer 95 buffers the supplied sound data and supplies it to the mixing processing unit 97 at a predetermined timing.

  Then, the video data processing unit 96 combines the data of each plane and outputs it as video data. The mixing processing unit 97 mixes (synthesizes) audio data (audio data output from the mixing processing unit 101) and sound data and outputs the result.

  By such processing, the main Clip AV stream, the sub Clip AV stream, and the text subtitle data are referred to and reproduced by the main path and sub path included in the PlayList. Since the main path and the sub path are provided in the PlayList, and the sub path is a clip AV stream different from the clip AV stream file specified by the main path, the sub clip AV stream can be specified. Data included in a sub clip, which is a different clip from the indicated main clip, and data included in the main clip can be reproduced together (at the same timing).

  Next, processing in the playback device 141 when an instruction to switch audio is given will be described. First, a process in the case where the user instructs to switch audio for the audio stream # 1 will be described. The audio switching process for the audio stream # 2 will be described later. Note that these processes are, for example, processes that are executed during the playback process of the playback device 141.

  First, the controller 21 acquires an order list of audio stream numbers (which may be IDs). Specifically, the controller 21 refers to the STN_table () of the PlayItem, and acquires a list of the order of the audio stream numbers (ID) entered in the third STN_table () described with reference to FIG. . This process is executed when the reproduction process is started.

When a voice switching command is issued by the user using the operation input unit 29, the controller 21 receives a voice switching command for the audio stream # 1 from the user. At this time, the controller 21 may control an external display device (not shown) to display the stream number table on the external display device.
Since this processing is processing when audio switching for the audio stream # 1 is instructed, for example, the user instructs audio switching of A_SN = 1, A_SN = 2, A_SN = 3 in the stream number table. To do. Specifically, the user commands audio switching from audio 2 with A_SN = 1 to audio 1 with A_SN = 2. The controller 21 accepts the audio switching for the audio stream # 1 from the user.

  The controller 21 acquires the audio stream number next to the audio stream number being reproduced. For example, when an audio stream file corresponding to A_SN = 1 is being reproduced, the audio stream number corresponding to A_SN = 2 is acquired.

  The controller 21 has a function of reproducing the audio stream (audio stream # 1) corresponding to the acquired number by itself (reproducing apparatus 141) based on the contents described in stream_attribute () (FIG. 18). It is determined whether or not. For example, when type = 1 in Stream_entry (), the controller 21 determines whether or not it has a function of playing back an audio stream included in the main clip corresponding to the number acquired by the playback device 141. Since the audio stream corresponding to the acquired number is the audio stream # 1 decoded by the audio decoder 75-1, this process determines whether or not the audio decoder 75-1 can decode the audio stream # 1. In other words, it can be determined.

  When the controller 21 does not have a function of reproducing the audio stream corresponding to the acquired number, the controller 21 acquires the stream number next to the current stream number. That is, when the function of reproducing the audio stream of the current stream number is not provided, the stream number is skipped (not subject to reproduction), and the audio stream having the function of reproducing itself (audio stream # 1) The process of acquiring the next stream number is repeated until the number of) is acquired.

On the other hand, when the controller 21 has a function of playing back an audio stream corresponding to the acquired number, the controller 21 examines a Clip including the audio stream corresponding to the acquired number.
For example, based on the type of Stream_entry (), the controller 21 checks whether the audio stream corresponding to the acquired number is in the main clip (type = 1) or in the sub clip (type = 2). For example, in Stream_entry (), if type = 1, the controller 21 confirms that the audio stream corresponding to the acquired number is in the main clip. Further, for example, when type = 2, the controller 21 confirms that the audio stream corresponding to the acquired number is in the sub clip.

  Then, the controller 21 specifies a desired audio stream, specifically, a desired audio stream in the main clip or sub clip to which the stream corresponding to the acquired number belongs. For example, when it is confirmed that the audio stream corresponding to the acquired number is included in the main clip, the controller 21 determines that the main clip is based on the information (ref_to_stream_PID_of_mainClip) entered with type = 1 of Stream_entry (). A predetermined audio stream is identified. Further, for example, when it is confirmed that the audio stream corresponding to the acquired number is included in the sub clip, the controller 21 includes the information (ref_to_SubPath_id, ref_to_SubClip_entry_id, ref_to_stream_PID_of_SubClip) entered with type = 2 of Stream_entry (). Based on this, a predetermined audio stream in the sub clip is specified. This identified audio stream becomes the audio stream # 1 decoded by the audio decoder 75-1.

  Then, the controller 21 instructs the storage drive 22 to read the audio stream specified from the Clip AV stream (main Clip AV stream or sub Clip AV stream) included in the Clip in which the desired audio stream is multiplexed, Based on this instruction, the storage drive 22 reads a desired audio stream from the target Clip and supplies it to the AV decoder unit 151. Alternatively, the controller 21 reads out a desired audio stream from a Clip AV stream (main Clip AV stream or sub-Clip AV stream) recorded in the local storage 24 and multiplexed with a desired audio stream, and an AV decoder To the unit 151. Then, the controller 21 supplies the read desired audio stream (that is, audio stream # 1) to the audio decoder 75-1 of the AV decoder unit 151, and also decodes the audio stream so as to decode the audio stream. To instruct.

  The audio decoder 75-1 of the AV decoder unit 151 decodes the audio stream # 1 (the audio stream that is instructed to be read in the process of step S58), and outputs the decoded audio stream # 1 to the mixing processing unit 101. Here, when the audio stream # 2 is decoded by the audio decoder 75-2, the mixing processing unit 101 mixes the decoded audio stream # 1 and the audio stream # 2, and performs a subsequent mixing processing unit. 97. On the other hand, when the audio stream # 2 is not decoded by the audio decoder 75-2, the mixing processing unit 101 supplies the decoded audio stream # 1 to the subsequent mixing processing unit 97 as it is. The mixing processing unit 97 decodes the audio data supplied from the mixing processing unit 101 (audio data obtained by decoding only the audio stream # 1 or audio stream # 1 and audio stream # 2 respectively) Data) and sound data output from the buffer 95 are mixed and output as an audio signal.

  Next, a process in the case where the user instructs to switch audio for the audio stream # 2 will be described. This process is also a process executed during the reproduction process. In general, when the audio stream # 2 is reproduced, since the audio stream # 1 is also reproduced, this processing is executed in parallel with the reproduction processing of the audio stream # 1, for example, and the audio stream # 1 is reproduced. If # 2 is not selected by the user, only the reproduction process of the audio stream # 1 is executed.

  First, the controller 21 refers to the STN_table () of the PlayItem, and obtains a list of the order of the audio stream numbers (ID) entered in the third STN_table () described with reference to FIG. This process is executed when the reproduction process is started.

  When a voice switching command is issued by the user using the operation input unit 29, the controller 21 receives a voice switching command for the audio stream # 2 from the user. At this time, the controller 21 may control an external display device (not shown) to display the stream number table in FIG. 26 on the external display device. Since this process is a process in the case where the audio switching for the audio stream # 2 is instructed, for example, the user instructs the audio switching corresponding to A_SN2 = 1 and A_SN2 = 2 in the stream number table. . The controller 21 accepts the audio switching for the audio stream # 2 from the user. The controller 21 acquires the audio stream number next to the audio stream number being reproduced.

  Then, based on the contents described in stream_attribute () (FIG. 18), the controller 21 has a function of playing back the audio stream (audio stream # 2) corresponding to the acquired number by itself (the playback device 141). It is determined whether or not it has. Since the audio stream corresponding to the acquired number is the audio stream # 2 decoded by the audio decoder 75-2, this process determines whether or not the audio decoder 75-2 can decode the audio stream # 2. In other words, it can be determined.

  It is assumed that the audio stream # 1 decoded by the 1st audio decoder 75-1 is more scalable and multi-stream than the audio stream # 2 decoded by the 2nd audio decoder 75-2. For example, when the 1st audio decoder 75-1 has higher performance than the 2nd audio decoder 75-2 (the audio stream # 1 is a normal movie sound, the audio stream # 2 is a director's comment on the movie, etc.) In the case of the audio stream # 2, the audio stream # 2 decoded by the 2nd audio decoder 75-2 is likely to be restricted to some extent, and the stream that cannot be decoded by the audio decoder 75-2 is the audio stream # 2. Likely not set as This process may be omitted.

  When the controller 21 does not have a function of reproducing the audio stream corresponding to the acquired number, the controller 21 acquires the stream number next to the current stream number. That is, if the audio stream having the current stream number is not reproduced, the stream number is skipped (not subject to reproduction), and the audio stream (audio stream # 2) having the function of reproducing itself The process of acquiring the next stream number is repeated until the number of) is acquired.

  On the other hand, when the controller 21 has a function of playing back an audio stream corresponding to the acquired number, the controller 21 determines whether the audio stream corresponding to the acquired number is in the main clip based on the type of Stream_entry () (type = 1). And whether it is in the sub clip (type = 2). In Stream_entry (), if type = 1, the controller 21 confirms that the audio stream corresponding to the acquired number is in the main clip. In Stream_entry (), if type = 2, the controller 21 confirms that the audio stream corresponding to the acquired number is in the sub clip.

  Then, the controller 21 specifies a desired audio stream in the main clip or sub clip to which the stream corresponding to the acquired number belongs. For example, when it is confirmed that the audio stream corresponding to the acquired number is included in the main clip, the controller 21 determines that the main clip is based on the information (ref_to_stream_PID_of_mainClip) entered with type = 1 of Stream_entry (). A predetermined audio stream is identified. Further, for example, when it is confirmed that the audio stream corresponding to the acquired number is included in the sub clip, the controller 21 includes the information (ref_to_SubPath_id, ref_to_SubClip_entry_id, ref_to_stream_PID_of_SubClip) entered with type = 2 of Stream_entry (). Based on this, a predetermined audio stream in the sub clip is specified. This identified audio stream becomes the audio stream # 2 decoded by the audio decoder 75-1.

Then, the controller 21 instructs the storage drive 22 to read out the Clip AV stream (main Clip AV stream or sub Clip AV stream AV stream) in which the desired audio stream is multiplexed. The storage drive 22 responds to this instruction. Based on this, the target Clip AV stream is read and supplied to the AV decoder 151.
Alternatively, the controller 21 reads out a Clip AV stream (a main Clip AV stream or a sub Clip AV stream) in which a desired audio stream is multiplexed from the local storage 24 and supplies it to the AV decoder 151. Then, the controller 21 instructs the audio decoder 75-2 of the AV decoder unit 151 to decode a desired audio stream (that is, audio stream # 2) included in the read Clip.

  The AV decoder unit 151 decodes the audio stream (audio stream # 2) and outputs the audio. Specifically, the audio decoder 75-2 of the AV decoder unit 151 decodes the audio stream # 2 (the audio stream that is instructed to be read in the process of step S78), and outputs the decoded audio stream # 2 to the mixing processing unit 101. In this process, the audio stream # 2 is decoded by the audio decoder 75-2. This means that the audio stream # 1 is also decoded by the audio decoder 75-1 in the process, and the decoded audio stream # 1 is mixed. Since it is output to the processing unit 101, the mixing processing unit 101 mixes the decoded audio stream # 1 and audio stream # 2 and supplies the mixed audio stream # 1 to the subsequent mixing processing unit 97. The mixing processing unit 97 receives the audio data supplied from the mixing processing unit 101 (audio data obtained by decoding the audio stream # 1 and the audio stream # 2 respectively) and the sound data output from the buffer 95. , Mixing processing and output as an audio signal.

  In this way, two audio streams can be selected, mixed and reproduced. That is, when there is one audio stream to be reproduced, the switch 59 selects to supply the audio stream supplied from the PID filter 55 or the PID filter 56 to the 1st audio decoder 75-1, and the audio stream to be reproduced is If there are two, the switch 59 selects to supply the audio stream supplied from the PID filter 55 or PID filter 56 to the 2nd audio decoder 75-2.

That is, the controller 21 can control switching of audio (audio) based on STN_table () of PlayItem, and selects only a stream for which the controller 21 has a playback function by referring to the stream_attribute of STN_table (). Thus, playback switching can be controlled. Furthermore, the controller 21 can control to superimpose and reproduce two audio streams based on STN_table (). The controller 21 mixes and reproduces one main audio stream # 1 and an audio stream # 2 that is the same type of stream (audio stream) that is reproduced in synchronization with the audio stream # 1. Can be controlled.

  In this process, the audio is switched based on the audio stream number. However, the audio may be switched based on the audio stream ID (audio_stream_id). In this case, the audio stream ID is obtained by subtracting 1 from the audio stream number.

  Further, the processing when the third STN_table is used is not limited to the audio stream, and can be similarly applied to various stream files as long as the stream types are the same. For example, this configuration can also be applied to the case of picture-in-picture display in which two video streams (streams whose stream types are video streams) are combined and played back. As described above, according to the present invention, two stream files of the same type can be synthesized and reproduced.

  That is, this configuration also displays the AV stream video stream (video stream # 1) referred to in the main path as the main screen, as will be described later with reference to FIG. In addition, it can be used for so-called PinP (picture-in-picture) display in which a video stream (video stream # 2) referred to in the video path of the main path is combined and displayed as a child screen.

  As described above, the playback apparatus 141 also has a main path and a sub path in the PlayList when preparing audio and subtitles in a stream or data file different from the main AV stream, as in the playback apparatus 1. Since the configuration is adopted, an operation such as audio switching or subtitle switching by the user can be selected from another stream or another data file different from the main ClipAV stream.

  In addition, since the STN_Table that defines the type of data that is multiplexed in the AV stream file and the data that is referenced by the Sub Path is provided in the PlayItem of the main path, a stream with higher expandability is provided. Can be realized.

  Furthermore, as with the playback apparatus 1, the playback apparatus 141 can sequentially select and play back only the streams corresponding to the functions of the playback apparatus 141 by referring to the stream_attribute of the STN_table ().

  The case where the playback device 141 described above performs playback processing with reference to the third STN_table of FIG. 35 is summarized as follows.

  The playback device 141 has a main path that is a main playback path indicating the position of the main Clip AV stream file including at least one stream, and each of the sub Clip AV stream files that are played back in accordance with the timing of the main Clip AV stream file. A PlayList is obtained as reproduction management information composed of a plurality of sub-paths indicating positions. Then, the playback device 141 accepts selection of a stream to be played based on STN_table () for selecting a stream to be played back included in the PlayList. STN_table () is a predetermined type of stream (for example, audio stream # 1) of the main Clip AV stream file and the same type of stream file (for example, audio stream #) that is played in accordance with the playback timing of the stream. 2), the selection of the stream to be played back is accepted based on this STN_table () (STN_table () indicated by PlayItem).

  Here, when two streams of the same type (audio stream) are selected as playback streams, the playback device 141 reads the two selected stream files, and the mixing processing unit 101 of the playback device 141 reads these two stream files. Two audio streams (decoded audio streams) are synthesized, and the mixing processing unit 97 (AV decoder unit 151) reproduces (outputs) the synthesized stream.

  Thereby, the two audios selected (selected by switching) are mixed (synthesized) and reproduced.

  Further, as in the case of the playback apparatus 1 described above, since the main path and sub path are included in the play list and the clips having different main paths and sub paths are referred to, the stream can be extended. For example, as described with reference to FIG. 8, the configuration is such that a plurality of files can be referred to by one Sub Path, so that the user can select from a plurality of different streams.

  Further, in the Play item of the Main Path, the attached data (for example, an audio stream) multiplexed (included) in the AV stream file referenced by the Main Path and the attached data referenced by the Sub Path are defined. Since the STN_table () of FIG. 35 is provided as the table to be performed, a more highly scalable stream can be realized. In addition, Sub Path can be easily extended by entering STN_table ().

  Further, since the stream_attribute () of FIG. 18 which is the attribute information of the stream is provided in STN_table (), the playback device 141 can determine whether or not the selected stream can be played back, and has a playback function. Only the streams that have them can be selected and played back.

  Furthermore, in STN_table () (FIG. 35), two same type stream files (here, audio stream files) are defined, and two audio decoders (1st audio decoder 75) are added to the playback device 141 of FIG. -1 and 2nd audio decoder 75-2) and a mixing processing unit 101 for synthesizing (mixing) audio data decoded by the two audio decoders are provided. Can be played at the same time.

  Similarly to the case of the playback apparatus 1 described above, the Sub Path includes the Sub Path_type indicating the Sub Path type (type such as audio and text subtitles) described with reference to FIG. 11, and the Sub Clip referred to by the Sub Path. 12 is included, and SubPlayItem_IN_time in FIG. 12 indicating the In point of the Clip AV stream referenced by the Sub Path and SubPlayItem_OUT_time in FIG. 12 indicating the Out point are included, so that the data referred to by the Sub Path is accurately identified. Can be specified.

  Further, as in the case of the playback apparatus 1 described above, in the Sub Path, sync_PlayItem_id in FIG. 12 which is designation information for designating an AV stream file on the Main Path for playing back the Sub Path to the Main Path at the same timing. (For example, sync_PlayItem_id in FIGS. 7 and 8) and sync_start_PTS_of_PlayItem (for example, FIG. 7 and FIG. 7), which is the time on the main path at which the in point of the data referred to by the sub path starts synchronously on the time axis of the main path. 8 sync_start_PTS_of_PlayItem), the data (file) referenced by the Sub Path can be reproduced in synchronization with the main ClipAV stream file referenced by the Main Path, as shown in FIGS.

  Further, when two audio streams are mixed and played back, the 1st audio decoder 75-1 of the playback device 141 decodes the audio stream # 1, and the 2nd audio decoder 75-2 decodes the audio stream # 2. It may be further determined whether or not the codec (encoding) method is the same (for example, Dolby AC-3 audio in FIG. 19). This is because even if the 1st audio decoder 75-1 and the 2nd audio decoder 75-2 are in the middle of decoding the audio stream, if the codec system is the same, the data system in the middle of decoding is the same. This is because two streams decoded up to the same stage in the middle can be mixed by the mixing processing unit 101.

  Note that the data supplied to the AV decoder unit 151 of the playback device 141 described with reference to FIG. 36 is recorded on a recording medium 11 such as a DVD (Digital Versatile Disc) as in the playback device 1 described above. May be data stored in the local storage 24 composed of a hard disk or the like, data downloaded via the network 2, or a combination of these. It may be processed data.

  Also, when the playback process is performed with reference to the third STN_table described with reference to FIG. 35, the audio stream # 1 decoded by the 1st audio decoder 75-1 of the playback device 141 of FIG. The audio stream # 2 decoded by the audio decoder 75-2 is configured to be selectable by the user separately. However, when the combination of the audio stream # 1 and the audio stream # 2 is defined and mixed reproduction is performed, A configuration may be employed in which the user selects from this combination. That is, a combination of the audio stream # 1 and the audio stream # 2 that can be mixed and reproduced may be defined.

  Hereinafter, a case where a combination of streams to be reproduced is defined will be described.

  FIG. 37 is a diagram illustrating a fourth example (fourth STN_table ()) of syntax of STN_table () when defining a combination of audio stream # 1 and audio stream # 2. The playback process executed with reference to the fourth STN_table () can be executed by the playback device 141 described with reference to FIG. In the figure, the same parts as those in FIG. 35 are repeated, and the description thereof is omitted.

In FIG. 37, audio_stream_id2 is given from 0 to one audio elementary stream identified for each stream_entry () in order in the for loop of audio stream ID2 (audio_stream_id2). At this time, Combination_of_Primary_and_Secondary is newly defined for audio_stream_id2 in addition to stream_entry () and stream_attribute (). Combination_of_Primary_and_Secondary is information defining a combination of audio stream # 1 (Primary) and audio stream # 2 (Secondary).
That is, information indicating which audio stream # 1 can be combined with the audio stream # 2 specified by the audio stream ID2 (audio_stream_id2) is defined in the for loop of the audio stream ID2. In other words, the audio stream # 2 that can be combined with the audio stream # 1 is defined by Combination_of_Primary_and_Secondary. Thus, instead of defining the audio stream ID2 to be combined in the for loop of the audio stream ID1, the audio stream ID1 and the audio stream ID2 are individually defined, and the combination is defined in the for loop of the audio stream ID2. Thus, even when the registerable number of the audio stream ID1 is determined, the registerable number can be used effectively by not counting the combination registration as one.

  As described with reference to FIG. 35, audio stream number 2 (audio_stream_number2) may be used instead of audio stream ID2 (audio_stream_id2). In this case, audio_stream_number2 is given from 1 instead of 0. That is, audio_stream_number2 is obtained by adding 1 to the value of audio_stream_id2. The audio stream number 2 is an audio stream number 2 that can be seen by the user and used for audio switching.

  Here, an example of the bitmap syntax of Combination_of_Primary_and_Secondary in FIG. 37 will be described with reference to FIG.

  In the example of FIG. 38, the bitmap syntax of Combination_of_Primary_and_Secondary is 32 bits. In FIG. 38, audio_stream_id indicates audio_stream_id (audio stream ID) defined in the third STN_table () of FIG. 37, and the value of the indicator is an audio stream specified by the corresponding audio_stream_id2 (audio stream ID2). This is a flag indicating audio_stream_id that identifies an audio stream # 1 that can be combined with # 2. That is, the audio_stream_id of the audio stream # 1 that can be combined with the audio stream # 2 given by the audio_stream_id2 can be specified by the audio_stream_id and the indicator.

  That is, there is a Combination_of_Primary_and_Secondary for each audio_stream_id2 so that Combination_of_Primary_and_Secondary is defined in the for_loop of audio_stream_id2 in FIG. 37, and a combination is possible for audio stream # 2 specified by a specific audio_stream_id2 Audio stream # 1 is specified by audio_stream_id indicated by the indicator.

  Specifically, when the value of the Indicator is 0, it indicates that the audio_stream_id2 does not correspond to the audio stream # 1 specified by the audio_stream_id. When the value of the indicator is 1, the audio_stream_id2 is specified by the audio_stream_id It corresponds to the audio stream # 1.

  That is, in the example of FIG. 38, audio stream # 2 specified by audio_stream_id2 (audio stream ID2) is combined with two audio streams (audio stream # 1) specified by audio_stream_id = 0 and audio_stream_id = 1. It has been shown to be possible.

  In FIG. 38, the audio stream # 1 that can be combined is specified by audio_stream_id, but an audio stream number may be used instead of audio_stream_id. In this case, audio_stream_id is given from 1 instead of 0.

  As shown in FIG. 38, Combination_of_Primary_and_Secondary is defined by 32-bit bitmap syntax, so that it is possible to define audio streams # 2 that can be combined with 32 audio streams # 1.

  According to FIGS. 37 and 38, a combination of audio stream # 1 and audio stream # 2 can be defined using audio_stream_id, audio_stream_id2, and Combination_of_Primary_and_Secondary. That is, since it is not necessary to define a contradictory combination of audio streams, the user can always select reproduction of a reproducible combination of audio streams.

  A specific example will be described with reference to FIG. FIG. 39 is a diagram illustrating an example of a stream number table representing a relationship between an audio signal provided to a user and a caption signal. In the figure, the description of the subtitle signal (sub-picture) is the same as that in FIG.

  In FIG. 39, A_SN is given to each of the audio streams # 1 (audio streams entered with audio_stream_id) entered in the fourth STN_table () of PlayItem configuring the Main Path of the PlayList.

  Specifically, audio 2 is given to A_SN = 1, audio 1 is given to A_SN = 2, and audio 3 is given to A_SN = 3.

  At this time, audio 4 (audio_stream_id2 = 0, that is, audio 4 with audio stream number 2 being 1) can be combined with audio 2 with A_SN = 1. For audio 1 with A_SN = 2, audio 4 (audio_stream_id2 = 0, ie, audio 4 with audio stream number 2 being 1) and audio 5 (audio_stream_id2 = 1, ie, audio stream number 2 being 2) It can be combined with some audio 5).

  Specifically, in the case of audio 4 (audio 4 with audio_stream_id2 = 0), the combination_of_Primary_and_Secondary bitmap syntax in FIG. 38 has both the audio_stream_id = 0 and the indicator values for audio_stream_id = 1, and audio_stream_id = 2 to audio_stream_id The values of the indicators up to = 31 are all 0. Also, in the case of audio 5 (audio 5 with audio_stream_id2 = 1), the bitmap syntax of Combination_of_Primary_and_Secondary in FIG. 38 is such that only the indicator value for audio_stream_id = 1 is 1, audio_stream_id = 0, audio_stream_id = 2 to audio_stream_id = 31 The values of the indicators up to are all 0.

  The user selects the audio stream # 1 to be reproduced from the audio streams to which A_SN is given, and the audio stream # 2 to be mixed with the selected audio stream # 1 is an audio stream to which A_SN is given. Select from the ones displayed on the right side. In the case of the example of FIG. 39, when audio 2 (A_SN = 1) is selected as the audio stream # 1, the audio stream # 2 that can be mixed is only the audio 4 displayed on the right side of the audio 2 diagram. Therefore, it is selected whether or not to perform mixing reproduction. In FIG. 39, the audio 4 corresponding to the audio 2 with A_SN = 1 is “x”, that is, the state in which mixing reproduction is not selected. The user can set the state where mixing reproduction of the audio 2 and the audio 4 is selected by switching the “x” to “◯”.

  For example, in the case of FIG. 39, when audio 1 (A_SN = 2) is selected as the audio stream # 1, the audio stream # 2 that can be mixed is the audio displayed on the right side of the audio 2 in the figure. 4 and audio 5 can be selected. Therefore, the user selects with which audio stream # 2 the audio stream # 1 is to be reproduced. In FIG. 39, the audio 4 and the audio 5 corresponding to the audio 1 of A_SN = 2 are “x”, that is, a state in which mixing reproduction is not selected. The user can set the state where mixing reproduction of audio 2 and audio 4 or audio 2 and audio 5 is selected by switching “x” to “o”.

  As a specific example of this playback selection, when audio 2 with A_SN = 1 is selected and the user commands switching of audio, a combination of audio 2 with audio 4 and audio 4 is selected. When the playback is switched to playback (the audio 4 corresponding to the audio 2 in FIG. 39 becomes “◯”) and the user instructs to switch the audio further, the audio is played back in the audio 1 of A_SN = 2 (in FIG. 39). When the audio 4 corresponding to the audio 2 is switched to “x”) and the user further instructs to switch the audio, the audio is reproduced as a combination of the audio 1 and the audio 4 of A_SN = 2 (FIG. 39). The audio 4 corresponding to the audio 1 is switched to “◯”. Further, when the user instructs to switch the audio further, the audio is reproduced as a combination of the audio 1 and the audio 5 of A_SN = 2 (the audio 4 corresponding to the audio 1 in FIG. Becomes “○”.

  The user can switch the selection of the audio stream to be reproduced in this way, and select reproduction of a combination of audio 1 and audio 5 of A_SN2 as shown in FIG. 40, for example. Thereby, audio 1 (audio stream # 1) and audio 5 (audio stream # 2) can be mixed and reproduced.

  Thus, by defining the combination of audio stream # 1 and audio stream # 2 using audio_stream_id and audio_stream_id2 and Combination_of_Primary_and_Secondary, it is not necessary to define conflicting combinations, and the user can , A combination of audio streams that can always be played back can be selected. That is, in the example of FIG. 39, only the selectable items can be provided to the user by not defining the audio 2 + audio 5 that is a combination of contradictory audio streams.

  In other words, in the fourth STN table () (FIG. 37) in PlayItem (), the audio stream # 1 decoded by the 1st audio decoder 75-1 in FIG. 36 and the 2nd audio decoder 75-2 are decoded. Since the combination with the audio stream # 2 is defined using Combination_of_Primary_and_Secondary, the two audio streams can be mixed and reproduced. That is, streams of the same type (in this example, an audio stream) can be mixed (superimposed or combined) from a plurality of types of streams and played back simultaneously.

  With this configuration, the user does not need to individually select two audio streams to be mixed, and can select an audio stream to be reproduced from a combination of the two audio streams. .

  Next, when a combination of audio stream # 1 and audio stream # 2 is defined and mixed reproduction is performed, that is, the user is allowed to select reproduction from a combination of audio stream # 1 and audio stream # 2. A second example of the case will be described with reference to FIG. In other words, FIG. 41 is a second example in which an audio stream that can be mixed with audio stream # 1 is defined as audio stream # 2.

  FIG. 41 is a diagram illustrating another example of the fifth example (fifth STN_table ()) of the syntax of STN_table () when defining a combination of the audio stream # 1 and the audio stream # 2. The playback process executed with reference to the fifth STN_table () can also be executed in the playback device 141 described with reference to FIG. In the figure, the same parts as those in FIG. 35 are repeated, and the description thereof is omitted.

  In FIG. 41, audio_stream_id2 is given from 0 to one audio elementary stream whose recording position is specified for each stream_entry () in order in the for loop of audio stream ID2 (audio_stream_id2). At this time, number_of_combinations is given to audio_stream_id2, and in the for statement after number_of_combinations, audio_stream_id is given by the number of number_of_combinations. The for statement after number_of_combinations and number_of_combinations is information defining the combination of audio stream # 1 and audio stream # 2, and the number of audio streams # 1 that can be combined with audio stream # 2 specified by audio_stream_id2 is , Number_of_combinations. Then, audio_stream_id that identifies the audio stream # 1 that can be combined with the audio stream # 2 identified by audio_stream_id2 is defined in a for statement after number_of_combinations.

  That is, unlike FIG. 37, in the for statement of the audio stream ID2 (audio_stream_id2) of the fifth STN_table () of FIG. 41, the audio stream # 2 is compared with the audio stream # 2 specified by the audio stream ID2. The number that can be combined with 1 is defined by number_of_combinations, and the audio_stream_id that identifies the target audio stream # 1 is directly defined in the for statement after number_of_combinations.

  Thus, in the fifth STN_table (), information indicating which audio stream # 1 can be combined with the audio stream # 2 specified by the audio stream ID2 (audio_stream_id2) is the audio stream ID2. Defined in a for statement after number_of_combinations. That is, the audio stream # 2 that can be combined with the audio stream # 1 is defined by the for statement after the number_of_combinations of the audio stream ID2.

  As in the case described with reference to FIG. 35, audio stream number 2 (audio_stream_number2) may be used instead of audio stream ID2 (audio_stream_id2). Also, an audio stream number (audio_stream_number) may be used instead of the audio stream ID (audio_stream_id). That is, audio_stream_id defined in the for statement after number_of_combinations may be an audio stream number.

  According to FIG. 41, the combination of audio stream # 1 and audio stream # 2 can be defined using number_of_combinations and the subsequent for statement. That is, since it is not necessary to define a contradictory combination of audio streams, the user can always select reproduction of a reproducible combination of audio streams.

  A specific example of the stream number table in this case is the same as that described with reference to FIGS.

  That is, when FIG. 39 corresponds to the fifth STN_table () of FIG. 41, audio stream # 1 that can be combined with audio 4 (audio_stream_id2 = 0, that is, audio 4 with audio stream number 2 being 1). As described above, audio 2 (audio_stream_id = 0) with A_SN = 1 and audio 1 (audio_stream_id = 1) with A_SN = 2 are defined. In this case, the number of number_of_combinations corresponding to audio_stream_id2 = 0 in FIG. 41 is two, and the defined audio_stream_id is audio_stream_id = 0 and audio_stream_id = 1. Also, for audio 5 (audio_stream_id2 = 1, ie, audio 5 with audio stream number 2 being 2), audio 1 (audio_stream_id = 1) with A_SN = 2 is defined as an audio stream # 1 that can be combined. Will be. In this case, the number_of_combinations corresponding to audio_stream_id2 = 1 in FIG. 41 is one, and the defined audio_stream_id is audio_stream_id = 1.

  The user switches the selection of audio reproduction shown in FIG. 39, and selects reproduction of a combination of audio 1 and audio 5 with A_SN = 2, for example, as shown in FIG. Thereby, audio 1 (audio stream # 1) and audio 5 (audio stream # 2) can be mixed and reproduced.

  In this way, by defining the combination of audio stream # 1 and audio stream # 2 using number_of_combinations in FIG. 41 and the subsequent for statement, it is not necessary to define contradictory combinations. The user can select a combination of audio streams that can always be reproduced. That is, in the example of FIG. 39, only the selectable items can be provided to the user by not defining the audio 2 + audio 5 that is a combination of contradictory audio streams.

  In other words, in the fifth STN table () (FIG. 41) in PlayItem (), the audio stream # 1 decoded by the 1st audio decoder 75-1 in FIG. 36 and the 2nd audio decoder 75-2 are decoded. Since the combination with the audio stream # 2 is defined using number_of_combinations and the subsequent for statement, the two combinable audio streams can be mixed and reproduced. That is, streams of the same type (in this example, an audio stream) can be mixed (superimposed or combined) from a plurality of types of streams and played back simultaneously.

  Further, by using the fifth STN table (), the user does not need to individually select two audio streams to be mixed, and selects an audio stream to be reproduced from a combination of the two audio streams. be able to.

  Next, FIG. 42 is a diagram illustrating still another example of the sixth example (sixth STN_table ()) of the syntax of STN_table () when a combination of the audio stream # 1 and the audio stream # 2 is defined. It is. The playback process executed with reference to the sixth STN_table () can also be executed by the playback device 141 described with reference to FIG. In the figure, the same parts as those in FIGS. 35 and 41 are repeated, and the description thereof is omitted.

  In the for loop of the audio stream ID (audio_stream_id) in the example of FIG. 42, audio_stream_id is assigned from 0 to one audio stream # 1 that is specified for each stream_entry () in order, and is arranged after that. The stream_attribute () is the same as in the example of FIG.

  However, the for loop of the audio stream ID (audio_stream_id) in the example of FIG. 42 further includes the following information that defines a combination of the audio stream # 1 and the audio stream # 2. That is, if number_of_audio_stream2_entries is not 0 (if (number_of_audio_stream2_entries! = 0)), number_of_audio_stream2_ref_entries is given. number_of_audio_stream2_ref_entries indicates the number of audio streams # 2 that can be combined with the audio stream # 1 specified by audio_stream_id. Then, audio_stream2_id_ref for specifying an audio stream # 2 that can be combined with the audio stream # 1 specified by audio_stream_id is defined by a for statement after number_of_audio_stream2_ref_entries.

  As in the other examples, the audio stream number 2ref (audio_stream_number2_ref) may be used instead of the audio stream ID2ref (audio_stream_id2_ref). Also, an audio stream number (audio_stream_number) may be used instead of the audio stream ID (audio_stream_id).

  In this way, the combination of the audio stream # 1 and the audio stream # 2 can also be defined using the sixth STN_table () in the example of FIG. That is, since it is not necessary to define a contradictory combination of audio streams, the user can always select reproduction of a reproducible combination of audio streams.

  By the way, as described above, the application format of data recorded on the recording medium mounted on the playback device of the present invention or on the local storage of the playback device has two layers, PlayList and Clip. A PlayList is composed of a main path (Main Path) created by a sequence of one or more PlayItems (by consecutive PlayItems) and a parallel (in parallel) to the Main Path (by a sequence of one or more SubPlayItems (non- One or more sub paths (sub paths) created by sub play items that may be continuous or may be continuous may be included.

  Therefore, for example, as shown in FIG. 43, an AV stream (a video stream 311 described as Primary Video in FIG. The audio stream 312 corresponding to the audio stream 312) described in the above is reproduced, and the AV stream (denoted as Secondary Video in FIG. 43) is referred to in the sub-screen 302 arranged on the full screen 301 by the sub path. A method of reproducing content different from the AV stream referred to in the main path, such as a commentary of a movie director corresponding to the video stream 313 and the audio stream 314) described as Secondary Audio It becomes possible to apply.

  In other words, it is possible to apply a technique in which different contents are simultaneously reproduced on the full screen (parent screen) 301 and the child screen 302, respectively.

  Such a method is called a picture-in-picture method.

  Note that the reproduction of the audio corresponding to the audio stream 312 described as Primary Audio and the audio corresponding to the audio stream 314 described as Secondary Audio is performed on these two audio streams as described above. This is the sound output obtained as a result of the mixing process.

  In addition, as described above, the actual reproduction target is audio or an image corresponding to each stream. However, in the following description, there are cases where each stream is used for the sake of simplicity. That is, an expression such as playing a predetermined stream may be used.

  In order to realize an application to which the picture-in-picture method is applied, for example, the following syntax is required. That is, for example, in this embodiment, a video stream defined by video_stream_id in the STN_table (hereinafter referred to as a primary video stream. For example, in the example of FIG. 43, a video stream 311 described as Primary Video is represented as a primary video stream. The number of streams 311) is one. In view of this, if there is a syntax that defines a combination of streams that can be reproduced simultaneously with the primary video stream, it is possible to realize a picture-in-picture technique, that is, an application that applies picture-in-picture. .

  Here, for example, in the present embodiment, the following first to fourth types of streams are prepared as streams that can be reproduced simultaneously with the primary video stream by the picture-in-picture method.

  The first type of stream is an audio stream defined by audio_stream_id in the STN_table, that is, the above-described audio stream # 1 (hereinafter referred to as a primary audio stream. For example, in the example of FIG. 43, Primary Audio is described. The audio stream 312 is referred to as a primary audio stream 312).

  The second type of stream is a video stream defined by video_stream_id2 in the STN_table (hereinafter referred to as a secondary video stream. For example, in the example of FIG. 43, a video stream 313 described as Secondary Video is a secondary video stream. 313).

  The third type of stream is an audio stream defined by audio_stream_id2 in the STN_table, that is, the above-described audio stream # 2 (hereinafter referred to as a secondary audio stream. For example, in the example of FIG. 43, it is described as Secondary Audio. Audio stream 314 is referred to as a secondary audio stream 314).

  The fourth type of stream is a text subtitle stream (txtST) referenced in a sub-path, a stream (PG, Presentation Graphics stream) obtained by run-length encoding a bitmap subtitle such as a DVD sub-picture, and the like. Say. For example, in the example of FIG. 43, the stream 315 described as Subtitle is an example of a fourth type of stream. Hereinafter, the fourth type of stream, that is, txtST, PG, and the like are collectively referred to as a subtitle stream. Note that the playback position of the subtitle stream (the display position of subtitles and the like) does not have to be in the child screen, and may be any position on the entire screen. For this reason, in the example of FIG. 43, the subtitle stream 315 is drawn so as to protrude from the child screen 302.

  As described above, in order to realize an application to which the picture-in-picture method is applied, for example, the definition of the combination of streams that can be simultaneously reproduced in combination with the primary video stream, that is, the secondary video stream, the primary audio stream, the secondary It is necessary to define a combination of an audio stream and a subtitle stream.

  Furthermore, by defining a plurality of combinations, not only switching of the primary video stream and primary audio stream reproduced on the full screen 301 but also switching of the secondary video stream reproduced on the sub-screen 302 and switching of the subtitle stream are performed. Can also be easily performed. However, an example of the switching operation will be described later with reference to FIGS. 46 and 47.

  Such a combination of streams that can be simultaneously reproduced in combination with the primary video stream can be defined in, for example, a seventh STN_table as shown in FIGS. That is, FIG. 44 and FIG. 45 show the recording position (stream_entry), attribute (stream_attribute) of the secondary video stream, primary audio stream, secondary audio stream, and subtitle stream that can be simultaneously played in combination with the primary video stream, and FIG. 17 is a diagram illustrating a seventh example (seventh STN_table ()) of syntax of STN_table () when a combination is defined. In the figure, the same parts as those in FIG. 41 described above are repeated, and the description thereof is omitted.

  In the example of FIGS. 44 and 45, a combination of a secondary video stream, a primary audio stream, a secondary audio stream, and a subtitle stream that can be simultaneously reproduced in combination with the primary video stream is defined as follows. That is, first, one or more secondary video streams that can be simultaneously reproduced in combination with the primary video stream are defined. Then, for each of the one or more secondary video streams, an audio stream (primary audio stream and secondary audio stream) and a subtitle stream that can be reproduced simultaneously are defined.

  Specifically, in FIG. 44, number_of_video_stream2_entries indicates the number of streams to which video_stream_id2 entered (registered) in the seventh STN_table () is given. video_stream_id2 is information for identifying a secondary video stream, and video_stream_number2 is a secondary video stream number that is used for video switching and is visible to the user.

  In FIG. 45, in a for loop of video stream ID2 (video_stream_id2), one video elementary stream (video elementary stream to be a secondary video stream) specified for each stream_entry () in order is assigned video_stream_id2 from 0 to 0. Given.

  At this time, number_of_Audio_combinations_for_video2 is given to video_stream_id2, and in the subsequent for statement, audio_stream_id and audio_stream_id2 are given by the number of number_of_Audio_combinations_for_video2. number_of_Audio_combinations_for_video 2 and the subsequent for statement are information defining a combination of audio streams that are simultaneously played with the secondary video stream, that is, a combination of a primary audio stream specified by audio_stream_id and a secondary audio stream specified by audio_stream_id2 It is. The number of sets of audio streams (a set of primary audio streams and secondary audio streams) that can be combined with the secondary video stream specified by video_stream_id2 is number_of_Audio_combinations_for_video2. Then, as a set of audio streams that can be combined with the secondary video stream identified by video_stream_id2, audio_stream_id that identifies the primary audio stream and audio_stream_id2 that identifies the secondary audio stream are defined in a for statement after number_of_Audio_combinations_for_video2, respectively. .

  Also, number_of_Subtitle_combinations_for_video2 is given to the same video_stream_id2, and PG_textST_stream_id is given to the same video_stream_id2 by the number of number_of_Subtitle_combinations_for_video2. The number_of_Subtitle_combinations_for_video2 and the subsequent for sentence are information defining a combination of subtitle streams that are reproduced simultaneously with the secondary video stream, that is, a combination of subtitle streams specified by PG_textST_stream_id. The number of subtitle streams that can be combined with the secondary video stream specified by video_stream_id2 is number_of_Subtitle_combinations_for_video2. Then, PG_textST_stream_id for specifying a subtitle stream that can be combined with the secondary video stream specified by video_stream_id2 is defined in a for sentence after number_of_Subtitle_combinations_for_video2.

  As described with reference to FIGS. 35 and 41, each number is used instead of each ID, for example, an audio stream number (audio_stream_number) is used instead of audio_stream_id, or an audio stream number 2 is used instead of audio_stream_id2. (Audio_stream_number2) may be used. The same applies to the video stream and the subtitle stream.

  According to the seventh STN_tabke shown in FIG. 44 and FIG. 45, the video_stream_id2 can be used to define a secondary video stream that is played back simultaneously with the primary video stream, and the video_stream_id2, audio_stream_id, audio_stream_id2, In addition, a combination of a primary audio stream, a secondary audio stream, and a subtitle stream that can be played back simultaneously with the secondary video stream can be defined using PG_textST_stream_id. That is, a combination of a secondary video stream, a primary audio stream, a secondary audio stream, and a subtitle stream that can be reproduced simultaneously with the primary video stream can be defined.

  Therefore, since it is not necessary to define a conflicting combination as a combination of the secondary video stream, the primary audio stream, the secondary audio stream, and the subtitle stream that can be played back simultaneously with the primary video stream, the user can It is possible to easily select a combination of streams that can be reproduced simultaneously.

  A specific example will be described with reference to FIGS. 46 and 47. FIG. FIG. 46 shows an audio signal corresponding to the primary audio stream or secondary audio stream provided to the user, a video signal corresponding to the secondary video stream, and a caption signal corresponding to the subtitle stream when the seventh STN_table is used. It is a figure which shows the example of the stream number table showing the relationship of these combinations. In the figure, the description of the audio signal (audio) and the subtitle signal (sub-picture) is the same as in FIG. 26 or FIG.

  In FIG. 46, V2_SN (Video2 Stream Number: secondary video stream number) is assigned to each secondary video stream (secondary video stream entered with video_stream_id2) entered in the seventh STN_table () of the PlayItem constituting the Main Path of the PlayList. Also referred to as).

  Specifically, video 2 is given to V2_SN = 1, and video 1 is given to V2_SN = 2.

  At this time, for video 2 with V2_SN = 1, audio 2 (primary audio stream entered with audio_stream_id in the for statement when video_stream_id2 in FIG. 45 is an ID for identifying video 2) is the primary audio stream. As a secondary audio stream, audio 4 (secondary audio stream entered with audio_stream_id2 in the for statement when video_stream_id2 in FIG. 45 is an ID for specifying video 2) is sub-picture 3 (video_stream_id2 in FIG. 45 is The subtitle streams entered by PG_textST_stream_id in the for sentence in the case of the ID for specifying the video 2 can be combined.

  For video 1 with V2_SN = 2, audio 1 (primary audio stream entered with audio_stream_id in the for statement when video_stream_id2 in FIG. 45 is an ID for identifying video 1) is used as a primary audio stream. As a secondary audio stream, audio 4 or audio 5 (each of the two secondary audio streams entered in each of the two audio_stream_id2 in the for statement when video_stream_id2 in FIG. 45 is an ID for identifying video 1) is a subtitle stream. Sub-picture 1 (sub-title streams entered with PG_textST_stream_id in the for sentence when video_stream_id 2 in FIG. 45 is an ID for identifying video 1) can be combined.

  That is, in the for sentence in the case where video_stream_id2 in FIG. 45 is an ID for identifying video 1, the following first set and second set are set as audio stream sets that can be played back simultaneously with video 1 with V2_SN = 2. Are defined respectively. The first set refers to a set in which the primary audio stream is audio 1 and the secondary audio stream is audio 4. The second set refers to a second set in which the primary audio stream is audio 1 and the secondary audio stream is audio 5.

  When the stream number table of FIG. 46 is presented to the user (in this case, the presentation form is not particularly limited), the user first starts a secondary video stream corresponding to a video such as a director's commentary to be reproduced simultaneously with the main movie or the like. That is, one of video 2 (VS_SN = 1) and video 1 (VS_SN2) is selected as the secondary video stream corresponding to the video to be displayed on the sub-screen (sub-screen 302 in the example of FIG. 43). (Such selection operation is possible).

  For example, when video 2 (VS_SN = 1) is selected as the secondary video stream, as shown in FIG. 46, the combinations that can be reproduced simultaneously with video 2 (VS_SN = 1) are audio 2 and audio 4 , And a combination of sub-pictures 3 only. Therefore, when the video 2 (VS_SN = 1) is selected as the secondary video stream, the user's selection operation ends.

  On the other hand, as shown in FIG. 47, when video 1 (VS_SN = 2) is selected as the secondary video stream, the combinations that can be reproduced simultaneously with video 1 (VS_SN = 2) include audio 1 and audio 4 and a first combination of sub-picture 1 and two combinations of audio 1, audio 5 and a second combination of sub-picture 1 exist. Therefore, when video 1 (VS_SN = 2) is selected as the secondary video stream, the user can further select one of the first combination and the second combination (such as such Select operation).

  Specifically, for example, here, as in the selection operation described with reference to FIG. 39, the user selects the right of the corresponding display on the stream number table with respect to the stream to be simultaneously played with video 1 (VS_SN = 2). On the other hand, when selecting playback, it is possible to execute a selection operation such as switching from “×” to “◯”, while holding “X” when not selecting playback. As in the case described with reference to FIG. 39, “◯” is a symbol indicating a state in which reproduction is selected, while “x” is a symbol indicating a state in which reproduction is not selected. .

  In the example of FIG. 47, the result of selecting the first combination of audio 1, audio 4, and sub-picture 1 by such a selection operation is shown. That is, as shown in FIG. 47, since audio 1 is selected as the primary audio stream that is played back simultaneously with video 1 with V2_SN = 2, the selection state is switched from “×” to “◯”. Also, since audio 4 is selected as the secondary audio stream that is played back simultaneously with video 1 with V2_SN = 2 (because audio 5 is not selected), the selection state of audio 4 is switched from “×” to “◯”. On the other hand, the selection state of the audio 5 is kept “x”. Further, since sub-picture 1 is selected as a subtitle stream that is played back simultaneously with video 1 with V2_SN = 2, the selection state is switched from “×” to “◯”.

  In other words, the video 1, audio 1, audio 4, and sub-picture 1 are selected by the above-described series of selection operations, and can be reproduced together with the main video (which is the primary video stream) (such reproduction is performed). State to be selected).

  In the above-described example, as an operation for selecting a stream to be reproduced simultaneously with the primary video stream, an operation with the secondary video stream as an axis, that is, after selecting the secondary video stream first, the primary audio stream, the secondary audio stream, and The operation for selecting a subtitle stream has been described.

  However, as such a selection operation, not only the above-described example but also, for example, in the same manner as the operation described above with reference to FIG. 39 or the like, after the primary audio stream is first selected, the secondary video stream, the secondary audio stream, and An operation for selecting a subtitle stream may be employed. That is, any one of the audio 2 with A_SN = 1, the audio 1 with A_SN = 2, and the audio 3 with A_SN = 3 in the stream number table of FIG. 47 is set as the primary audio stream. A selection operation of selecting a secondary video stream, a secondary audio stream, and a subtitle stream by the same operation as that described above with reference to FIG.

  Further, for example, an operation of selecting a primary audio stream, a secondary video stream, and a secondary audio stream after selecting a subtitle stream first may be employed. That is, any one of sub-picture 3 with S_SN = 1, sub-picture 1 with S_SN = 2, and sub-picture 2 with S_SN = 3 in the stream number table of FIG. However, although not shown in FIG. 47, a selection operation such as selecting a secondary video stream, a primary audio stream, or a secondary audio stream may be employed.

  In the present embodiment, the user switching operation of the secondary audio stream is prohibited. For this reason, in the example of FIG. 47, A_SN for the secondary audio stream is not given. That is, in the example of FIG. 47, audios 1 to 3 are used as primary audio streams, and audios 4 and 5 are used as secondary audio streams.

  Incidentally, when the picture-in-picture method (see FIG. 43) is realized by the syntax using the seventh STN_table () shown in FIGS. 44 and 45 described above, the type of Subpath_type is the first in FIG. 11 described above. When the example type is used, for example, a PlayList shown in FIG. 48 is generated.

In the example of FIG. 48, Video (video stream) and Audio (audio) included in Clip AV stream file-0 (main Clip AV stream file) referred to by play item 1 (PlayItem1) constituting the main path (Main path). Each of the primary video stream and the primary audio stream.
That is, in STN_table, type = 1 is set in the stream_entry corresponding to video_stream_id and audio_stream_id. That is, in correspondence with the example of FIG. 43, each of Video and Audio included in Clip AV stream file-0 is included in Clip AV stream file-0 corresponding to each of primary video stream 311 and primary audio stream 312. Is reproduced on the full screen 301, and the audio included in the Clip AV stream file-0 is reproduced and output.

  On the other hand, the 2ndVideo (video stream) included in the Clip AV stream file-1 (sub Clip AV stream file) referred to as the sub play item (SubPlayItem: SPI) included in the sub path 1 (Sub path-1) is the secondary. A 2ndAudio (audio stream) included in the Clip AV stream file-1 referred to by the sub play item (SPI) included in the sub path 2 (Sub path-2) is a secondary audio stream. That is, this is a case where type = 2 is set in the stream_entry of video_stream_id2 and audio_stream_id2 in STN_table. That is, in the example of FIG. 43, 2ndVideo and 2ndAudio included in Clip AV stream file-1 correspond to secondary video stream 313 and secondary audio stream 314, respectively, and Clip AV stream file-1 The included 2nd Video is reproduced on the child screen 302, and the 2nd Audio included in the Clip AV stream file-1 is reproduced and output.

  However, Audio included in the Clip AV stream file-0 is reproduced as a primary audio stream in association with the entire screen 301, and 2ndAudio included in the Clip AV stream file-1 is associated with the child screen 302 as a secondary audio stream. As described above, “reproducing” indicates that the audio obtained as a result of performing the mixing process on the two audio streams is output.

  As described above, in the example of FIG. 48, the type of the first example of FIG. 11 is used as the type of SubPath_type, and as a result, the picture-in-picture technique is realized by the main path, the subpath 1, and the subpath 2. ing.

  However, in the example of FIG. 48, each of the secondary video stream and the secondary audio stream that are played back simultaneously with the Video of the Clip AV stream file-0 that is the primary video stream is included in the same Clip AV stream file-1. In spite of being 2ndVideo and 2ndAudio, they are defined in separate subpath 1 and subpath 2, respectively. This is because, in the example of FIG. 11, only the SubPath_type type that only one ES (elementary stream) is referenced is prepared. That is, in the example of FIG. 11, each type of ES such as Audio, IG, Text, and Video is included in each Clip AV stream file included in the same Clip. This is because only the SubPath_type type that SubPath_type is always assigned one by one is prepared.

  That is, regardless of the example of FIG. 48, when the type of SubPath_type in the example of FIG. 11 is used, when there are two or more ESs to be reproduced, apart from the ES referenced in the main path, The above ESs are divided into separate sub-paths, that is, defined as two different SPIs.

  In this case, if these two or more ESs are included in the Clip AV stream file included in the same Clip, the above-mentioned SubPlayItem_IN_time, SubPlayItem_OUT_time, etc. are the same even though they are the same. As a result, there is a need for multiple times), and the first problem is that the information of SubPlayItem_IN_time and SubPlayItem_OUT_time becomes redundant.

  In this case, since a subpath must be defined for each ES to be played back, a second problem that the structure of the PlayList becomes more complicated than necessary occurs.

  Furthermore, from the second problem, the following third problem also occurs.

  That is, for example, in this embodiment, the playback device (for example, the playback device 141 in FIG. 36 described above or the playback device 401 in FIG. 53 described later) includes two TSs (that is, each included in two Clips). There is a restriction (binding) that can only be read up to (TS). In a playback apparatus with such restrictions, when a PlayList having such a complicated structure is to be played, it is immediately determined which of the main path defined in the PlayList and which sub-path should be combined for playback. The third problem arises that it is not possible, that is, it is necessary to execute complicated processing to determine such a combination.

  Therefore, in order to solve the first to third problems, the present inventor adds, as the type of SubPath_type, a type that can handle a plurality of ESs, that is, a type that can simultaneously reference a plurality of ESs. The method was further invented. FIG. 49 shows a second example of the type of SubPath_type to which such a technique is applied.

  That is, FIG. 49 shows a second example of the type of SubPath_type (subpath type), which is different from that of FIG. 11, that is, for the types that also exist in the example of FIG. It is a figure explaining the example which added the kind which can solve 3 problems. That is, in order to solve the first to third problems, for example, the second SubPath_type shown in FIG. 49 may be adopted.

  In FIG. 49, SubPath_type = 0 to 4 are the same as the corresponding types in FIG. 11, so the description thereof is omitted here.

  However, in “Meaning” with SubPath_type = 0 to 4, the description “Out-of-mux” is described in the example of FIG. 49, but the description is omitted in the example of FIG. The description will be described. That is, the description of “Out-of-mux” is the type (type) of the subpath when the TS including the ES referred to in the subpath is different from the TS including the ES referred to in the main path. That is, the ES referred to in the sub path indicates that the sub path type is not multiplexed in the TS including the play item referred to in the main path. Hereinafter, this type is referred to as a main path TS non-multiplex type path.

  Further, when the ES referred to in the main path and the ES referred to in the sub path are asynchronous as in SubPath_type = 2, 3, the type (kind) of the sub path is hereinafter referred to as an asynchronous path. On the other hand, when the ES referenced by the main path and the ES referenced by the sub path are synchronized as in SubPath_type = 4, 5, the type (type) of the sub path is hereinafter referred to as a synchronous path.

  SubPath_type = 5 is Out-of-mux and AV Synchronized type of one or more elementary streams path (Primary audio / PG / IG / Secondary audio path) .Out of mux and AV synchronized type of Picture-in-Picture presentation path which Contains one or more elementary streams paths. In other words, SubPath_type = 5 is a main path TS non-multiplexed and synchronous path, and includes one or more ES (Primary audio / PG / IG / Secondary audio) paths, picture-in-picture presentation paths, and so on. Has been.

  That is, SubPath_type = 5 includes the types indicated by SubPath_type = 5 and 6 in the example described with reference to FIG. 11, and refers to the 2nd Audio Presentation path (second audio presentation path) (refers to the 2nd audio stream). For example) and a 2nd Video Presentation path (second video presentation path) (path for referring to the 2nd video stream).

  Here, the picture-in-picture presentation path refers to a primary audio stream stream and a secondary video stream for a predetermined primary video stream (a video stream referred to in the main path) in the above-described picture-in-picture method (see FIG. 43). , One or more paths (types of such sub paths) of the secondary audio stream and the subtitle stream.

  SubPath_type = 6 is Out-of-mux and AV non-Synchronized type of Picture-in-Picture presentation path which contains one or more elementary streams paths. That is, SubPath_type = 6 is a main path TS non-multiplexed and asynchronous path, and is a picture-in-picture presentation path (one or more ES paths).

  SubPath_type = 7 is In-mux type and AV Synchronized type of Picture-in-Picture presentation path which contains one or more elementary streams paths.

  Here, the description “In-mux” is a subpath type (kind) when a TS including an ES referenced by a subpath is the same as a TS including one or more ESs referenced by a main path. That is, it is indicated that the ES referred to in the subpath is the type (kind) of the subpath multiplexed in the TS including the ES referred to in the main path. In other words, the main TS included in the main clip includes the main Clip AV stream and the sub Clip AV stream. Hereinafter, this type is referred to as a main path TS multiplexing type path.

  That is, SubPath_type = 7 is a main path TS multiplexing type and synchronous type path, which is a picture-in-picture presentation path (one or more ES paths).

  SubPath_type = 8-255 is reserved.

  By using the type of Subpath_type in the example of FIG. 49, it is possible to use newly added SubPath_type = 5, 6, and 7. Here, in SubPath_type = 7, MainTS referred to by PlayItem, that is, the main Clip AV stream includes SubST referred to by SubPath. Therefore, instead of Stream_entry () described with reference to FIG. 17, type = 3 (type = 3 in FIG. 50 is different from type = 3 in the case described with reference to FIG. 15) is defined. The third Stream_entry () shown in FIG. 50 is used.

  That is, type = 3 defined in the third Stream_entry () is for identifying an elementary stream that is included in the MainClip and referenced by the SubPath when SubPath_type = 7 in the corresponding PlayItem. .

  50, the description of the same parts as those in FIG. 17 will be omitted as appropriate.

  That is, when type = 3, SubPath refers to the same Clip as MainPath, in other words, a plurality of elementary streams are multiplexed on MainClip, and both MainPath and SubPath are multiplexed on MainClip. When referring to any one of the streams, Ref_to_stream_PID_of_MainClip is used to specify one elementary stream referenced by SubPath from among a plurality of elementary streams of MainClip when SubPath_type = 7. MainClip packet ID (PID) is specified. Also, SubPath_id is indicated by ref_to_SubPath_id, the SubPath defined in PlayList () of FIG. 9 is specified by this SubPath_id, and the corresponding SubPlayItem is called from this Subpath (FIG. 10), so that the elementary stream playback time ( (In_time, Out_time) and the like can be grasped (FIG. 12).

  As described above, when a type (three types of type 1 to type 3) is used and a PlayItem and one or more SubPaths to be played back are prepared, a clip referred to by the PlayItem is further Even if it is referenced from SubPath, one elementary stream can be specified from the Clip.

  Further, by using the type of Subpath_type in the example of FIG. 49 as described above, that is, a plurality of (one or more) elements newly added together with SubPath_type = 1 to 4 that also existed in the example of FIG. As a PlayList for realizing the picture-in-picture method (see FIG. 43) by using SubPath_type = 5, 6 and 7 that can handle a mental stream, instead of a PlayList having a complicated structure as in the example of FIG. For example, a PlayList having a simple structure as shown in FIG. 51 can be created / used.

  That is, in the example of FIG. 51, each of the Video (video stream) and Audio (audio stream) included in the Clip AV stream file-0 referred to by the play item 1 (PlayItem 1) included in the main path (Main path) is obtained. , Primary video stream and primary audio stream respectively. That is, in the example of FIG. 43, each of Video and Audio included in clip AV stream file-0 corresponds to each of primary video stream 311 and primary audio stream 312, and clip AV stream file-0 The included video is reproduced on the full screen 301, and the audio included in the clip AV stream file-0 is reproduced and output.

  In STN_table, when stream_entry is defined for each of video_stream_id, audio_stream_id, video_stream_id2, and audio_stream_id2, and in stream_entry of FIG. 50, specified in MainPath and present in MainClip (type = 1), specified in SubPath, specified in SubClip If it exists, it is a primary audio stream (audio stream indicated by audio_stream_id) because each is defined (type = 2), but exists in MainClip but specified by SubPath (type = 3, FIG. 50). However, it may be specified by SubPath. Similarly, in this configuration, the primary video stream (video stream indicated by video_stream_id) can also be specified by SubPath. However, since the primary video stream is considered as the premise AV data, the primary video stream The stream is specified by MainPath. Note that when the primary video is designated by SubPath, only the definition and addition of SubPath_type need be changed.

  Also in the example of FIG. 51, each of the secondary video stream and the secondary audio stream that are played back simultaneously with the Video of the Clip AV stream file-0 that is the primary video stream is included in the same Clip AV stream file-1. 2ndVideo and 2ndAudio respectively.

  Therefore, in the example of FIG. 51, as the type of sub path A (Sub path-A), the above-mentioned SubPath_type = 5, that is, the main path TS non-multiplexed type and synchronous type path, which is a picture-in-picture presentation path. Is adopted. As a result, two ESs of 2ndVideo and 2ndAudio included in the Clip AV stream file-1 can be referenced simultaneously as a sub play item (SPI) referenced by the sub path A (Sub path-A). As a result, when corresponding to the example of FIG. 43, each of the 2ndVideo and 2ndAudio included in the Clip AV stream file-1 referred to by only one sub path A (Sub path-A) is the secondary video stream 313 and the secondary video stream 313. Corresponding to each of the audio streams 314, 2ndVideo included in the Clip AV stream file-1 is reproduced on the child screen 302, and 2ndAudio included in the Clip AV stream file-1 is reproduced and output.

  At this time, when the above-described SubPath_type = 5 is adopted as the type of sub path A (Sub path-A), the secondary video stream 313 and the secondary audio stream 314 are reproduced in synchronization with the primary video stream 311. It will be.

  In other words, when it is desired to reproduce the secondary video stream 313 and the secondary audio stream 314 asynchronously with the primary video stream 311 (or when it is necessary to reproduce asynchronously), the sub path A (Sub path-A) As the type, SubPath_type = 6 described above, that is, a main path TS non-multiplexed type and asynchronous type path, and a picture-in-picture presentation path (one or more ES paths) may be adopted.

  As described above, in the example of FIG. 51, as a result of adopting the type of Subpath_type of the example of FIG. 49, the picture-in-picture technique is realized only by one subpath other than the main path (for example, subpath A of FIG. ing.

  That is, when realizing a picture-in-picture method in which 2ndvideo and 2ndAudio included in the same Clip AV stream file-1 are used as the secondary video stream 313 and the secondary audio stream 314, respectively, the Subpath_type in the example of FIG. In the PlayList in the example of FIG. 48 generated using the type, each of 2ndvideo and 2ndAudio included in the same Clip AV stream file-1 is referred to by two separate subpaths. On the other hand, in the PlayList of the example of FIG. 51 generated using the Subpath_type of the example of FIG. 49, each of 2ndvideo and 2ndAudio included in the same Clip AV stream file-1 is referred to by only one subpath. It can be done.

  As a result, it is only necessary to have one SubPlayItem_IN_time, SubPlayItem_OUT_time, etc. for each of 2ndvideo and 2ndAudio, so that the first problem described above can be solved.

  Further, as apparent from comparing the PlayList in the example of FIG. 48 and the example of FIG. 51, the structure can be simplified, so that the second problem and the third problem can be solved.

  Further, SubPath_type = 7 of the types of Subpath_type in the example of FIG. 49, that is, a main path TS multiplexing type and synchronous type path, and a picture-in-picture presentation path (one or more ES paths) is used. Thus, for example, a PlayList as shown in FIG. 52 can be created. As a result, the picture-in-picture technique using the PlayList in the example of FIG. 52 can be easily realized.

  In the example of FIG. 52, the Main TS referred to by the play item 1 (PlayItem 1) included in the main path (Main Path) includes the Clip AV stream file-0 (however, the Clip AV stream file-0 of FIG. Different Clip AV stream files) are included, and Clip AV stream file-0 includes MainST referenced by PlayItem-1 and SubST referenced by SubPlayItem. Each of Video (video stream) and Audio (audio stream) included in MainST becomes a primary video stream and a primary audio stream, respectively. That is, in correspondence with the example of FIG. 43, the Video and Audio of MainST included in the Clip AV stream file-0 correspond to the primary video stream 311 and the primary audio stream 312, respectively. Video included in 0 is reproduced on the entire screen 301, and Audio included in Clip AV stream file-0 is reproduced and output.

  In the example of FIG. 52, each of the secondary video stream and the secondary audio stream that are played back simultaneously with the MainST Video that is the primary video stream is included in the Clip AV stream file-1 included in the same Main TS. And 2ndAudio.

  In this case, in order to create the PlayList in the example of FIG. 52, SubPath_type = 7 may be adopted as the type of sub path A (Sub path-A). As a result, the SubST 2ndVideo and 2ndAudio included in the Clip AV stream file-0 of the Main TS that is also referenced in the main path as the ES that is referenced by the sub play item (SPI) included in the sub path A (Sub path-A). The two ESs can be referred to. As a result, according to the example of FIG. 43, the 2ndVideo and 2ndAudio included in the Clip AV stream file-0 referred to by the sub path A (Sub path-A) are converted into the secondary video stream 313 and the secondary audio stream 314, respectively. 2ndVideo included in the Clip AV stream file-0 can be reproduced on the child screen 302, and 2nd Audio included in the Clip AV stream file-0 is reproduced and output. That is, in the case of FIG. 52, the main TS included in the main clip includes MainST that is the main stream and SubST that is the sub stream.

  In addition, as shown in FIG. 52, as a sub-path for PlayItem-1, in addition to sub-path A, a Clip included in a TS (Sub TS) different from the Main TS including the Clip AV stream file-1 is included. When the subpath B (Subpath-B) that refers to the AV stream file-1 is defined, as described above, the playback device according to the present embodiment can simultaneously read two TSs. In addition to the Video, Audio, 2nd Video, and 2nd Audio included in the main TS, the PG included in the sub TS can be read.

  The example of syntax necessary for applying the picture-in-picture method (see FIG. 43), that is, the seventh STN_table () in FIGS. 44 and 45 and the type of SubPath_type in the example of FIG. 49 have been described.

  In order to realize such a picture-in-picture method, it is necessary to apply a playback device having a function of simultaneously playing a primary video stream and a secondary video stream, for example, a playback device as shown in FIG.

  Next, different examples of the playback apparatus to which the present invention is applied will be described. The playback apparatus 401 is basically the playback apparatus 141 described with reference to FIG. 36 except that an AV decoder section 403 is provided instead of the AV decoder section 151 of the playback apparatus 141 described with reference to FIG. It has the same configuration as. FIG. 53 is a block diagram illustrating a configuration example of the AV decoder unit 403. The playback device 401 including the AV decoder unit 403 applies a picture-in-picture technique, and as described with reference to FIG. 47, the AV that is referred to by the main path on the entire screen 301 of one screen. Play back content such as the main movie corresponding to the stream, and play back other content such as a movie director's commentary corresponding to the AV stream referenced in the sub path on the sub-screen 302 arranged on the full screen 301. Is possible.

  53 is a PlayList having the above-described main path and sub path, and performs playback according to a PlayList (for example, the PlayList in the examples of FIGS. 51 and 52) for realizing the picture-in-picture method. Can do.

  In FIG. 53, portions (blocks) corresponding to those in FIG. 36 are denoted by corresponding reference numerals, and description thereof will be omitted as appropriate.

  The playback device 401 in the example of FIG. 53 is provided with a storage drive 22, a switch 32, and a controller 21 that have the same functions and configurations as the playback device 141 in the example of FIG. However, the function of the controller 21 is added. Details of the added function will be described later. Further, the playback device 401 in the example of FIG. 53 is provided with an AV decoder unit 403 having a slightly different function and configuration from the AV decoder unit 151 of the playback device 141 in the example of FIG.

  That is, the AV decoder unit 403 in the example of FIG. 53 is provided with a buffer 51 through a mixing processing unit 101 having substantially the same functions and configuration as the AV decoder unit 151 in the example of FIG. The AV decoder unit 151 in this example is provided with PID filters 411 to preload buffers 414 that are not provided.

  Note that “almost” is described as “having substantially the same function and configuration as the AV decoder unit 151 in the example of FIG. 36” because the name of the video decoder 72 and the video plane generating unit 92 is the 1st video decoder. This is because there are some differences such as a change to 72 or the 1st video plane generation unit 92, or an increase in the output of the PID filter 56 by one.

  In the following, portions of the AV decoder unit 403 in the example of FIG. 53 that are different from the AV decoder unit 151 in the example of FIG. 36 will be described.

  The AV decoder unit 403 needs to decode the primary video stream and the secondary video stream in order to realize the picture-in-picture method. Therefore, each of the video decoder 72 and the video plane generation unit 92 provided in the AV decoder unit 151 in the example of FIG. 36 is the first video for the primary video stream in the AV decoder unit 403 in the example of FIG. Each of the decoder 72 and the 1st video plane generating unit 92 is provided. Further, the AV decoder unit 403 in the example of FIG. 53 includes a 2nd video decoder 412 and a 2nd video plane for the secondary video stream having the same functions and configurations as the 1st video decoder 72 and the 1st video plane generation unit 92, respectively. Each of the generators 413 is provided.

  The PID filter 411 receives a main ClipAV Stream video stream or a sub Clip AV Stream video stream via the PID filter 161 or the PID filter 56. Therefore, the PID filter 411 distributes the input video stream to the subsequent 1st video decoder 72 or 2nd video decoder 412 according to the PID (packet ID) and outputs the result. That is, when a primary video stream is input, the PID filter 411 supplies the primary video stream to the 1st video decoder 72. In contrast, when a secondary video stream is input, the PID filter 411 supplies the secondary video stream to the 2nd video decoder 412.

  In the present embodiment, only the main ClipAV Stream video stream is used as the primary video stream, and the sub Clip AV Stream video stream is not used. However, the sub-Clip AV Stream video stream supplied from the PID filter 56 can also be supplied to the 1st video decoder 72 in consideration that the sub-Clip AV Stream video stream may be used as the primary video stream in the future. The PID filter 411 is configured. That is, only the video stream included in the sub clip specified by Subpath is played, the main clip is not used for playing the video stream, and the audio stream is played using the sub clip or the main clip. Therefore, it is possible to employ a configuration in which the main clip is not used (MainPath is not used) and only the SubClip is used (only SubPath is used) for video stream and audio stream playback.

  The preload buffer 414 temporarily stores the interactive graphics stream supplied from the interactive graphics decoder 74. That is, as described above, in the present embodiment, the playback apparatus 401 has a restriction that only two TSs can be read at a time. Therefore, the playback device 401 obtains the interactive graphics stream in advance (by preloading), so that when the video stream or the audio stream is played back, the preloaded interactive graphics stream is simultaneously received. It has the function of being able to reproduce. In order to realize such a function, that is, to store the preloaded interactive graphics stream until the timing when the video stream or the audio stream is reproduced, the preloading buffer 414 includes an AV decoding unit in the example of FIG. 403 is provided.

  53 is an example of processing by the picture-in-picture technique executed by the playback apparatus 401 in the example of FIG. 53, and is a series necessary for switching the secondary video stream to be played back during playback of the primary video stream or the like. An example of this process (hereinafter referred to as secondary video switching process) will be described with reference to the flowcharts of FIGS.

  In step S101 in FIG. 54, the controller 21 displays an order list of secondary video stream numbers (which may be IDs) and a list of primary audio streams, secondary audio streams, and subtitle streams combined with the secondary video streams. 44 and the seventh STN_table () described with reference to FIG.

  Here, after the processing of step S101, based on the acquired list, for example, the above-described stream number table as shown in FIG. 46 is generated, and the stream number table itself or a predetermined GUI (Graphical GUI) generated based thereon (User Interface) image is generated by the controller 21, and the display on the display device is controlled and presented to the user. As a result, the process proceeds to step S102.

  In step S102, the controller 21 receives a switching instruction for the secondary video stream from the user.

  That is, the content of the user switching operation described with reference to FIGS. 46 and 47 is received by the controller 21 as a switching command for the secondary video stream from the user in step S102.

  In step S103, the controller 21 acquires the number of the next combination of the secondary video stream, the primary audio stream, the secondary audio stream, and the subtitle that are being reproduced.

  Specifically, for example, when the stream number table in the example of FIG. 46 is presented to the user, the following first combination to the following are combined as combinations of the secondary video stream, the primary audio stream, the secondary audio stream, and the subtitle. There will be 3 combinations. That is, the first combination is a combination of video 2, audio 2, audio 4, and sub-picture 3. The second combination is a combination of video 1, audio 1, audio 4, and sub-picture 1. The third combination is a combination of video 1, audio 1, audio 5, and sub-picture 1. Thus, for example, here, it is assumed that 1, 2-1, and 2-2 are given as the numbers of the combinations of the first combination, the second combination, and the third combination, respectively. For example, here, it is assumed that the selection operation result of the example of FIG. 47, that is, the second combination selection command is received by the controller 21 in the process of step S102. In this case, in the process of step S103, 2-1 is acquired as the next combination number.

  In step S104, the controller 21 corresponds to the acquired number (the number acquired in the process of the immediately preceding step S103 in the process of the first step S104) based on the contents described in stream_attribute () (FIG. 18). It is determined whether or not it has a function of reproducing a plurality of streams.

  Specifically, for example, focusing on the playback function of the audio stream, when the controller 21 accepts the selection operation result of the example of FIG. 47, that is, the second combination selection command, the audio 1 and the audio 4 The determination as to whether or not each has a reproduction function is executed as the process of step S104.

  In addition, it is also possible to determine whether or not there is a playback function for other streams such as a video stream as the processing in step S104.

  If it is determined in step S104 that the function of reproducing a plurality of streams corresponding to the acquired number is not provided, the controller 21 acquires the number of the next combination after the current combination number in step S105. That is, when the audio stream or the like specified by the current combination number is not reproduced, the combination number is skipped (not subject to reproduction), and the next combination number is acquired. Thereafter, the process returns to step S104, and the subsequent processes are repeated. That is, the loop processing of steps S104 and S105 is repeated until a combination number specifying an audio stream or the like having a function to be played back by the playback device 401 itself is acquired.

  In step S104, the acquired number (in the case of the first process, the number acquired in the process of the immediately preceding step S103 as described above, and in the process of the second and subsequent processes, it is acquired in the process of the immediately preceding step S105. If it is determined that it has a function of playing back a plurality of streams corresponding to the number (No.), the process proceeds to step S106 in FIG.

  In step S106, the controller 21 checks the Clip including each of the secondary video stream, the primary audio stream, the secondary audio stream, and the subtitle stream corresponding to the acquired number based on the type of Stream_entry (). That is, it is confirmed whether the secondary video stream, primary audio stream, secondary audio stream, and subtitle stream corresponding to the acquired number are in the main clip or the sub clip.

  In step S107, the controller 21 specifies each desired secondary video stream, primary audio stream, secondary audio stream, and subtitle stream. That is, in the process of step S107, each desired stream corresponding to the acquired number is specified from the main clip or the sub clip.

  In step S108, the controller 21 specifies the storage drive 22 from the clip (main clip or sub clip) in which each desired secondary video stream, primary audio stream, secondary audio stream, and subtitle stream are multiplexed. Each stream is instructed to be read, or each identified stream is read from the Clip recorded in the local storage 24 and supplied to the AV decoder unit 403. When the target Clip is recorded on the recording medium 11, the storage drive 22 reads the specified stream from the target Clip recorded on the recording medium 11 based on this instruction. Specifically, the controller 21 instructs the storage drive 22 to read each stream identified in the process of step S107 from the Clip recorded on the recording medium 11.

  Then, the controller 21 supplies each read stream to the AV decoder unit 403 and instructs the AV decoder unit 403 to reproduce the secondary video stream, the primary audio stream, the secondary audio stream, and the subtitle stream. Specifically, the controller 21 reads so as to cause the 2nd video decoder 412 to decode the read desired secondary video stream, and to cause the 1st audio decoder 75-1 to decode the read desired primary audio stream. Instruct the presentation graphics decoder 73 to decode the read desired subtitle stream so that the desired secondary audio stream is decoded by the second audio decoder 75-2.

  In step S109, the AV decoder unit 403 decodes and outputs the secondary video stream, primary audio stream, secondary audio stream, and subtitle stream, and the process ends.

  At this time, as described above, for the primary audio stream and the secondary audio stream, an audio signal obtained as a result of mixing the primary audio stream and the secondary audio stream is output.

  As a result of such secondary video switching processing, the picture-in-picture method described with reference to FIG. 43 is realized. That is, in the example of FIG. 43, while the video corresponding to the primary video stream 311 is being displayed (reproduced) on the full screen 311, the video corresponding to the secondary video stream 313 output in the process of step S109 is a child. The primary audio that is displayed (reproduced) on the screen 302 and is displayed (reproduced) at a predetermined position on the entire screen 311 and is displayed (reproduced) corresponding to the subtitle stream 315 output in the process of step S109. Audio obtained as a result of mixing the stream 311 and the secondary audio stream 402 is output (reproduced) from a speaker not shown in FIG.

  As described above, by adopting the seventh STN_table in the examples of FIGS. 44 and 45, it is possible to realize the picture-in-picture technique.

  When realizing such a picture-in-picture method, it is also possible to use a playlist (for example, see FIG. 48) generated using the type of SubPath_type in the example of FIG. 11, but as described above, the same The secondary video stream, secondary audio stream, etc. included in the Clip AV Stream are divided into separate sub-paths, that is, defined as two different SPIs. As a result, the various problems described above, for example, the problem that the structure of the PlayList becomes more complicated than necessary occurs.

  Therefore, when solving these various problems, two or more ESs (secondary video stream, secondary audio stream, etc.) in the same Clip AV Stream can be referred to by one sub-path, that is, the same Clip AV Stream. The type of SubPath_type that can define two or more ESs as one SPI, that is, the type of SubPath_type described with reference to FIG. 49, for example, may be employed.

  However, from the viewpoint of the creator of the PlayList, it can be said that the PlayList having a simple structure can be created by adopting the type of SubPath_type in the example of FIG. Although the structure has been simplified, the following difficult work is required to create a PlayList with such a simple structure. In other words, the creator of the PlayList decides for himself what kinds of streams can be combined and what kinds of streams cannot be combined in order to realize the picture-in-picture method. In other words, it is necessary to create a PlayList after determining by itself what kind of sub-paths to include. Therefore, it is not difficult to imagine that the creator of PlayList will come up with a desire to easily create PlayList.

  Therefore, in order to respond to such a request in advance, the present inventor limits the possible creation (subpaths that can be included in the PlayList) according to the type of PlayList using the SubPath_type described with reference to FIG. Was invented. Hereinafter, this method is referred to as a sub-path restriction method.

  By applying the sub-path restriction method, the sub-paths that can be included in one PlayList are limited. Therefore, it is easy for the creator to create sub-paths after determining what sub-paths are included in the PlayList. As a result, it is possible to produce an effect that the creation of the PlayList itself becomes easy.

  Hereinafter, the sub path limiting method will be described in detail.

  However, in the sub-path restriction method, the restriction contents vary depending on the number of TSs that can be read by the playback apparatus at one time. In the present embodiment, as described above, the number of TSs that can be read by the playback apparatus at a time is two. Thus, hereinafter, a sub-path limiting method in the case where the playback apparatus can read two TSs at a time will be described.

  In the present embodiment, the PlayList is roughly divided into two types, namely, a Browsable Slideshow (asynchronous type) and a Movie Type / Time-based Slideshow (synchronous type). The distinction as to whether the type of PlayList is asynchronous or synchronous can be made according to application_type (application type) included in clip_Information_file of the Clip referenced by the PlayList.

  application_type is described in ClipInfo () in the Clip Information file for a certain Clip. However, here, for simplicity of explanation, description of the syntax of ClipInfo () is omitted, and only the types of application_type that can be described in Clipinfo () will be described below with reference to FIG.

  That is, FIG. 56 is a diagram for explaining an example of the type of application_type.

  As shown in FIG. 56, application_type = 0 is reserved. application_type = 1 is Main TS for movie application. Here, “Main TS” refers to a Transport stream as a PlayItem referred to by the main path in the Playlist, that is, Main TS. application_type = 2 is a Main TS for Time-based Slideshow, that is, a Main TS for an image slide show. application_type = 3 is a TS for Browsable Slideshow, that is, a Main TS for a video slide show.

  application_type = 4 is a TS of a Browsable Slideshow for SubPath. This is given to Clip Info () for a Clip AV stream that holds BGM (sound) data that is reproduced asynchronously with application_type = 3, for example. application_type = 5 is a TS for SubPath interactive graphics. application_type = 6 is a TS for a SubPath text subtitle (text subtitle data). application_type = 7 is a TS for SubPath including one or more ESs (elementary streams). application_type = 8 to 255 is reserved.

  The Clip AV Stream File represented by application_type = 1 to application_type = 6 is acquired from the recording medium 11 via the storage drive 22 or acquired from the local storage 24. On the other hand, the Clip AV Stream File represented by application_type = 7 is acquired from the local storage 24 and is not acquired from the recording medium 11 via the storage drive 22.

  Therefore, in the present embodiment, when the application_type described in ClipInfo () for the Main TS referenced by the PlayList is application_type = 3, the PlayList is classified as a Browsable Slideshow (asynchronous type), and application_type = 1or2 In this case, it is assumed that the PlayList is classified as Movie Type / Time-based Slideshow (synchronous type).

  In this case, for the type of PlayList (asynchronous type or synchronous type), the SubPath restriction method imposes a restriction that only the combinations shown in FIG. 57 are allowed for the combinations of SubPaths that the PlayList can have. Become.

  In FIG. 57, the item “number of SubPaths” describes the number of types of SubPaths indicated by the item “SubPath_type” on the left side that can be possessed by the Main TS indicated by the PlayList.

  In addition, each description value (SubPath_type type) of the item “SubPath_type” in FIG. 57 corresponds to the type of the example of FIG. 49 instead of the type of the example of FIG. 11 described above.

  As shown in FIG. 57, when the type of PlayList is asynchronous, that is, in the case of a Browsable slideshow with application_type (of Main TS) = 3, (1) zero or more SubPath_type = 2 SubPaths, (2) Only zero or more SubPath_type = 3 SubPaths are allowed. In other words, combinations of SubPaths other than (1) and (2) are prohibited.

  Even in the combination of (2), when IG is included in the Main TS of application_type = 3, SubPath of SubPath_type = 3 is prohibited (see * 2 in FIG. 57). This is due to the following restrictions.

  That is, from the viewpoint of IG, PlayList including IG is roughly divided into the following first type and second type. The first type is a type in which an IG is multiplexed on a referenced Main TS, and Audio / Video and IG of the Main TS are reproduced in synchronization. The second type includes a SubPath of SubPath_type = 3 and IG is preloaded (this IG is used for a pop-up menu). Note that, as described above, the playback device 401 in FIG. 53 is provided with the preload buffer 414 in order to enable playback of the second type PlayList.

  In addition, since the playback device of the present embodiment has only one IG decoder, specifically, for example, the playback device 401 of FIG. 53 has only one interactive graphics decoder 74. Is imposed. That is, in the first type PlayList described above, there is a restriction that another IG cannot be preloaded while the IG decoder is decoding the IG of the Main TS. On the other hand, in the second PlayList described above, there is a limitation that when the first IG preloaded in the IG decoder is being input, the second IG from another Main TS cannot be input to the IG decoder. Imposed.

  From the above, the PlayList including STN_table () that refers to the IG multiplexed in the Main TS is subject to the restriction that it cannot have a SubPath of SubPath_type = 3. Because of this restriction, when IG is included in MainTS with application_type = 3, SubPath with SubPath_type = 3 is prohibited (see * 2 in FIG. 57).

  On the other hand, when the type of PlayList is synchronous, that is, in the case of Movie Type / Time-based Slideshow where application_type (of Main TS) = 1 or 2, (3) zero or more SubPath_type = 3 SubPaths, (4 ) 0 or more SubPath_type = 4 SubPaths, (5) 0 or more SubPath_type = 5 SubPaths, (6) 0 or more SubPath_type = 6 SubPaths, (7) 0 or 1 SubPath_type = 7 Only SubPath is allowed. In other words, the combination of SubPaths with SubPath_type = 2 is prohibited.

  Even in the combination of (3), when IG is included in MainTS of application_type = 3, or when IG is included in TS that is referenced by SubPath of SubPath_type = 5, SubPath of SubPath_type = 3 is Prohibited (refer to * 1 in FIG. 57). This is also due to the constraints described above.

  Furthermore, in this embodiment, as described above, the number of Clips that can be referred to by PlayList at a time, that is, the number of TSs that can be read by the playback device at a time is at most two. Therefore, it is guaranteed that ESs referenced by the same PlayList are included in at most two Clips, that is, ESs that are played back simultaneously are referenced simultaneously from at most two Clip AV stream files. In order to do so, a restriction that the following condition is satisfied is imposed by the subpath restriction method.

  That is, combining PG_textST_stream other than primary_video_stream, primary_audio_stream, Text subtitle stream, IG_stream other than IG for Pop-up menu, secondary_video_stream, and secondary_audio_stream referenced by STN_table () imposed by the subpath restriction method The time conditions are as follows.

  ES identified by a certain primary_video_stream_number, ES identified by a certain primary_audio_stream_number, ES identified by a certain PG_textST_stream_number, ES identified by a certain IG_stream_number value, ES identified by a certain secondary_video_stream_number In the ES specified by a certain value of secondary_audio_stream_number, combinations that can be reproduced simultaneously need to be stored in Clip AV stream files that are simultaneously referenced by at most two PlayLists.

  Note that the ES referenced by STN_table () may be stored in three or more Clip AV stream files, but ESs that can be played simultaneously are stored in at most two Clip AV stream files. It is made to be done.

  Further, the restriction that the combination of the number of PlayItems (MainPath) and SubPaths in the PlayList needs to be any one of the following (a) to (c) as shown in FIG. Is also imposed by the subpath restriction technique. That is, the combination (a) refers to a combination of only one PlayItem (the combination in the top row in FIG. 58). The combination (b) refers to a combination of one PlayItem and one SubPath (combination of the center row in FIG. 58). The combination of (c) is a combination of one PlayItem, one SubPath, and one SubPath_type = 7, that is, one PlayItem and two SubPaths (provided that one of them is SubPath_type = 7) (The combination of the bottom row in FIG. 58).

  As a result of the various restrictions described above, a technique of restricting SubPath_type that can be created for a predetermined type of PlayList is an example of a subpath restriction technique.

  In other words, the sub-path restriction method is the type of PlayList (in the above example, the type distinguished by the application_type of the Main TS), and the number of TSs that can be read at once by the playback device (in the above example, two). And there is a condition (restriction) such as whether pre-loaded IG is included in the ES referenced by the sub-path (SubPath) of the PlayList (in the above example, whether it is a SubPath such as SubPath_type = 3) Then, it can be said that the SubPath included in the PlayList is determined so as to satisfy these conditions, and in turn, a PlayList including the SubPath determined in this way is created.

  Hereinafter, a specific example of the PlayList created by the subpath restriction method will be described with reference to FIGS. 59 to 62.

  FIG. 59 shows an example of a PlayList for a Browsable Slideshow (asynchronous type), in which there is no SubPath with SubPath_type = 3, that is, a PlayList with only Sub_Path-1 to Sub_Path-3 with SubPath_type = 2. Is shown. The reason why SubPath of SubPath_type = 3 does not exist is that IG is included in Clip AV stream file-0 (ie, Main TS) referenced by PlayItem-1 / PlayItem-2 of PlayList. Since this example represents a Browsable Slideshow in which a plurality of still images are continuously displayed by a user operation, the “video” stream included in the Clip AV stream file-0 includes still image data. keeping.

  The contents of the restrictions (conditions) used to create the PlayList in the example of FIG. 59 are as follows. That is, the restriction “(1) Zero or more SubPath_type = 2 SubPaths, (2) Only zero or more SubPath_type = 3 SubPaths” described above with reference to FIG. 57, and “(1) The restriction that “SubPath combinations other than (2) are prohibited” and “SubPath of SubPath_type = 3 are prohibited if IG is included in MainTS of application_type = 3 even in the combination of (2). The PlayList in the example of FIG. 59 is created using the limitation “* 2 in FIG. 57”.

  In contrast to the example of FIG. 59, FIG. 60 shows a PlayList of a Browsable Slideshow (asynchronous type), where a SubPath of SubPath_type = 3 exists, that is, Sub_Path−1, 2 of SubPath_type = 2. In addition, an example of a PlayList in the case where Sub_Path-3 of SubPath_type = 3 exists is shown. Note that the reason for the presence of SubPath of SubPath_type = 3 is that IG is not included in the Clip AV stream file-0 of Main-ST referenced by PlayItem-1 / PlayItem-2 of PlayList, that is, SubPath_type = 3 This is because the IG of the Clip AV stream file-3 different from the Clip AV stream file-0 is referred to by the SubPath.

  The contents of the restrictions (conditions) used to create the PlayList in the example of FIG. 60 are the same as the above-described restrictions of the example of FIG.

  In other words, the restriction indicated by * 2 in FIG. 57, that is, “Subpath of SubPath_type = 3 is prohibited if IG is included in MainTS of application_type = 3 even in the combination of (2)”. An example of the PlayList when it is necessary to impose such a restriction is the example of FIG. 59, and an example of the PlayList when there is no need to impose such a restriction is the example of FIG.

  As described above, FIG. 59 and FIG. 60 illustrate an example of a PlayList of Browsable Slideshow (asynchronous type). On the other hand, FIGS. 61 and 62 show an example of PlayList of Movie Type / Time-based Slideshow (synchronous type).

  That is, FIG. 61 shows a PlayList when the secondary video stream (2ndVideo) and the secondary audio stream (2ndAudio) are not multiplexed in the Main TS (ie, Clip AV stream file-0) in the synchronous PlayList. An example of a PlayList when Sub_Path-1, 2 of SubPath_type = 6 exists is shown.

  The contents of the restrictions (conditions) used to create the PlayList in the example of FIG. 61 are as follows. That is, the PlayList in the example of FIG. 61 is created by using the restriction “(6) Allow 0 or more SubPath_type = 6 SubPaths” described above with reference to FIG.

  In contrast to the example of FIG. 61, FIG. 62 shows that the MainTS, that is, the Clip AV stream file-0 referenced by the synchronous PlayList includes MainST and SubST, and includes the secondary video stream (2ndVideo) and secondary audio. This is a PlayList when the stream (2ndAudio) is multiplexed as SubST in the Clip AV stream file-0 that constitutes the Main TS. Therefore, Sub_Path-1 of SubPath_type = 7 exists, and SubPath_type = 5 An example of the PlayList when Sub_Path-2,3 exists is shown.

  The contents of the restrictions (conditions) used to create the PlayList in the example of FIG. 62 are as follows. That is, as described above with reference to FIG. 57, the restriction “(5) Allow 0 or more SubPath_type = 5 SubPaths” and “(7) Allow 0 SubPath and SubPath_type = 7 SubPaths”. The PlayList in the example of FIG. 62 is created using the restrictions.

  Also, the PlayList created by the sub-path restriction method as described above can be reproduced by, for example, the reproduction device 401 in FIG.

  For example, in the case of an asynchronous PlayList, that is, in the case of a PlayList of Browsable slideshow with application_type = 3, the playback device 401 can play back the PlayList as follows. That is, when the PlayList refers to one or more SubPaths of SubPath_type = 3 (Interactive graphics presentation menu), the playback device 401 plays back a Clip AV stream for SubPath_type = 3, that is, an interactive graphics stream, as a PlayList. Preload before and store in the preload buffer 414. Further, when the PlayList refers to one or more SubPath_type = 2 SubPaths, the playback apparatus 401 can refer to SubPaths only one by one (two TSs can be read at one time, of which Since one is a MainTS referred to by MainPath), reproduction is performed with reference to a predetermined one of one or more SubPaths with SubPath_type = 2.

  The interactive graphics presentation menu includes a “Pop-up menu” that can be displayed or canceled based on the ON / OFF operation input by the user, and an “Always-on” that is always displayed. The detailed display example of PlayList playback processing with application_type = 3 will be described later with reference to FIGS. 63 and 64.

  Further, for example, in the case of a synchronous PlayList, that is, in the case of a PlayList including a clip of Movie Type / Time-based Slideshow where application_type = 1or2, the playback device 401 can play back the PlayList as follows. . That is, when the PlayList refers to one or more SubPath_type = 3 or SubPath_type = 4 SubPaths, the playback apparatus 401 uses a Clip AV stream for SubPath_type = 3 or SubPath_type = 4, that is, an interactive graphics stream or text. The subtitle stream is preloaded before the PlayList is played back and stored in the preload buffer 414 or the buffer 54. In addition, when the PlayList refers to one or more SubPaths of SubPath_type = 5or6, the Player can refer to the SubPath only one by one (two TSs can be read at a time, Since one of them is a MainTS referred to by MainPath), reproduction is performed by referring to a predetermined one of one or more SubPaths of SubPath_type = 5 or 6. However, if one or more SubPaths of SubPath_type = 7 are included, the SubPath refers to the ES included in the Main TS, so the playback device 401 uses one or more SubPaths of SubPath_type = 5 or 6. During playback while referring to a predetermined one of them, playback can be performed with reference to one SubPath of SubPath_type = 7.

  A detailed example of PlayList playback processing with application_type = 1or2 will be described later with reference to FIGS.

  Next, with reference to the flowcharts of FIGS. 63 and 64, an example of playback processing of a PlayList including a clip of application_type = 3 will be described.

  For example, when the controller 21 in FIG. 53 checks the application_type included in clipinfo () in the Clip_information file referenced by the MainPath of the PlayList to be played back and recognizes that application_type = 3 by the check, the application_type Start playback processing of PlayList with = 3.

  In step S121 of FIG. 63, the controller 21 determines whether or not the Main TS includes an IG (interactive graphics stream).

  If it is determined in step S121 that the Main TS does not include IG, the process proceeds to step S126 in FIG. However, the processing after step S126 will be described later.

  On the other hand, when it is determined in step S121 that the Main TS includes IG, the process proceeds to step S122.

  In step S122, the controller 21 determines whether or not there is a Sub Path of SubPath_type = 3.

  When it is determined in step S122 that there is no Sub Path of SubPath_type = 3, the process proceeds to step S126 in FIG. However, the processing after step S126 will be described later.

  On the other hand, if it is determined in step S122 that there is a Sub Path of SubPath_type = 3, the process proceeds to step S123. In step S123, the controller 21 examines the Clip referred to by the selected Sub Path (Sub Path_Type = 3 Sub Path).

  Then, the controller 21 instructs the storage drive 22 to read the stream specified by the Sub Path, that is, the IG from the Clip. Based on this instruction, the storage drive 22 reads the target IG. Alternatively, the controller 21 reads the target IG from the local storage 24. Then, the controller 21 instructs the interactive graphics decoder 74 to decode the read IG. Accordingly, the process proceeds to step S124.

  In step S124, the interactive graphics decoder 74 decodes the read IG. In step S125, the interactive graphics decoder 74 stores the decoded IG in the preload buffer 414.

  When the process in step S125 is completed in this way, or when it is determined NO in step S121 or S122 as described above, the process proceeds to step S126 in FIG.

  In step S126, the controller 21 determines whether there is a Sub Path of SubPath_type = 2.

  If it is determined in step S126 that there is no SubPath of SubPath_type = 2, the process proceeds to step S129. However, the processing after step S129 will be described later.

  On the other hand, if it is determined in step S126 that there is a Sub Path of SubPath_type = 2, the process proceeds to step S127. In step S127, the controller 21 examines the Clip referred to by the selected SubPath (SubPath_type = 2 Sub Path).

  Then, the controller 21 instructs the storage drive 22 to read out the stream specified by the SubPath from the Clip, that is, the audio stream. Based on this instruction, the storage drive 22 reads the target audio stream. Alternatively, the controller 21 reads out the target audio stream from the local storage 24. Then, the controller 21 instructs the 2nd audio decoder 75-2 to decode the read audio stream, for example. Accordingly, the process proceeds to step S128.

  In step S128, the 2nd audio decoder 75-2 decodes the read audio stream.

  In this way, when the process of step S128 ends or when it is determined NO in the process of step S126, the process proceeds to step S129.

  In step S129, the AV decoder unit 403 decodes the main Clip AV Stream. Here, the main Clip AV Stream is a video stream, an audio stream, or the like referred to by the MainPath of the playlist to be played back, and is included in the Main TS. That is, in the AV decoder unit 403, the 1st video decoder 72, the 1st audio decoder 75-1, and the like execute the process of step S129.

  In step S <b> 130, the AV decoder unit 403 combines the decoded main Clip AV Stream and the decoded SubPath stream, and outputs them. Here, the decoded SubPath stream is the IG decoded in step S124 in FIG. 63 and stored in the preload buffer 414 in step S125, or the audio decoded in step S128 in FIG. A stream.

  In step S131, the controller 21 determines whether or not there is a subpath switching instruction (user switching operation) of SubPath_type = 2 while outputting the stream synthesized in the process of step S130.

  If it is determined in step S131 that a subpath switching instruction of SubPath_type = 2 has been issued while outputting the combined stream, the process returns to step S127, and the subsequent processes are repeated. That is, the audio stream to be combined with the main Clip AV Stream is switched.

  On the other hand, if it is determined in step S131 that there is no SubPath_type = 2 SubPath switching instruction while outputting the combined stream, the playback process of the PlayList of application_type = 3 ends.

  Next, an example of PlayList playback processing including a Clip with application_type = 1or2 will be described with reference to the flowcharts of FIGS.

  For example, when the controller 21 in FIG. 53 investigates the application_type of the Clip referenced by the MainPath of the PlayList to be played back and recognizes that application_type = 1or2 as a result of the investigation, the playback processing of the PlayList with application_type = 1or2 To start.

  In step S141 in FIG. 65, the controller 21 determines whether or not the Main TS includes an IG (interactive graphics stream).

  If it is determined in step S141 that the Main TS does not include IG, the process proceeds to step S146. However, the processing after step S146 will be described later.

  On the other hand, when it is determined in step S141 that Main TS includes IG, the process proceeds to step S142.

  In step S142, the controller 21 determines whether or not there is a Sub Path of SubPath_type = 3.

  If it is determined in step S142 that there is no Sub Path of SubPath_type = 3, the process proceeds to step S146. However, the processing after step S146 will be described later.

  On the other hand, if it is determined in step S142 that there is a Sub Path of SubPath_type = 3, the process proceeds to step S143. In step S143, the controller 21 checks the Clip referred to by the selected SubPath (Sub Path of SubPath_type = 3).

  Then, the controller 21 instructs the storage drive 22 to read the stream specified by the SubPath, that is, the IG from the Clip. Based on this instruction, the storage drive 22 reads the target IG. Alternatively, the controller 21 reads the target IG from the local storage 24. Then, the controller 21 instructs the interactive graphics decoder 74 to decode the read IG. Accordingly, the process proceeds to step S144.

  In step S144, the interactive graphics decoder 74 decodes the read IG. In step S145, the interactive graphics decoder 74 stores the decoded IG in the preload buffer 414.

  When the process of step S145 is completed in this way, or when it is determined NO in step S141 or S142 as described above, the process proceeds to step S146.

  In step S146, the controller 21 determines whether or not there is a Sub Path of SubPath_type = 4.

  If it is determined in step S146 that there is no SubPath of SubPath_type = 4, the process proceeds to step S149 in FIG. However, the processing after step S149 will be described later.

  On the other hand, if it is determined in step S146 that there is a Sub Path of SubPath_type = 4, the process proceeds to step S147. In step S147, the controller 21 examines the Clip referred to by the selected SubPath (Sub Path of SubPath_type = 4).

  Then, the controller 21 instructs the storage drive 22 to read the stream specified by SubPath, that is, the text subtitle stream, from the Clip. Based on this instruction, the storage drive 22 reads the target dext subtitle stream. Alternatively, the controller 21 reads the target text subtitle stream from the local storage 24. Then, the controller 21 instructs the Text-ST composition 76 to decode the read text subtitle stream. Accordingly, the process proceeds to step S148.

  In step S148, the Text-ST composition 76 decodes the read text subtitle.

  Thus, when the process of step S148 is completed or when it is determined NO in the process of step S146 described above, the process proceeds to step S149 of FIG.

  In step S149, the controller 21 determines whether or not there is a Sub Path of SubPath_type = 7.

  When it is determined in step S149 that there is no Sub Path of SubPath_type = 7, the process proceeds to step S152 in FIG. However, the processing after step S152 will be described later.

  On the other hand, when it is determined in step S149 that there is a Sub Path of SubPath_type = 7, the process proceeds to step S150. In step S150, the controller 21 checks the Clip referred to by the selected SubPath (SubPath_type = 7 Sub Path).

  Then, the controller 21 instructs the storage drive 22 to read out the stream specified by the SubPath, that is, the video / audio stream, from the Clip. The storage drive 22 reads the target video / audio stream based on this instruction. Alternatively, the controller 21 reads a target video / audio stream from the local storage 24. Then, the controller 21 instructs the 2nd video decoder 412 / 2nd audio decoder 75-2 to decode the read video / audio stream, for example. Accordingly, the process proceeds to step S151.

  In step S151, the 2nd video decoder 412 / 2nd audio decoder 75-2 decodes the read video / audio stream.

  Thus, when the process of step S151 is completed or when it is determined NO in the process of step S149 described above, the process proceeds to step S152 of FIG.

  In step S152, the controller 21 determines whether or not there is a Sub Path of SubPath_type = 5.

  If it is determined in step S152 that there is no SubPath_type = 5 Sub Path, the process proceeds to step S156. However, the processing after step S156 will be described later.

  In contrast, if it is determined in step S152 that there is a Sub Path of SubPath_type = 5, the process proceeds to step S153. In step S153, the controller 21 checks the Clip referred to by the selected SubPath (SubPath_type = 5 Sub Path).

  In step S154, the controller 21 determines whether or not there is a function for playing back an audio stream in the Clip based on the contents described in stream_attribute () (FIG. 18).

  If it is determined in step S154 that there is no function for reproducing the audio stream in the Clip, the process proceeds to step S156. However, the processing after step S156 will be described later.

  On the other hand, if it is determined in step S154 that there is a function for playing back an audio stream in the Clip, the controller 21 reads the stream specified by the SubPath, that is, the audio stream, from the Clip. The storage drive 22 is instructed. Based on this instruction, the storage drive 22 reads the target audio stream. Alternatively, the controller 21 reads out the target audio stream from the local storage 24. Then, the controller 21 instructs the 2nd audio decoder 75-2 to decode the read audio stream, for example. Accordingly, the process proceeds to step S155.

  In step S155, the 2nd audio decoder 75-2 decodes the read audio stream.

  In this way, when the process of step S155 ends, or when it is determined NO in the process of step S152 or S154 described above, the process proceeds to step S156.

  In step S156, the controller 21 determines whether or not there is a Sub Path of SubPath_type = 6.

  If it is determined in step S156 that there is no SubPath of SubPath_type = 6, the process proceeds to step S161 in FIG. However, the processing after step S161 will be described later.

  On the other hand, if it is determined in step S156 that there is a Sub Path of SubPath_type = 6, the process proceeds to step S157. In step S157, the controller 21 checks the clip referred to by the selected SubPath (SubPath_type = 6 Sub Path).

  In step S158, the controller 21 determines whether or not there is a function for playing back an audio stream in the Clip based on the contents described in stream_attribute () (FIG. 18).

  If it is determined in step S158 that there is no function for playing back an audio stream in the Clip, the controller 21 instructs the storage drive 22 to read a video stream out of the streams specified by the SubPath from the Clip. To do. Based on this instruction, the storage drive 22 reads the target video stream. Alternatively, the controller 21 reads the target video stream from the local storage 24. In this case, however, the controller 21 instructs the prohibition of decoding of the audio stream, and instructs the 2nd video decoder 412 to decode the read video stream, for example. Accordingly, the process proceeds to step S160. However, the processing after step S160 will be described later.

  On the other hand, if it is determined in step S158 that there is a function for playing back the audio stream in the Clip, the controller 21 reads the stream specified by the SubPath, that is, the video stream and the audio stream from the Clip. The storage drive 22 is instructed. Based on this instruction, the storage drive 22 reads the target video stream and audio stream. Then, the controller 21 instructs the 2nd audio decoder 75-2 to decode the read audio stream, for example, and instructs the 2nd video decoder 412 to decode the read video stream, for example. To do. Accordingly, the process proceeds to step S159.

  In step S159, the 2nd audio decoder 75-2 decodes the read audio stream.

  In this way, when the process of step S159 ends, or when it is determined NO in the process of step S158 described above, the process proceeds to step S160.

  In step S160, the 2nd video decoder 412 decodes the read video stream.

  Thus, when the process of step S160 ends or when it is determined NO in the process of step S156 described above, the process proceeds to step S161 of FIG.

  In step S161, the AV decoder unit 403 decodes the main Clip AV Stream. Here, the main Clip AV Stream is a video stream, an audio stream, or the like referred to by the MainPath of the playlist to be played back, and is included in the Main TS. The main Clip AV Stream means, for example, a primary video stream, a primary audio stream, or the like in the picture-in-picture method. That is, in the AV decoder unit 403, the 1st video decoder 72, the 1st audio decoder 75-1, etc. execute the processing of step S161.

  In step S162, the AV decoder unit 403 combines the decoded main Clip AV Stream and the decoded SubPath stream, and outputs them. Here, the decoded SubPath stream is the IG decoded in step S144 in FIG. 65 and stored in the preload buffer 414 in step S145, and the text subtitle decoded in step S148 in FIG. 66, the audio stream decoded in step S151 in FIG. 66, the audio stream decoded in step S155 in FIG. 67, the audio stream decoded in step S159 in FIG. 66, and step S160 in FIG. This means zero or more SubPath streams among the video streams decoded by the above process.

  In step S163, the controller 21 determines whether or not there is a SubPath switching instruction (user switching operation) while outputting the stream synthesized in the process of step S162.

  If it is determined in step S163 that there has been no SubPath switching instruction while outputting the combined stream, the PlayList playback process with application_type = 1or2 ends.

  On the other hand, if it is determined in step S163 that there has been a SubPath switching instruction while outputting the combined stream, the process proceeds to step S164.

  In step S164, the controller 21 checks the SubPath_type of the SubPath selected by the switching instruction. Specifically, when a specific secondary audio _stream or secondary video _stream is selected from one or more secondary_audio_stream (audio_stream_id2) or secondary_video_stream (video_stream_id2) defined in the STN_table, the specific secondary audio _stream based on the STN_table Alternatively, the stream entry corresponding to the secondary video_stream is checked, the subpath_id is obtained from the ref_to_SubPath_id recorded in the stream entry, the Subpath of the PlayList () in FIG. 9 is specified by the subpath_id, and the SubPath ( ) To obtain Subpath_type.

  In step S165, the controller 21 determines whether or not the investigation result in step S164 is SubPath_type = 3.

  If it is determined in step S165 that SubPath_type = 3, the process returns to step S143 in FIG. 65 and the process is repeated. That is, the IG combined with the main Clip AV Stream is switched.

  On the other hand, if it is determined in step S165 that SubPath_type = 3 is not satisfied, the process proceeds to step S166.

  In step S166, the controller 21 determines whether or not the investigation result in step S164 is SubPath_type = 4.

  If it is determined in step S166 that SubPath_type = 4, the process returns to step S147 of FIG. 65 and the process is repeated. That is, the text subtitle combined with the main Clip AV Stream is switched.

  On the other hand, if it is determined in step S166 that SubPath_type = 4 is not satisfied, the process proceeds to step S167.

  In step S167, the controller 21 determines whether or not the investigation result in step S164 is SubPath_type = 7.

  If it is determined in step S167 that SubPath_type = 7, the process returns to step S150 of FIG. 66 and the process is repeated. That is, the video / audio stream synthesized with the main Clip AV Stream is switched.

  On the other hand, when it is determined in step S167 that SubPath_type = 7 is not satisfied, the process proceeds to step S168.

  In step S168, the controller 21 determines whether or not the investigation result in step S164 is SubPath_type = 5.

  If it is determined in step S168 that SubPath_type = 5, the process returns to step S153 in FIG. 67 and the process is repeated. That is, the audio stream to be combined with the main Clip AV Stream is switched.

  On the other hand, if it is determined in step S168 that SubPath_type = 5 is not satisfied, the process proceeds to step S169.

  In step S169, the controller 21 determines whether or not the investigation result in step S164 is SubPath_type = 6.

  If it is determined in step S169 that SubPath_type = 6, the process returns to step S157 of FIG. 67 and the process is repeated. That is, the audio stream and video stream to be combined with the main Clip AV Stream are switched.

  On the other hand, if it is determined in step S169 that SubPath_type = 6 is not satisfied, the PlayList playback process with application_type = 1or2 is terminated.

  As described above, in order to realize an application to which the picture-in-picture method is applied, for example, the definition of the combination of streams that can be simultaneously reproduced in combination with the primary video stream, that is, the secondary video stream, the primary audio stream, the secondary It is necessary to define a combination of an audio stream and a subtitle stream.

  Furthermore, by defining a plurality of combinations, not only switching of the primary video stream and primary audio stream reproduced on the full screen 301 but also switching of the secondary video stream reproduced on the sub-screen 302 and switching of the subtitle stream are performed. Can also be easily performed.

  Different examples of STN_table that define combinations of streams that can be simultaneously reproduced in combination with the primary video stream as described with reference to FIGS. 44 and 45, that is, examples of the eighth STN_table, FIG. 69 and FIG. It explains using.

  That is, FIG. 69 and FIG. 70 are used to define a combination of a secondary video stream, a primary audio stream, a secondary audio stream, and a subtitle stream that can be simultaneously reproduced in combination with a primary video stream. STN_table () syntax for defining a combination with a primary audio stream, a combination of a secondary video stream and a secondary audio stream, or a combination of presentation graphics (PG) or text subtitles for a secondary video stream and a picture-in-picture application It is a figure which shows the example of an 8th example (8th STN_table ()). The playback process executed with reference to the eighth STN_table () can also be executed by the playback device 401 described with reference to FIG. In the figure, the same parts as those in FIGS. 44 and 45 described above are repeated, and the description thereof is omitted.

  An 8-bit field of number_of_audio_stream2_entries indicates the number of secondary audios registered in the STN_table.

  The number of secondary video streams is obtained by audio_stream2_number = audio_stream2_id + 1.

  An 8-bit field of number_of_video_stream2_entries indicates the number of secondary videos registered in the STN_table.

  The number of secondary video streams is obtained by video_stream2_number = video_stream2_id + 1.

  In addition, in the stream_entry () block of each video_stream2_id, the type field must be set to 2.

  The 8-bit field of number_of_PiP_PG_textST_stream_entries indicates the number of PG / text subtitle streams for the PiP application registered in the STN_table.

  The number of PG (presentation graphics) or text subtitle streams for the PIP application is obtained by PiP_PG_textST_stream_number = PiP_PG_textST_stream_id + 1.

  A block of comb_info_Secondary_audio_Primary_audio () indicates a combination of secondary audio and primary audio.

  In the comb_info_Secondary_audio_Primary_audio () block, the 8-bit field of number_of_audio_stream_ref_entries indicates the number of primary audio streams that can be combined with the secondary audio stream. The 8-bit field of audio_stream_id_ref indicates the ID of the primary audio stream that can be combined with the secondary audio stream.

  Next, a block of comb_info_Secondary_video_Secondary_audio () indicates a combination of a secondary video stream and a secondary audio stream.

  In the comb_info_Secondary_video_Secondary_audio () block, the 8-bit field of number_of_audio_stream2_ref_entries indicates the number of secondary audio streams that can be combined with the secondary video stream. The 8-bit field of audio_stream_id2_ref indicates the ID of a secondary audio stream that can be combined with the secondary video stream.

  A block of comb_info_Secondary_video_PiP_PG_textST () indicates a combination of a secondary video stream and a PG / text subtitle stream for a picture-in-picture application.

  In the comb_info_Secondary_video_PiP_PG_textST () block, the 8-bit field of number_of_PiP_PG_textST_ref_entries indicates the number of PG / text subtitle streams for the PIP application that can be combined with the secondary video stream. Also, the 8-bit field of PiP_PG_textST_stream_id_ref indicates the ID of the PG / text subtitle stream for the picture-in-picture application that can be combined with the secondary video stream.

  As in the case described with reference to FIG. 35, FIG. 41, FIG. 42, or FIG. 44 and FIG. 45, each number instead of each ID, for example, an audio stream number (audio_stream_number) instead of audio_stream_id Or audio stream number 2 (audio_stream_number2) may be used instead of audio_stream_id2. The same applies to the video stream and the subtitle stream.

  69 and 70, a combination of secondary audio and primary audio, a combination of secondary video and secondary audio, and a combination of secondary video and a PG / text subtitle for a picture-in-picture application can be defined.

  Therefore, a secondary video stream (# 2 video stream), a primary audio stream (# 1 audio stream), a secondary audio stream (# 2 audio stream), and a subtitle that can be played back simultaneously with the primary video stream (# 1 video stream) In addition to the combination of streams (PG TextST stream), it is possible to prevent a conflicting combination from being selected as a combination of a PG / text subtitle stream (PiP PG TextST stream) for a picture-in-picture application.

  In the playback device 401, a plurality of registers are provided in a part of the rewritable memory area of the memory 23 of the playback device 401, and playback processing is executed based on the values set in these registers. The

  Among the registers, for example, a value set for a function unique to the playback device 401, such as a register value in which it is set whether or not the playback device 401 can play the # 2 audio stream, the value is There is no change. On the other hand, the setting of the predetermined one of the registers is changed, for example, when a predetermined operation input is received from the user or by a navigation command generated by the navigation program.

  The navigation command may be recorded separately from the stream data on a recording medium, an internal hard disk, or the like, preloaded by the playback device 401, and supplied to the controller 21 or embedded in the stream data. The data may be recorded on a recording medium, an internal hard disk, and the like, loaded as the playback device 401 plays back the stream data, and supplied to the controller 21.

  The register holds values indicating various settings necessary for the reproduction process in the reproduction apparatus 401. The register defines, for example, a register that defines a subtitle stream (PG TextST stream) to be played back by the playback device 401, and a PG / text subtitle stream (PiP PG TextST stream) for a picture-in-picture application that is played back by the playback device 401. Register, register indicating stream ID of subtitle stream (PG TextST stream) reproducible by playback device 401, register indicating language code number (or ID) of subtitle stream (PG TextST stream) reproducible by playback device 401, initial It is preferable that a register indicating an ID of a set language, a register indicating a country code number (or ID) corresponding to a subtitle stream (PG TextST stream) that can be played back by the playback device 401, and the like are included.

  When the picture-in-picture application is activated, the playback device 401 takes into account the ease of display, and the subtitle stream (PG TextST stream) or the PG / text subtitle stream (PiP PG TextST stream) for the picture-in-picture application Only one of them may be displayed, or both a subtitle stream (PG TextST stream) and a PG / text subtitle stream (PiP PG TextST stream) for picture-in-picture applications can be displayed It may be a thing.

  71, when the picture-in-picture application is activated, whether or not the playback device 401 plays back and displays the primary (that is, not for the picture-in-picture application) subtitle stream (PG TextST stream), and A register to be referred to in order to determine which stream is to be played back when displaying, and whether to display and display a PG / text subtitle stream (PiP PG TextST stream) for a picture-in-picture application, and A description will be given of a register that is referred to in order to determine which stream is to be reproduced in the case of reproduction display.

The # 1 register is a register indicating the ID number of the primary PG TextST stream.
That is, the # 1 register indicates a PG TextST stream to be reproduced in the primary PG TextST registered in the STN_table of the current PlayItem.

  In the # 1 register, when the value of disp_flag (display_flag) is 0, the primary PG TextST stream is not displayed, and when the value of disp_flag is 1, the primary PG TextST stream is displayed. In the # 1 register, PG TextST stream number is the ID number of the primary PG Text ST registered in STN_table () of the current PlayItem ().

  The # 2 register is a register indicating the ID number of the PG TextST stream for the picture-in-picture application.

  In the # 2 register, when the value of disp_flag is 0, the PG TextST stream for the picture-in-picture application is not displayed, and when the value of disp_flag is 1, the PG TextST stream for the picture-in-picture application is displayed. . PiP PG TextST stream number indicates the ID number of the PG TextST stream for picture-in-picture application registered in STN_table () of the current PlayItem (). When the value of invalid_flag is 0, the ID number of the PG TextST stream for the picture-in-picture application is valid, and when it is 1, it is invalid.

  When 0xFF is registered in PG TextST stream number, the PG TextST stream is not selected or does not exist. When 0xFF is registered in the PiP PG TextST stream number, the TextST stream for picture-in-picture is not selected or does not exist.

  Also, even when other than 0xFF is registered in PG TextST stream number or PiP PG TextST stream number, the validity or invalidity of display is determined by the combination of invalid_flag and disp_flag. That is, when disp_flag = 0 in the # 1 register and the # 2 register is in the display invalid state, the PG TextST stream number or the PiP PG TextST stream number is not always set to 0xFF. For example, when disp_flag = 0 in the # 1 register or # 2 register is in the display invalid state, even if a value other than 0xFF is registered in the PG TextST stream number or PiP PG TextST stream number, the PG TextST stream number or PiP PG TextST stream The value registered in number is ignored and not displayed.

  By referring to the values set in these registers, the playback device 401 executes playback processing of each stream.

  For example, when the picture-in-picture application is activated, the playback apparatus 401 has only one of a subtitle stream (PG TextST stream) and a PG / text subtitle stream (PiP PG TextST stream) for the picture-in-picture application. When only the PG TextST stream number corresponding to the value set in the PG TextST stream number of the # 1 register is reproduced, the value of disp_flag is 1, and the invalid_flag of the # 2 register is used. Is set to 1, or the display of Secondary Video is disabled, that is, the picture-in-picture display is not executed. When the value of the # 1 register is changed, the playback of the primary PG TextST stream is also changed immediately.

  In addition, when the value indicated by the PG TextST stream number in the # 1 register does not match the number of the PG TextST stream registered in the STN_table of PlayItem, the primary PG TextST stream to be played back is the PG TextST stream Selected by automatic selection routine.

  In the automatic selection routine, first, a language in which a value that can be displayed on the playback device 401 or a value that represents a language that cannot be displayed is referred to, and a language that the playback device 401 has display capability is preferentially selected. Next, priority is given to the language code equivalent to the one set in the playback device 401, and then the priority is selected from the smallest number in the order registered in STN_table (). .

  Then, the PG TextST stream for the picture-in-picture application corresponding to the value set in the PiP PG TextST stream number of the # 2 register is played back when the secondary video display is enabled and the # The value of disp_flag in register 2 is set to 1, invalid_flag is set to 0, and the ID number of the PiP PG TextST for the picture-in-picture application registered in the # 2 register is the picture-in registered in the STN_table of the current PlayItem This is when it is included in the PiP PG TextST stream for picture applications.

  Even though the secondary video display is enabled and the disp_flag value of the # 2 register is set to 1 and the invalid_flag is set to 0, the value indicated in the # 2 register is the STN_table of the PlayItem. If the number does not match the registered PiP PG TextST stream number, the PG TextST stream for the picture-in-picture application to be reproduced is selected by the PiP PG TextST stream automatic selection routine.

  In the automatic selection routine, first, a language in which a value that can be displayed on the playback device 401 or a value that represents a language that cannot be displayed is registered, and the language that the playback device 401 has display capability is preferentially selected. Next, priority is given to the language code set in the playback device 401, and then the number is selected in descending order of the number registered in STN_table ().

  Next, an example in which a primary PG TextST stream is played back when executing picture-in-picture display will be described with reference to FIG.

  For example, when the disp_flag of the # 1 register is 1 in a state where the picture-in-picture display is not executed (a state indicated as “No PiP” in the figure), the # 2 register is set regardless of the setting of the # 2 register. A value set in PG TextST stream number of one register is referred to, and PG TextST of PG TextST number = 0x05 registered in STN_table () is reproduced and displayed.

  Then, when a command is generated by the navigation program or a predetermined operation input is performed by the user, execution of picture-in-picture display is started (PiP on in the figure). Even if execution of picture-in-picture display is started, if invalid_flag of the # 2 register is set to 1 (for example, invalid_flag is set to 1 by a navigation command generated by the navigation program), the user can use the picture-in-picture application. Even if the display of the PiP PG TextST stream for use is selected (even if disp_flag = 1), the PiP PG TextST stream for the picture-in-picture application is not displayed. That is, even when picture-in-picture display is being executed (in the figure, indicated as “PiP enabled” in the figure), the value set in the PG TextST stream number of the # 1 register is referred to, and STN_table ( PG TextST registered in PG TextST number = 0x05 is reproduced and displayed.

  The navigation command may be recorded on a recording medium or the like separately from the stream data, preloaded by the playback device 401, and supplied to the controller 21. However, the navigation command is embedded in the stream data and the like. When the stream data is recorded and loaded as the playback device 401 is played back and supplied to the controller 21, the command execution timing can be easily controlled.

  In addition, as an example of a case where a navigation command that can set invalid_flag is generated, for example, a content creator can automatically change invalid_flag in a stream (for example, in an interactive graphic stream (IG stream)). Navigation commands may be embedded, for example, button information and control information for changing invalid_flag are recorded in the IG stream, and buttons that can be selected and operated by the user can be displayed during stream playback. Accordingly, the invalid_flag may be changed based on an operation input in which the corresponding button is pressed (selected) by the user.

  Then, when the execution of picture-in-picture display is finished (PiP off in the figure) and the picture-in-picture display is not executed again (state shown as “No PiP” in the figure) If the disp_flag of the # 1 register is 1, regardless of the setting of the # 2 register, the value set in the PG TextST stream number of the # 1 register is referred to and the PG TextST number registered in the STN_table () = 0x05 PG TextST is played back and displayed.

  Next, an example in which a PG TextST stream for a picture-in-picture application is played back when executing picture-in-picture display will be described with reference to FIG.

  For example, when the disp_flag of the # 1 register is 1 in a state where the picture-in-picture display is not executed (a state indicated as “No PiP” in the figure), the # 2 register is set regardless of the setting of the # 2 register. A value set in PG TextST stream number of one register is referred to, and PG TextST of PG TextST number = 0x05 registered in STN_table () is reproduced and displayed.

  Then, a command (for example, a navigation command embedded in the stream) is generated by the navigation program, or execution of picture-in-picture display is started by a predetermined operation input by the user (in the figure, PiP on), when invalid_flag of the # 2 register is set to 0 and disp_flag is set to 1, the value set in the PiP PG TextST stream number of the # 2 register is referred to in the STN_table () A PiP PG TextST stream for a picture-in-picture application with registered PiP PG TextST number = 0x02 is displayed. If the user does not select display of the PiP PG TextST stream for the picture-in-picture application (disp_flag = 0), the PiP PG TextST stream for the picture-in-picture application is not displayed.

  Then, when the execution of picture-in-picture display is finished (PiP off in the figure) and the picture-in-picture display is not executed again (state shown as “No PiP” in the figure) If the disp_flag of the # 1 register is 1, regardless of the setting of the # 2 register, the value set in the PG TextST stream number of the # 1 register is referred to and the PG TextST number registered in the STN_table () = 0x05 PG TextST is played back and displayed.

  72 and 73, when 0xFF is registered in the PiP PG TextST stream number, the TextST stream is not selected or does not exist. For example, even when a value other than 0xFF is registered in the PiP PG TextST stream number, the display validity or invalidity is determined by the combination of invalid_flag and disp_flag. That is, when invalid_flag = 1 and the display is invalid, the value registered in PG TextST stream number or PiP PG TextST stream number is ignored and displayed even if other than 0xFF is registered in PiP PG TextST stream number It is made not to be done.

72 and 73, the # 1 register for primary video image (caption) information and the # 2 register for secondary video (PiP) image (caption) information are set separately. The number of registers may be one.

  Furthermore, two disp_flags in this example are set to one, and a PG TextST stream that is an image (caption) stream of a primary video or a PiP PG_TextST_stream that is an image (caption) stream of a secondary video (PiP) is displayed. Or a flag indicating whether or not image information is indicated. In this case, invalid_flag is configured to be a flag that determines which image of primary video image information or PiP image information is to be displayed.

  Next, the PG TextST playback processing of an application including PiP (picture-in-picture) will be described with reference to the flowchart of FIG.

  In step S301, the controller 21 checks the corresponding PG TextST from the STN_table () of the current PlayItem from the PG TextST number (PG TextST stream number) set in the # 1 register.

  In step S302, the controller 21 specifies a desired PG TextST, reads the specified stream from the clip recorded in the recording medium 11 or the local storage 24, and performs reproduction according to disp_flag or the like.

  For example, when the disp_flag of the # 1 register is 1, the controller 21 refers to the value set in the PG TextST stream number of the # 1 register, and determines the PG TextST of the PG TextST number registered in STN_table (). Read and supply to the AV decoder unit 403. The read PG TextST is supplied to the presentation graphics plane generation unit 93 by processing of each buffer, PID filter, switch, and the like, and a presentation graphics plane as a rendering image is generated, and is supplied to the video data processing unit 96. Supplied, synthesized with a primary video stream, etc., and reproduced and output together with corresponding audio data.

  In step S303, the controller 21 starts picture-in-picture display (PiP is turned ON).

  Here, when picture-in-picture display is set in synchronization with the main path, the start of picture-in-picture display (PiP ON) and the end of picture-in-picture display (PiP OFF) depend on the picture-in-picture. This process is executed at a predetermined timing indicated by IN_time and Out_time of SubPlayItem in FIG. On the other hand, when picture-in-picture display is set asynchronously with the main path, the start of picture-in-picture display (PiP ON) and the end of picture-in-picture display (PiP OFF) are input by user operation. Or by a navigation command.

  In step S304, the controller 21 embeds invalid_flag of the # 2 register as necessary by a navigation command or the like embedded in the stream or stored and preloaded at a position different from the stream. Change the setting value.

  In step S305, the controller 21 determines whether invalid_flag = 0 (PiP PG TextST is valid) in the # 2 register.

  When it is determined in step S305 that invalid_flag = 0, in step S306, the controller 21 determines whether or not disp_flag = 1 (PiP PGTextST display) in the # 2 register. That is, when the picture-in-picture display is set asynchronously with the main path, the controller 21 instructs the picture-in-picture display by the user's operation input, and whether the value of disp_flag of the # 2 register is set to 1 Judge whether or not.

  In addition, in the process of step S305 and the process of step S306, any determination may be made first.

  If it is determined in step S306 that disp_flag of the # 2 register is 1, PiP PG TextST display processing described later is executed using the flowchart of FIG. 75 in step S307, and the processing proceeds to step S311 described later. .

  At this time, when disp_flag = 1 is set in the # 1 register, the primary PG TextST stream may be displayed together with the PiP PG TextST stream for the picture-in-picture application.

  If it is determined in step S305 that invalid_flag = 0 is not satisfied (that is, invalid_flag = 1), or if it is determined in step S306 that disp_flag = 1 is not satisfied (that is, disp_flag = 0), In step S308, the controller 21 determines whether or not disp_flag = 1 (PGTextST display) in the # 1 register.

  If it is determined in step S308 that disp_flag = 1 in the # 1 register, in step S309, the controller 21 selects a PG TextST corresponding to the PG TextST number (PG TextST stream number) set in the # 1 register. The PG TextST to be played back is identified and displayed by checking the STN_table () of the current PlayItem, and the process proceeds to step S311 described later. It should be noted that disp_flag is set to one, a PG TextST stream that is an image (caption) stream of a primary video, or a PiP PG_TextST_stream that is a PiP image (caption) stream, that is, “ In the case of “flag indicating whether or not image information is indicated”, the same disp_flag is referred to in the processing of step S306 and step S308.

  For example, when the disp_flag of the # 1 register is 1, the controller 21 refers to the value set in the PG TextST stream number of the # 1 register, and determines the PG TextST of the PG TextST number registered in STN_table (). Read and supply to the AV decoder unit 403. The read PG TextST is supplied to the presentation graphics plane generation unit 93 by processing of each buffer, PID filter, switch, and the like, and a presentation graphics plane as a rendering image is generated, and is supplied to the video data processing unit 96. Supplied, synthesized with a primary video stream, etc., and reproduced and output together with corresponding audio data.

  If it is determined in step S308 that disp_flag is not 1 (that is, disp_flag = 0), in step S310, the controller 21 does not display PG or TextST (for example, any of the full screen 301 and the child screen 302). The subtitles corresponding to are not displayed), and the process proceeds to step S311 described later.

  After the process of step S307, step S309, or step S310 is completed, in step S311, the controller 21 determines whether or not the picture-in-picture (PiP) display is terminated.

  As described above, when picture-in-picture display is set in synchronization with the main path, the start of picture-in-picture display (PiP ON) and the end of picture-in-picture display (PiP OFF) It is executed at a predetermined timing regardless of the input. On the other hand, when picture-in-picture display is set asynchronously with the main path, the start of picture-in-picture display (PiP ON) and the end of picture-in-picture display (PiP OFF) are input by user operation. Is set by

  If it is determined in step S311 that picture-in-picture (PiP) display is not terminated, the process returns to step S304, and the subsequent processes are repeated. If it is determined in step S311 that picture-in-picture (PiP) display is to be terminated, the processing is terminated.

  When picture-in-picture display is executed by such processing, the primary (ie, not for picture-in-picture application) subtitle stream (PG TextST) is based on the values set in the # 1 register and the # 2 register. (Stream) is reproduced and displayed, and whether or not a PG / text subtitle stream (PiP PG TextST stream) for a picture-in-picture application is reproduced and displayed is determined. The stream to be played back and displayed is basically determined by the setting of the # 1 or # 2 register, but the setting of the # 1 or # 2 register is registered in the STN_table (). If the content does not match, the stream to be reproduced and displayed is determined by the above-described automatic selection routine.

  Next, the PiP PG TextST display process executed in step S307 in FIG. 74 will be described with reference to the flowchart in FIG.

  In step S351, the controller 21 acquires the PiP PG TextST number (PiP PG TextST stream number) set in the # 2 register. For example, the PiP PG TextST number (PiP PG TextST stream number) set in the # 2 register is selected and set by referring to the stream number table, or the setting value when the previous picture-in-picture display was performed. Is held, is set based on a user operation input, or is a value set by the navigation command described above.

  In step S352, the controller 21 determines whether or not a value obtained by adding 1 to the PiP_PG_textST_stream_id_ref value of the eighth STN_table described with reference to FIGS. 69 and 70 is stored in the # 2 register.

  If it is determined in step S352 that the value obtained by adding 1 to the value of PiP_PG_textST_stream_id_ref of the eighth STN_table (PiP_PG_textST_id_ref + 1) is stored in the # 2 register, the controller 21 sets the value in the # 2 register in step S353 In other words, the PiP PG TextST corresponding to the saved PiP PG TextST stream number is displayed, and the process returns to step S307 in FIG. 74 and proceeds to step S311.

  That is, the controller 21 reads PG TextST corresponding to the value set in the PiP PG TextST stream number of the # 2 register from the recording medium 11 or the local storage 24 via the storage drive 22. The read PG TextST is supplied to the presentation graphics plane generation unit 93 by processing of each buffer, PID filter, switch, and the like, and a presentation graphics plane as a rendering image is generated, and is supplied to the video data processing unit 96. Is supplied, synthesized with a primary video stream, a secondary video stream, and the like, and reproduced and output together with corresponding audio data.

  If it is determined in step S352 that a value obtained by adding 1 to the PiP_PG_textST_stream_id_ref value of the eighth STN_table is not stored in the # 2 register, the controller 21 in step S354 determines that the stream_attribute () of the STN_table (FIG. 18). ) PG_language_code and textST_language_code, refer to another register different from the # 1 register or # 2 register, that is, a register in which the initial setting of the language is registered, and the initial setting of the set language is It is determined whether or not it is a reproducible language of the (playing device 401) and is defined by a value (PiP_PG_textST_id_ref + 1) obtained by adding 1 to the PiP_PG_textST_id_ref value of the STN_table.

  If it is determined in step S354 that the initial setting set in another register is a reproducible language and is defined by a value obtained by adding 1 to the PiP_PG_textST_id_ref value of STN_table, in step S355, The controller 21 displays the PiP PG TextST corresponding to the initially set language, and the process returns to step S307 in FIG. 74 and proceeds to step S311.

  That is, the controller 21 causes the PG TextST corresponding to the value set in the register in which the initial language setting is registered to be read from the recording medium 11 or the local storage 24 via the storage drive 22. The read PG TextST is supplied to the presentation graphics plane generation unit 93 by processing of each buffer, PID filter, switch, and the like, and a presentation graphics plane as a rendering image is generated, and is supplied to the video data processing unit 96. Supplied, synthesized with a primary video stream, etc., and reproduced and output together with corresponding audio data.

  If it is determined in step S354 that the initial setting set in another register is not a reproducible language or is not defined in PiP_PG_textST_id_ref + 1 of the eighth STN_table, in step S356, the controller 21 The set value of i is i = 0. Here, the value i is a value representing the number of PiP_PG_textST_id_ref defined in the STN_table that confirms whether or not reproduction is possible in step S357 described later.

  In step S357, the controller 21 refers to a predetermined register, that is, a register in which information indicating a language that can be decoded by the playback device 141 is registered, thereby confirming the language that the playback device 141 can decode. It is determined whether or not the PiP PG textST corresponding to the value obtained by adding 1 to the i-th PiP_PG_textST_id_ref can be reproduced.

  If it is determined in step S357 that reproduction cannot be performed, in step S358, the controller 21 sets the value i to i = i + 1, the process returns to step S357, and the subsequent processes are repeated. That is, it is determined whether or not PiP PG textST can be reproduced in order from the lowest number in the order registered in STN_table ().

  If it is determined in step S357 that playback is possible, in step S359, the controller 21 displays PG textST defined by a value obtained by adding 1 to the i-th PiP_PG_textST_id_ref, and the processing is performed in step S307 of FIG. Return to step S311.

  That is, the controller 21 determines the PG TextST determined by adding 1 to the i-th PiP_PG_textST_id_ref determined to be reproducible in step S357 from the recording medium 11 via the storage drive 22 or the local storage. 24. The read PG TextST is supplied to the presentation graphics plane generation unit 93 by processing of each buffer, PID filter, switch, and the like, and a presentation graphics plane as a rendering image is generated, and is supplied to the video data processing unit 96. Supplied, synthesized with a primary video stream, etc., and reproduced and output together with corresponding audio data.

  By such processing, when the value indicated in the PiP PG TextST stream number of the # 2 register matches the number of the PiP PG TextST stream registered in the STN_table of the PlayItem, it is indicated in the PiP PG TextST stream number. PiP PG textST corresponding to the value is played back. If the value indicated by the PiP PG TextST stream number in the # 2 register does not match the number of the PiP PG TextST stream registered in the STN_table of the PlayItem, the PG TextST for the picture-in-picture application to be played back A stream is selected by a PiP PG TextST stream automatic selection routine.

  Next, different examples of STN_table that define combinations of streams that can be simultaneously reproduced in combination with the primary video stream, that is, a ninth example of STN_table will be described with reference to FIGS. 76 to 78. FIG.

  That is, FIGS. 76 to 78 are particularly useful for defining a combination of a secondary video stream, a primary audio stream, a secondary audio stream, and a subtitle stream that can be simultaneously played in combination with the primary video stream. In addition to defining the primary audio, the combination of secondary audio stream and primary audio stream, the combination of secondary video stream and secondary audio stream, presentation graphics (PG) or text subtitles for secondary video stream and picture-in-picture applications In addition to the combination of secondary video and picture Is a diagram showing an example of a ninth example of the syntax of STN_table () in the case of further defining the combination of the primary audio for turbocharger applications (ninth STN_table ()).

  The playback process executed with reference to the ninth STN_table () can also be executed in the playback device 401 described with reference to FIG. In the figure, the same parts as those shown in FIGS. 69 and 70 are repeated, and the description thereof is omitted.

  A newly added 8-bit field of number_of_PiP_primary_audio_stream_entries indicates the number of primary audio streams for picture-in-picture display registered in the STN_table. The number of primary audio streams for picture-in-picture display is obtained by PiP_primary_audio_stream_number = PiP_primary_audio_stream_id + 1.

  A block of comb_info_Secondary_video_PiP_primary_audio () indicates a combination of a secondary video stream and a primary audio stream for picture-in-picture display.

  In the comb_info_Secondary_video_PiP_primary_audio () block, the 8-bit field of number_of_PiP_primary_audio_ref_entries_SV indicates the number of primary audio streams for picture-in-picture display that can be combined with the secondary video stream. The 8-bit field of PiP_primary_audio_id_ref_SV indicates the ID of the primary audio stream for picture-in-picture display that can be combined with the secondary video stream.

  As in the case of each STN_table described above, each number is used instead of each ID, for example, an audio stream number (audio_stream_number) is used instead of audio_stream_id, or an audio stream number 2 (audio_stream_number2) is used instead of audio_stream_id2 May be used. The same applies to the video stream and the subtitle stream.

  According to the ninth STN_table () of FIGS. 76 to 78, the combination of secondary audio and primary audio, the combination of secondary video and secondary audio, and the combination of secondary video and PG / text subtitles for picture-in-picture application. In addition, a combination of secondary video and primary audio for picture-in-picture display can be defined.

  Accordingly, a secondary video stream (# 2 video stream), a primary audio stream (# 1 audio stream), a secondary audio stream (# 2 audio stream), a subtitle stream (which can be played back simultaneously with the primary video stream (# 1 video stream) ( PG TextST stream) and PG / text subtitle stream (PiP PG TextST stream) for picture-in-picture applications, as well as conflicting combinations of secondary video and primary audio for picture-in-picture display You can prevent it from being selected.

  Further, in the playback device 141, as described above, a plurality of registers are provided in a part of the rewritable memory area of the memory 23 of the playback device 141, and based on the values set in these registers, Playback processing is executed. In addition to the registers described with reference to FIG. 70, a part of the rewritable memory area of the memory 23 of the playback device 141 further includes audio for playback when the picture-in-picture application is activated. Registers for setting are provided, and when the ninth STN_table () of FIGS. 76 to 78 is used, these registers are referred to.

  79, when the ninth STN_table () of FIGS. 76 to 78 is applied, when the picture-in-picture application is started, the playback device 141 determines which audio stream to play back. A register that is referred to for this purpose will be described.

  The # 3 register is a register indicating the ID number of the primary audio stream. That is, the # 3 register is a register in which a primary audio stream to be played is set among primary audio streams registered in the STN_table of the current PlayItem.

  When picture-in-picture display is not being performed, the audio stream having the ID number set in the # 3 register is played back. However, when the audio stream having the ID number set in the # 3 register is not registered in the STN_table () of the current PlayItem (), the audio stream to be played back is selected by the primary audio stream automatic selection routine.

  On the other hand, whether or not the audio stream having the ID number set in the # 3 register is reproduced during picture-in-picture display depends on the setting of the # 5 register, which will be described later. A specific example of audio reproduction during picture-in-picture display will be described later.

  The # 4 register is a register indicating the ID number of the secondary audio stream. That is, the # 4 register is a register in which a secondary audio stream to be played is set among secondary audio streams registered in the STN_table of the current PlayItem.

  When picture-in-picture display is not being performed, the audio stream of the ID number set in the # 4 register is played based on user operation input or based on various settings. However, when the audio stream having the ID number set in the # 4 register is not registered in the STN_table () of the current PlayItem (), the audio stream to be reproduced is selected by the secondary audio stream automatic selection routine. On the other hand, an audio stream having an ID number set in the # 4 register may be played back when picture-in-picture is displayed. A specific example of audio reproduction during picture-in-picture display will be described later.

  The # 5 register is a register indicating the ID number of the primary audio stream for picture-in-picture display. That is, the # 5 register is a register in which a primary audio stream for picture-in-picture display to be reproduced is set among primary audio streams for picture-in-picture display registered in the STN_table of the current PlayItem. . In the # 5 register, when the value of invalid_flag is 0, the ID number of the primary audio stream for the picture-in-picture application is valid, and when it is 1, it is invalid.

  If the audio stream with the ID number set in the # 5 register is not a primary audio stream for picture-in-picture display that is allowed to be combined with a secondary video stream in STN_table () of PlayItem (), the audio to be played back The stream is selected by a routine for automatically selecting a primary audio stream for picture-in-picture display.

  In each audio stream automatic selection routine, first, a register indicating whether audio can be played back by the playback apparatus 141 or audio that cannot be played back is referred to, and the playback apparatus 141 preferentially selects audio having playback capability. Next, audio data in a language equivalent to the language code set in the playback device 141 is selected with priority, and then in order from the lowest number in the order registered in STN_table () It is designed to be prioritized and selected.

  If 0xFF is registered in Secondary audio stream number, the secondary audio stream is not selected or does not exist. If 0xFF is registered in the PiP primary audio stream number, the primary audio stream for picture-in-picture display is not selected or does not exist.

  Further, even when other than 0xFF is registered in the PiP primary audio stream number, the validity or invalidity of reproduction is determined by invalid_flag. That is, when invalid_flag = 1 and the playback is disabled, the PiP primary audio stream number is not always set to 0xFF. For example, when invalid_flag = 1 and playback is disabled, even if a value other than 0xFF is registered in PiP primary audio stream number, the value registered in PiP primary audio stream number is ignored and is not played back. Yes.

  By referring to the values set in these registers, the playback device 141 executes playback processing of each stream.

  Next, referring to FIG. 80, when executing picture-in-picture display, primary audio and secondary audio are played back (secondary audio is played back based on user operation input or based on various settings). An example of the case will be described.

  For example, in a state where picture-in-picture display is not being executed (a state indicated as “No PiP” in the figure), the value set in the primary audio stream number of the # 3 register is referred to, and STN_table () The primary audio stream of audio stream number = 0x05 registered in is reproduced.

  At this time, if a value other than 0xFF is registered in the secondary audio stream number of the # 4 register, the audio stream having the set ID number is reproduced based on a user operation input or based on various settings. .

  Then, when a command is generated by the navigation program or a predetermined operation input is performed by the user, execution of picture-in-picture display is started (PiP on in the figure). Even if the execution of picture-in-picture display is started, if invalid_flag of the # 5 register is set to 1 (for example, invalid_flag is set to 1 by a navigation command generated by the navigation program), it is used for a picture-in-picture application. The primary audio stream is not played back. That is, even when picture-in-picture display is being performed (in the state indicated as “PiP enabled” in the drawing), the value set in the Primary stream number of the # 3 register and the # 4 register The value set in Secondary stream number is referred to, audio stream number = 0x07 registered in STN_table () is reproduced, and audio stream 2 is set based on user operation input or various settings. An audio stream of number = 0x01 is mixed and reproduced.

  Then, when the execution of picture-in-picture display is finished (PiP off in the figure) and the picture-in-picture display is not executed again (state shown as “No PiP” in the figure) , The value set in the Primary stream number of the # 3 register is referred to, and the audio stream number = 0x05 registered in STN_table () is reproduced.

  Also at this time, if other than 0xFF is registered in the secondary audio stream number of the # 4 register, the audio stream of the set ID number is reproduced based on the user's operation input or based on various settings. The

  Next, an example in which a primary audio stream for a picture-in-picture application is played back when executing picture-in-picture display will be described with reference to FIG.

  For example, in a state where picture-in-picture display is not executed (in the state indicated as “No PiP” in the figure), the value set in the Primary stream number of the # 3 register is referred to, and STN_table () is referred to. The registered audio stream number = 0x05 is played.

  At this time, if a value other than 0xFF is registered in the secondary audio stream number of the # 4 register, the audio stream having the set ID number is reproduced based on a user operation input or based on various settings. .

  Then, a command (for example, a navigation command embedded in the stream) is generated by the navigation program, or execution of picture-in-picture display is started by a predetermined operation input by the user (in the figure, PiP on) and when the invalid_flag in the # 5 register is set to 0, the value set in the PiP primary audio stream number in the # 5 register is referenced and the PiP primary audio registered in the STN_table () A primary audio stream for a picture-in-picture application with stream number = 0x07 is reproduced.

  Then, when the execution of picture-in-picture display is finished (PiP off in the figure) and the picture-in-picture display is not executed again (state shown as “No PiP” in the figure) , The value set in the Primary stream number of the # 3 register is referred to, and the audio stream number = 0x05 registered in STN_table () is reproduced.

  Also at this time, if other than 0xFF is registered in the secondary audio stream number of the # 4 register, the audio stream of the set ID number is reproduced based on the user's operation input or based on various settings. The

  80 and 81, when 0xFF is registered in PiP primary audio stream number, the primary audio stream for picture-in-picture display is not selected or does not exist. For example, even when a value other than 0xFF is registered in the PiP primary audio stream number, whether playback is valid or invalid is determined by the invalid_flag setting. That is, when invalid_flag = 1 and playback is disabled, even if a value other than 0xFF is registered in PiP primary audio stream number, the value registered in PiP primary audio stream number is ignored and is not played back. Yes.

  Next, the audio stream reproduction processing 1 for an application including PiP (picture-in-picture) will be described with reference to the flowchart in FIG.

  In step S401, the controller 21 checks the corresponding primary audio from the STN_table () of the current PlayItem based on the primary audio stream number (primary audio stream number) set in the # 3 register.

  In step S402, the controller 21 specifies a desired primary audio stream, reads the specified stream from the clip recorded on the recording medium 11 or the local storage 24, and performs playback.

  For example, the controller 21 refers to the value set in the primary audio stream number of the # 3 register, reads the primary audio stream of the primary audio stream number registered in STN_table (), and supplies it to the AV decoder unit 403. . The read primary audio stream is supplied to the 1st audio decoder 75-1 through the processing of each buffer, PID filter, switch, etc., decoded, and reproduced and output together with the corresponding video data.

  In step S403, the controller 21 starts picture-in-picture display (PiP is turned ON).

  Here, when picture-in-picture display is set in synchronization with the main path, the start of picture-in-picture display (PiP ON) and the end of picture-in-picture display (PiP OFF) are the user's operation inputs. Regardless, it is executed at a predetermined timing. On the other hand, when picture-in-picture display is set asynchronously with the main path, the start of picture-in-picture display (PiP ON) and the end of picture-in-picture display (PiP OFF) are input by user operation. Is set by

  In step S404, the controller 21 embeds invalid_flag of the # 5 register as necessary by a navigation command or the like embedded in the stream or stored in a position different from the stream and preloaded. Change the setting value.

  In step S405, the controller 21 determines whether or not invalid_flag = 0 (PiP primary audio stream is valid) in the # 5 register.

  If it is determined in step S405 that invalid_flag = 0, in step S406, the controller 21 responds from the PiP primary audio stream number set in the # 5 register as described with reference to FIG. The primary audio for picture-in-picture display to be performed is checked from STN_table () of the current PlayItem.

  In step S407, the controller 21 identifies the primary audio for picture-in-picture display to be played back, is read from the clip and supplied to the AV decoder unit 403 for playback in the same manner as described above, and the process will be described later. The process proceeds to step S409.

  For example, the controller 21 reads the primary audio stream for picture-in-picture display of the Pip primary audio stream number registered in STN_table () with reference to the value set in the Pip primary audio stream number of the # 5 register , Supplied to the AV decoder unit 403. The read-out primary audio stream for picture-in-picture display is supplied to the 1st audio decoder 75-1 through processing of each buffer, PID filter, switch, and the like, and is reproduced and output together with corresponding video data. If the audio stream having the ID number set in the # 5 register is not registered in the STN_table () of the current PlayItem (), the controller 21 performs an automatic selection routine for a primary audio stream for picture-in-picture display. Select an audio stream to be played.

  If it is determined in step S405 that invalid_flag = 0 is not satisfied (that is, invalid_flag = 1), the controller 21 is set in the # 3 register in step S408 as described with reference to FIG. The primary and secondary audio streams corresponding to the primary audio stream number (primary audio stream number) and the secondary audio stream number (secondary audio stream number) set in the # 4 register are obtained from STN_table () of the current PlayItem. The primary and secondary audio streams to be reproduced are identified, read out from the Clip, supplied to the AV decoder unit 403, combined and reproduced in the same manner as described above.

  For example, the controller 21 refers to the values set in the primary audio stream number in the # 3 register and the secondary audio stream number in the # 4 register, and determines the primary and secondary audio streams registered in the STN_table (). Read and supply to the AV decoder unit 403. The read primary and secondary audio streams are supplied to the 1st audio decoder 75-1 and the secondary audio stream are supplied to the 2nd audio decoder 75-2 by processes such as buffers, PID filters, and switches. Are decoded and reproduced and output together with the corresponding video data. If the audio stream having the ID number set in the # 3 register or # 4 is not registered in the STN_table () of the current PlayItem (), the controller 21 performs playback by an automatic selection routine for each audio stream. Select the audio stream to be played.

  After the process of step S407 or step S408 is completed, in step S409, the controller 21 determines whether or not the picture-in-picture (PiP) display is terminated.

  As described above, when picture-in-picture display is set in synchronization with the main path, the start of picture-in-picture display (PiP ON) and the end of picture-in-picture display (PiP OFF) It is executed at a predetermined timing regardless of the input. On the other hand, when picture-in-picture display is set asynchronously with the main path, the start of picture-in-picture display (PiP ON) and the end of picture-in-picture display (PiP OFF) are input by user operation. Is set by

  If it is determined in step S409 that picture-in-picture (PiP) display is not terminated, the process returns to step S404, and the subsequent processes are repeated. If it is determined in step S409 that picture-in-picture (PiP) display is to be terminated, the processing is terminated.

  Through such processing, when picture-in-picture display is executed, it is determined which audio stream is to be played based on the values set in the # 3 register to the # 5 register. If the settings of these registers do not match the registered contents of STN_table (), the stream to be played is determined by the above-described automatic selection routine.

  Next, still another example of STN_table that defines a combination of streams that can be simultaneously reproduced in combination with the primary video stream, that is, a tenth example of STN_table will be described with reference to FIGS. 83 to 85.

  That is, FIGS. 83 to 85 show the case of the ninth example in defining a combination of a secondary video stream, a primary audio stream, a secondary audio stream, and a subtitle stream that can be simultaneously reproduced in combination with the primary video stream. Similarly, it defines primary audio for picture-in-picture applications, a combination of secondary audio streams and primary audio streams, a combination of secondary video streams and secondary audio streams, and presentation graphics for secondary video streams and picture-in-picture applications. PG or text subtitle combination, secondary video and pict In addition to the combination with the primary audio for the in-picture application, the tenth example of the syntax of STN_table () when the combination of the secondary audio and the primary audio for the picture-in-picture application is further defined (the tenth STN_table ( It is a figure which shows the example of)).

  The playback process executed with reference to the tenth STN_table () can also be executed by the playback device 401 described with reference to FIG. In the figure, the same parts as those shown in FIGS. 76 to 79 are repeated, and the description thereof is omitted.

  The newly added comb_info_Secondary_audio_PiP_primary_audio () block indicates a combination of a secondary audio stream and a primary audio stream for picture-in-picture display.

  In the comb_info_Secondary_audio_PiP_primary_audio () block, the 8-bit field of number_of_PiP_primary_audio_ref_entries_SA indicates the number of primary audio streams for picture-in-picture display that can be combined with the secondary audio stream. The 8-bit field of PiP_primary_audio_id2_ref_SA indicates the ID of the primary audio stream for picture-in-picture display that can be combined with the secondary audio stream.

  As in the case of each STN_table described above, each number is used instead of each ID, for example, an audio stream number (audio_stream_number) is used instead of audio_stream_id, or an audio stream number 2 (audio_stream_number2) is used instead of audio_stream_id2 May be used. The same applies to the video stream and the subtitle stream.

  83 to 85, according to the tenth STN_table (), a combination of secondary audio and primary audio, a combination of secondary video and secondary audio, a combination of secondary video and PG / text subtitle for picture-in-picture application, and In addition to a combination of secondary video and primary audio for picture-in-picture display, a combination of secondary audio and primary audio for picture-in-picture display can be defined.

  Accordingly, a secondary video stream (# 2 video stream), a primary audio stream (# 1 audio stream), a secondary audio stream (# 2 audio stream), a subtitle stream (which can be played back simultaneously with the primary video stream (# 1 video stream) ( PG TextST stream), PG / text subtitle stream (PiP PG TextST stream) for picture-in-picture applications, and secondary video and primary audio for picture-in-picture display, plus secondary audio and picture-in-picture It is possible to prevent a conflicting combination from being selected as the primary audio combination for display.

  Further, in the playback device 141, as described above, a plurality of registers are provided in a part of the rewritable memory area of the memory 23 of the playback device 141, and based on the values set in these registers, Playback processing is executed. When the tenth STN_table () shown in FIGS. 83 to 85 is used, a part of the rewritable memory area of the memory 23 of the playback device 141 includes a part of the rewritable memory area in addition to the register described with reference to FIG. A register for setting the audio to be reproduced when the picture-in-picture application described above is started is provided.

  By referring to the values set in these registers, the playback device 141 executes playback processing of each stream.

  In addition, when primary audio and secondary audio are played back during execution of picture-in-picture display, processing similar to that described with reference to FIG. 79 is executed.

  Next, an example in which a secondary audio stream and a primary audio stream for a picture-in-picture application are played back when executing picture-in-picture display will be described with reference to FIG.

  For example, in a state where picture-in-picture display is not executed (in the state indicated as “No PiP” in the figure), the value set in the Primary stream number of the # 3 register is referred to, and STN_table () is referred to. The registered audio stream number = 0x05 is played.

  Then, a command (for example, a navigation command embedded in the stream) is generated by the navigation program, or execution of picture-in-picture display is started by a predetermined operation input by the user (in the figure, PiP on) and the invalid_flag of the # 5 register is set to 0, the value set to the secondary stream number of the # 4 register and the value set to the PiP primary audio stream number of the # 5 register , The audio stream 2 number = 0x01 registered in STN_table () and the primary audio stream for PiP primary audio stream number = 0x07 for the picture-in-picture application are mixed and reproduced.

  Then, when the execution of picture-in-picture display is finished (PiP off in the figure) and the picture-in-picture display is not executed again (state shown as “No PiP” in the figure) , The value set in the Primary stream number of the # 3 register is referred to, and the audio stream number = 0x05 registered in STN_table () is reproduced.

  Also in FIG. 86, when 0xFF is registered in the PiP primary audio stream number, the primary audio stream for picture-in-picture display is not selected or does not exist. For example, even when a value other than 0xFF is registered in the PiP primary audio stream number, whether playback is valid or invalid is determined by the invalid_flag setting. That is, when invalid_flag = 1 and playback is disabled, even if a value other than 0xFF is registered in PiP primary audio stream number, the value registered in PiP primary audio stream number is ignored and is not played back. Yes.

  Next, an audio stream reproduction process 2 of an application including PiP (picture-in-picture) will be described with reference to the flowchart in FIG.

  In steps S451 to S455, basically the same processing as in steps S401 to S405 described with reference to FIG. 82 is executed.

  That is, the controller 21 checks the corresponding primary audio from the STN_table () of the current PlayItem from the primary audio stream number set in the # 3 register, identifies the desired primary audio stream, reads it from the clip, and plays it And start picture-in-picture display at a predetermined timing. When picture-in-picture display is set in synchronization with the main path, the start of picture-in-picture display (PiP ON) and the end of picture-in-picture display (PiP OFF) It is executed at a predetermined timing. On the other hand, when picture-in-picture display is set asynchronously with the main path, the start of picture-in-picture display (PiP ON) and the end of picture-in-picture display (PiP OFF) are input by user operation. Is set by

  The controller 21 changes the setting value of invalid_flag in the # 5 register as necessary by a navigation command (navigation command) embedded in the stream or stored at a different position from the stream and preloaded. To do. Then, the controller 21 determines whether or not invalid_flag = 0 (PiP primary audio stream is valid) in the # 5 register.

  If it is determined in step S455 that invalid_flag = 0, in step S456, the controller 21 determines the corresponding primary audio for picture-in-picture display from the PiP primary audio stream number set in the # 5 register. Check STN_table () of the current PlayItem.

  In step S457, the controller 21 checks the corresponding secondary audio from the STN_table () of the current PlayItem from the secondary audio stream number set in the # 4 register.

  In step S458, the controller 21 specifies the primary audio and secondary audio for picture-in-picture display to be reproduced, reads out from the clip and supplies them to the AV decoder unit 403 in the same manner as described above, and combines (mixing) them. Thus, the process proceeds to step S460 described later.

  For example, the controller 21 refers to the values set in the Pip primary audio stream number in the # 5 register and the secondary audio stream number in the # 4 register, and sets the Pip primary audio stream number registered in the STN_table (). A primary audio stream and a secondary audio stream for picture-in-picture display are read and supplied to the AV decoder unit 403. The read primary audio stream for picture-in-picture display and the secondary audio stream are supplied to the 1st audio decoder 75-1 by the processing of each buffer, PID filter, switch, and the like. The secondary audio stream is supplied to the 2nd audio decoder 75-2, decoded, and reproduced and output together with the corresponding video data. When the audio stream having the ID number set in the # 5 register or the # 4 register is not registered in the STN_table () of the current PlayItem (), the controller 21 performs an automatic selection routine for each audio stream. Select the audio stream to be played.

  If it is determined in step S455 that invalid_flag = 0 is not satisfied (that is, invalid_flag = 1), in step S459, the controller 21 is set in the # 3 register as described with reference to FIG. The primary and secondary audio streams corresponding to the primary audio stream number (primary audio stream number) and the secondary audio stream number (secondary audio stream number) set in the # 4 register are obtained from STN_table () of the current PlayItem. The primary and secondary audio streams to be reproduced are identified, read out from the Clip, supplied to the AV decoder unit 403, combined and reproduced in the same manner as described above.

  For example, the controller 21 refers to the values set in the primary audio stream number in the # 3 register and the secondary audio stream number in the # 4 register, and determines the primary and secondary audio streams registered in the STN_table (). Read and supply to the AV decoder unit 403. The read primary and secondary audio streams are supplied to the 1st audio decoder 75-1 and the secondary audio stream are supplied to the 2nd audio decoder 75-2 by processes such as buffers, PID filters, and switches. Are decoded and reproduced and output together with the corresponding video data. If the audio stream having the ID number set in the # 3 register or # 4 is not registered in the STN_table () of the current PlayItem (), the controller 21 performs playback by an automatic selection routine for each audio stream. Select the audio stream to be played.

  After the process of step S457 or step S459 is completed, in step S460, the controller 21 determines whether or not the picture-in-picture (PiP) display is terminated.

  As described above, when picture-in-picture display is set in synchronization with the main path, the start of picture-in-picture display (PiP ON) and the end of picture-in-picture display (PiP OFF) It is executed at a predetermined timing regardless of the input. On the other hand, when picture-in-picture display is set asynchronously with the main path, the start of picture-in-picture display (PiP ON) and the end of picture-in-picture display (PiP OFF) are input by user operation. Is set by

  If it is determined in step S460 that picture-in-picture (PiP) display is not terminated, the process returns to step S454, and the subsequent processes are repeated. If it is determined in step S460 that the picture-in-picture (PiP) display is to be terminated, the processing is terminated.

  Through such processing, when picture-in-picture display is executed, it is determined which audio stream is to be played based on the values set in the # 3 register to the # 5 register. If the settings of these registers do not match the registered contents of STN_table (), the stream to be played is determined by the above-described automatic selection routine.

  Next, with reference to FIG. 88 and FIG. 89, regarding the manufacturing method of the recording medium 11 on which data that can be reproduced by the reproducing apparatus 1, the reproducing apparatus 141, or the reproducing apparatus 401 is recorded, the recording medium 11 has a disc shape. A case where it is a recording medium will be described as an example.

That is, as shown in FIG. 88, a master disc made of, for example, glass is prepared, and a recording material made of, for example, a photoresist is applied thereon. As a result, a recording master is produced.

As shown in FIG. 89, in the software production processing unit, video data in a format that can be played back by the playback device 1, the playback device 141, or the playback device 401 encoded by the encoding device (video encoder). The audio data stored in the temporary buffer and encoded by the audio encoder is stored in the temporary buffer and further encoded by the data encoder other than the stream (for example, Indexes, Playlist, PlayItem, etc.) Is stored in a temporary buffer. Video data, audio data, and data other than the stream stored in each buffer are multiplexed together with a synchronization signal by a multiplexer (MPX), and an error correction code is added by an error correction code circuit (ECC). Is done.
Then, a predetermined modulation is applied by a modulation circuit (MOD), and the recording medium 11 that is once recorded on, for example, a magnetic tape or the like according to a predetermined format and can be played back by the playback device 1, the playback device 141, or the playback device 401. Software to be recorded on is produced.

  This software is edited (premastered) as necessary, and a signal in a format to be recorded on the optical disc is generated. In response to this recording signal, the laser beam is modulated, and this laser beam is irradiated onto the photoresist on the master. Thereby, the photoresist on the master is exposed in accordance with the recording signal.

  Then, this master is developed and pits appear on the master. The master plate thus prepared is subjected to a process such as electroforming to produce a metal master plate in which pits on the glass master plate are transferred. A metal stamper is further produced from this metal master, and this is used as a molding die.

  A material such as PMMA (acrylic) or PC (polycarbonate) is injected into this molding die by, for example, injection and fixed. Alternatively, 2P (ultraviolet curable resin) or the like is applied on the metal stamper and then cured by irradiation with ultraviolet rays. Thereby, the pits on the metal stamper can be transferred onto the replica made of resin.

  A reflective film is formed on the replica thus generated by vapor deposition or sputtering. Alternatively, a reflective film is formed on the generated replica by spin coating.

  Thereafter, the inner and outer diameters of the disk are processed, and necessary measures such as bonding two disks are performed. Further, a label is attached or a hub is attached and inserted into the cartridge. In this way, the recording medium 11 on which data that can be reproduced by the reproducing apparatus 1, the reproducing apparatus 141, or the reproducing apparatus 401 is recorded is completed.

  The series of processes described above can be executed by hardware or can be executed by software. In this case, the processing described above is executed by a personal computer 500 as shown in FIG.

  90, a CPU (Central Processing Unit) 501 performs various processes according to a program stored in a ROM (Read Only Memory) 502 or a program loaded from a storage unit 508 to a RAM (Random Access Memory) 503. Execute. The RAM 503 also appropriately stores data necessary for the CPU 501 to execute various processes.

  The CPU 501, ROM 502, and RAM 503 are connected to each other via an internal bus 504. An input / output interface 505 is also connected to the internal bus 504.

  The input / output interface 505 includes an input unit 506 including a keyboard and a mouse, a display including CRT and LCD, an output unit 507 including a speaker, a storage unit 508 including a hard disk, a modem, a terminal adapter, and the like. A communicator 509 is connected. A communication unit 509 performs communication processing via various networks including a telephone line and CATV.

  A drive 510 is connected to the input / output interface 505 as necessary, and a removable medium 521 made up of a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is appropriately mounted, and a computer program read therefrom is It is installed in the storage unit 508 as necessary.

  When a series of processing is executed by software, a program constituting the software is installed from a network or a recording medium.

  As shown in FIG. 90, this recording medium is not only composed of a package medium consisting of a removable medium 521 on which a program is recorded, which is distributed to provide a program to the user, separately from the computer. These are configured by a hard disk including a ROM 502 storing a program and a storage unit 508 provided to the user in a state of being pre-installed in the apparatus main body.

  In the present specification, the step of describing a computer program includes not only processing performed in time series according to the described order but also processing executed in parallel or individually even if not necessarily processed in time series. Is also included.

  The embodiment of the present invention is not limited to the above-described embodiment, and various modifications can be made without departing from the gist of the present invention.

  DESCRIPTION OF SYMBOLS 1 Playback apparatus, 11 Recording medium, 21 Controller, 22 Optical disk drive, 23 Memory, 24 Local storage, 26 AV decoder part, 32 switch, 51 thru | or 54 Buffer, 55, 56 PID filter, 57 thru | or 59 switch, 71 Background decoder , 72 video decoder / 1st video decoder, 73 presentation graphics decoder, 74 interactive graphics decoder, 75 audio decoder, 76 Text-ST composition, 77 switch, 91 background plane generator, 92 video plane generator / 1st video Plane generator, 93 Presentation graphics plane generator, 94 Interactive graphics plane generator 95 Buffer 96 Video data processing unit, 97 mixing processing unit, 101 mixing processing unit, 141 playback device, 151 AV decoder unit, 161 PID filter, 162 switch, 401 playback device, 403 AV decoder unit, 411 PID filter, 412 2nd video Decoder, 413 2nd video plane generator, 414 Preload buffer

Claims (4)

Information of PlayItems constituting the main playback path, including information indicating a playback section on the time axis of the first stream including the first video stream and the first audio stream, referred to by the main playback path. SubPlayItem information constituting the sub playback path including information indicating a playback section on the time axis of the second stream including the second video stream and the second audio stream, which is referred to by the sub playback path. Table information included in the PlayItem information and used to select a stream to be played back, and combination information indicating a combination of the second video stream and the second audio stream that can be played back. An acquisition unit for acquiring the reproduction management information including the table information from an optical disc mounted thereon ,
A reading unit for reading out the first stream, the second stream, and sound data as sound effect data from the optical disc ;
A first video decoder for decoding the first video stream included in the first stream;
A second video decoder for decoding the second video stream included in the second stream;
A first audio decoder for decoding the first audio stream included in the first stream;
A second audio decoder for decoding the second audio stream included in the second stream and indicated by the combination information to be playable in combination with the second video stream;
A first synthesis unit for synthesizing audio data obtained by decoding by the first audio decoder and audio data obtained by decoding by the second audio decoder ;
A playback device comprising: audio data after synthesis by the first synthesis unit; and a second synthesis unit that synthesizes the sound data . The playback apparatus according to claim 1, wherein the table information further includes identification information of the second audio stream that can be played back in combination with the second video stream. Information of PlayItems constituting the main playback path, including information indicating a playback section on the time axis of the first stream including the first video stream and the first audio stream, referred to by the main playback path. SubPlayItem information constituting the sub playback path including information indicating a playback section on the time axis of the second stream including the second video stream and the second audio stream, which is referred to by the sub playback path. Table information included in the PlayItem information and used to select a stream to be played back, and combination information indicating a combination of the second video stream and the second audio stream that can be played back. Including the table information including the reproduction management information including, from the mounted optical disk ,
Reading out the first stream, the second stream, and sound data as sound effect data from the optical disc ;
Decoding the first video stream included in the first stream by a first video decoder;
Decoding the second video stream included in the second stream by a second video decoder;
Decoding the first audio stream included in the first stream by a first audio decoder;
A second audio decoder that decodes the second audio stream included in the second stream and indicated by the combination information to be playable in combination with the second video stream;
Combining the audio data obtained by the decoding by the first audio decoder and the audio data obtained by the decoding by the second audio decoder ;
A reproduction method comprising a step of synthesizing the synthesized audio data and the sound data . On an optical disc that can be played back on a playback device,
A first stream referenced by a main playback path and including a first video stream and a first audio stream;
A second stream referenced by a secondary playback path and including a second video stream and a second audio stream;
Information that manages stream playback.
Including information indicating a playback section on the time axis of the first stream, information of PlayItems constituting the main playback path, and
SubPlayItem information constituting the sub playback path including information indicating a playback section on the time axis of the second stream;
Table information that is included in the PlayItem information and is used to select a stream to be played back, and includes combination information indicating a combination of the second video stream and the second audio stream that can be played back and reproduction management information including the table information,
Decoding the second audio stream indicated by the combination information that the audio data obtained by decoding the first audio stream and the second video stream can be reproduced in combination. And recording sound data, which is sound effect data, synthesized with the audio data synthesized with the audio data obtained as described above.

Images