JP5218872B2 - REPRODUCTION DEVICE, RECORDING MEDIUM, AND MANUFACTURING METHOD THEREOF - Google Patents

Description

The present invention relates to a playback device , a recording medium, and a method for manufacturing the same, and more particularly, to a playback device , a recording medium, and a method for manufacturing the same that are suitable for playback of streams.

  In the DVD (Digital Versatile Disc) video standard, when AV (Audio Visual) content such as a movie recorded on a recording medium is played, an interactive operation such as audio switching or subtitle switching is provided to the user. (For example, refer nonpatent literature 1). Specifically, for the AV content displayed on the display device 1 in FIG. 1, the user operates the audio switching button 11 or the subtitle switching button 12 of the remote controller 2 to switch the audio or subtitle. Switch. For example, when the voice 1 is set as the initial state and the user operates the voice switching button 11 of the remote controller 2, the voice 1 is switched to the voice 2 as shown in FIG.

  AV content on a DVD video is recorded in MPEG (Moving Picture Experts Group) 2 program stream format. As shown in FIG. 3, the MPEG2 program stream includes a video stream (video in FIG. 3), a plurality of audio streams (audios 1, 2, and 3 in FIG. 3), and a plurality of sub-picture streams (in FIG. 3). The sub-pictures 1, 2, 3) are multiplexed so as to be reproduced in AV synchronization with the video stream. The sub-picture stream (sub-picture 1, 2, 3) is a stream in which a bitmap image is run-length encoded, and is mainly used for subtitles.

  In general, a plurality of audio streams are used for recording audio in different languages, and a plurality of sub-picture streams are used for recording subtitles in different languages. When the video is being played back, the user can interactively select audio or subtitles in a desired language using the remote controller 2.

  Also, the DVD video includes audio numbers and subtitles provided to the user for a plurality of audio streams (audio 1, 2, 3) and a plurality of sub-picture streams (sub-pictures 1, 2, 3) in the program stream. Defines a table structure that represents the number relationship.

  FIG. 4 is a diagram for explaining a stream number table representing the relationship between audio signals and caption signals provided to the user. Here, the audio number is referred to as A_SN (Audio Stream Number), and the subtitle number is referred to as S_SN (SubPicture Stream Number). In FIG. 4, A_SN is given to each of the plurality of audio streams of the MPEG2 program stream, and S_SN is given to each of the plurality of sub-picture streams of the MPEG2 program stream. A_SN = 1: Audio 2; A_SN = 2: Audio 1; A_SN = 3: Audio 3 Also, S_SN = 1: sub-picture 3, S_SN = 2: sub-picture 1, and S_SN = 3: sub-picture 2. Here, the smaller the A_SN or S_SN number, the higher the priority as the audio signal provided to the user. That is, A_SN = 1 is an audio stream reproduced by default, and S_SN = 1 is a sub-picture stream reproduced by default.

Specifically, the sound 1 reproduced in the initial state of FIG. 1 corresponds to the audio 2 (FIG. 4) with A_SN = 1, and the sound 2 reproduced in FIG. 2 after the sound is switched. Corresponds to audio 1 (FIG. 4) with A_SN = 2.
DVD Specifications for Read-Only Disc Part 3; Version1.1

  However, in the case of DVD video, when a program stream video is being played back, the user's operations such as audio switching and subtitle switching are performed from among the audio stream and sub-picture stream multiplexed in the program stream being played back. However, I could not choose. In other words, when the MPEG2 program stream as shown in FIG. 3 is being reproduced, when switching the audio, the choice is one of the audio 1 to 3.

  For this reason, when an audio stream and subtitles are prepared in a different stream from the program stream being played back, the user cannot select an audio switching operation from the other stream, so there is no scalability. There was a problem.

  Furthermore, in the future, it is assumed that various types of content such as video as well as audio will be targeted as the content to be switched. Therefore, in the future, since these various types of content switching operations cannot be selected from other streams, the problem will not be scalable.

  The present invention has been made in view of such a situation, and selects audio, video, and the like included in another stream or data file different from the main AV stream in order to reproduce the main AV stream. The content data structure that can be used is not complicated, and such content can be reproduced.

The playback apparatus according to the first aspect of the present invention includes a main playback path indicating a position on the time axis of a main stream group, a first sub playback path indicating a position on the time axis of a first substream group, And a second sub-reproduction path indicating a position on the time axis of the second sub-stream group different from the first sub-stream group, and is predefined according to the type of the main stream group Reproduction management information classified into a predetermined type of a plurality of types, wherein each of the first sub reproduction path and the second sub reproduction path is permitted for the predetermined type. Obtaining means for obtaining the reproduction management information satisfying at least a first condition that the reproduction path is of a type selected, and selecting a stream to be reproduced based on the reproduction management information obtained by the obtaining means , Receiving means for receiving a first combination of a stream group and the first substream group, or a second combination of the main stream group and the second substream group; and When one combination is received, the main stream group is read with reference to the main reproduction path, and the first substream group is read with reference to the first sub reproduction path, and the reception is performed. When the second combination is received by the means, the main stream group is read with reference to the main reproduction path, and the second substream group is referred to the second sub reproduction path. The first sub stream is read together with the read means for reading and the main stream group read by the read means. And reproducing means for reproducing a person read by the reading means of the over arm group and the second sub-stream group, the type of the reproduction management information, there is a synchronous and asynchronous The first condition is that when the type of the reproduction management information is the asynchronous type, the type of the secondary reproduction path is a non-multiplex type path, and is an audio presentation path of a browsable slide show. When the type of the reproduction management information is the synchronous type when the type is one type or the non-multiplex type path and the second type is an interactive graphics menu, the sub reproduction path The type is a condition that the type is a type other than the first type, and the main stream group, the first substream group, and the second substream A group of streams is included in any one of the files, and the number of files that can be read at one time by the reading unit is determined in advance, and the reproduction management information is added to the first condition. Further, the second condition that one or more sub reproduction paths are determined so that the total number of the files read at one time is equal to or less than the read number is satisfied.

In the playback apparatus according to the first aspect of the present invention, the main playback path indicating the position of the main stream group on the time axis, and the first sub playback path indicating the position of the first substream group on the time axis And a second sub-reproduction path indicating a position on the time axis of a second sub-stream group different from the first sub-stream group, and a plurality of predefined sub-paths depending on the type of the main stream group Reproduction management information classified into a predetermined type of the first type, wherein each of the first sub reproduction path and the second sub reproduction path is a type of reproduction permitted for the predetermined type. Playback management information satisfying at least the first condition of being a pass is acquired, and based on the acquired playback management information, selection of a stream to be played is performed between the main stream group and the first substream group. First group Or the second combination of the main stream group and the second substream group, and when the first combination is received, the main stream group is read with reference to the main reproduction path, and When the first substream group is read with reference to the first sub-reproduction path and the second combination is accepted, the main stream group is read with reference to the main reproduction path, and The second substream group is read with reference to the second sub-reproduction path, and the reading of the first substream group and the second substream group is performed together with the read main stream group. If you have been issued Ru is played. There are two types of playback management information: synchronous type and asynchronous type. The first condition is that when the type of playback management information is asynchronous type, the secondary playback path type is non-multiplex type path. Either the first type that is the audio presentation path of the browsable slide show or the second type that is the non-multiplex type path and the interactive graphics menu, and the type of the reproduction management information is the synchronous type The sub playback path is of a type other than the first type, and the main stream group, the first sub stream group, and the second sub stream group are any files. The number of files that can be read at one time is determined in advance, and the reproduction management information is further added to the first condition. Meet as the total number of files to be read at one time is equal to or less than the read number, to determine one or more sub playback path, the second condition that.

Recording medium of the second aspect of the present invention is a recording medium on which data is recorded including the reproduction management information for managing the reproduction of the file including at least one stream, the reproduction management information, the main stream group A main reproduction path indicating a position on the time axis, a first sub reproduction path indicating a position on the time axis of the first substream group, and a second sub that is different from the first substream group A second sub-reproduction path indicating a position of the stream group on the time axis, and is classified into a predetermined type of a plurality of types defined in advance according to the type of the main stream group; Each of the secondary reproduction path and the second secondary reproduction path satisfy at least a condition that the reproduction path is a type of reproduction path permitted for the predetermined type, and the type of the reproduction management information includes , There is a period type and an asynchronous type, and the condition is that when the type of the reproduction management information is the asynchronous type, the type of the secondary reproduction path is a non-multiplex type path, and the audio of the browsable slide show When the presentation type is either the first type or non-multiplex type path and the second type is an interactive graphics menu, and the type of the reproduction management information is the synchronous type, The sub playback path is of a type other than the first type, and the main stream group, the first substream group, and the second substream group are any files. The total number of files to be read at one time can be read at a time determined in advance by the playback device. As will be less read number, data for determining the first sub playback path and the second sub playback path is recorded.

Oite to a second aspect of the recording medium of the present invention, the reproduction management information, indicating the main playback path indicating a position on a time axis of the main stream group, the position on the time axis of the first sub-stream group Including a first sub reproduction path and a second sub reproduction path indicating a position on the time axis of a second substream group different from the first substream group, and the type of the main stream group includes Accordingly, the reproduction is classified into a predetermined type among a plurality of types defined in advance, and each of the first sub reproduction path and the second sub reproduction path is a reproduction type permitted for the predetermined type. The condition of being a path is satisfied, and the type of the reproduction management information includes a synchronous type and an asynchronous type, and the condition is determined when the type of the reproduction management information is the asynchronous type. Playback path type is non-multiplexed A first type that is an audio presentation path of a browsable slideshow, or a second type that is a non-multiplexed type path and is an interactive graphics menu. When the type is the synchronous type, it is a condition that the type of the secondary reproduction path is a type other than the first type, and the main stream group, the first substream group, and the second type The substream group is included in any file, and the total number of the files to be read at one time is equal to or less than the number of readings that can be read at a time that is predetermined in the playback device. A first sub reproduction path and the second sub reproduction path are determined.

In the method for manufacturing a storage medium according to the second aspect of the present invention, the reproduction management information for managing reproduction includes a main reproduction path indicating a position on the time axis of the main stream group and a time axis of the first substream group. And a second sub reproduction path indicating a position on the time axis of a second substream group different from the first substream group, and Each of the first sub reproduction path and the second sub reproduction path is classified into a predetermined type out of a plurality of types defined in advance according to the type of the stream group. At least a condition that the playback path is a permitted type, and the type of the playback management information includes a synchronous type and an asynchronous type, and the condition is that the type of the playback management information is the above-mentioned If it is asynchronous type, The playback path type is a non-multiplex type path, the first type which is an audio presentation path of a browsable slide show, or a non-multiplex type path, the second type which is an interactive graphics menu. If the type of the reproduction management information is the synchronous type, the sub reproduction path type is a condition other than the first type, and the main stream group, the first One substream group and the second substream group are included in any one of the files, and the total number of the files to be read at a time can be read at a time determined in advance by the playback device. The first sub reproduction path and the second sub reproduction path are determined so as to be less than or equal to the number of possible readings. Generating data having the data structure, and records the generated data in a recording medium.

In the recording medium manufacturing method according to the second aspect of the present invention, the main reproduction path indicating the position of the main stream group on the time axis, and the first sub stream indicating the position of the first substream group on the time axis. And a second sub reproduction path indicating a position on the time axis of a second substream group different from the first substream group, and is defined in advance according to the type of the main stream group The first sub-reproduction path and the second sub-reproduction path are classified as a predetermined type of the plurality of types, and each of the second sub-reproduction paths is a type of reproduction path permitted for the predetermined type. The type of the reproduction management information includes a synchronous type and an asynchronous type, and when the type of the reproduction management information is the asynchronous type, the sub reproduction path type is A non-multiplexed path , Either a first type that is an audio presentation path of a browsable slide show, or a second type that is a non-multiplex type path and an interactive graphics menu, and the type of the reproduction management information is the synchronization The sub-reproduction path type is a condition other than the first type, the main stream group, the first sub-stream group, and the second sub-stream group. Is included in any one of the files, and the total number of the files to be read at one time is equal to or less than the number of readings that can be read at a time that is determined in advance by the playback device. And data having a data structure including reproduction management information for determining the reproduction path of the second sub and the second sub reproduction path Is generated, the generated data is recorded on the recording medium.

  According to the present invention, it is possible to prepare audio, video, and the like in another stream or data file different from the main AV stream. In particular, in that case, the data structure of the content can be prevented from becoming complicated, and such content can be reproduced.

  Further, according to another aspect of the present invention, when various types such as audio and video are prepared in other streams and data files different from the main AV stream, the data structure of the content is prevented from becoming complicated. it can.

It is a figure explaining the conventional voice switching. It is a figure explaining the conventional voice switching. It is a figure explaining the structure of an MPEG2 program stream. It is a figure explaining the stream number table showing the relationship between the audio | voice signal provided to a user, and a caption signal. It is a figure which shows the example of the application format on the recording medium with which the reproducing | regenerating apparatus to which this invention is applied is mounted | worn. It is a figure explaining the structure of a main path and a sub path. It is a figure explaining the example of a main path and a sub path. It is a figure explaining another example of a main path and a sub path. It is a figure which shows the syntax of PlayList (). It is a figure which shows the syntax of SubPath (). It is a figure explaining SubPath_type. It is a figure which shows the syntax of SubPlayItem (i). It is a figure which shows the syntax of PlayItem (). It is a figure which shows the syntax of STN_table (). It is a figure which shows the syntax of stream_entry (). It is a figure which shows the syntax of stream_attribute (). It is a figure explaining stream_cording_type. It is a figure explaining video_format. It is a figure explaining frame_rate. It is a figure explaining aspect_ratio. It is a figure explaining audio_presentation_type. It is a figure explaining sampling_frequency. It is a figure explaining Character code. It is a figure explaining the example of the stream number table showing the relationship between the audio | voice signal provided to a user, and a caption signal. It is a block diagram which shows the structural example of the reproducing | regenerating apparatus to which this invention is applied. FIG. 26 is a flowchart for describing playback processing in the playback apparatus of FIG. 25. FIG. FIG. 26 is a flowchart for describing playback processing in the playback apparatus of FIG. 25. FIG. FIG. 26 is a flowchart for describing playback processing in the playback apparatus of FIG. 25. FIG. It is a flowchart explaining a process in case the switching of the audio | voice with respect to audio stream # 1 is instruct | indicated by the user. It is a flowchart explaining a process in case the switching of the audio | voice with respect to audio stream # 2 is instruct | indicated by the user. It is a flowchart explaining the example of the detailed process of FIG.29 S60. It is a figure explaining the example which mixes and reproduces two audios. It is a figure which shows the example of the syntax of STN_table () which defines the combination of audio stream # 1 and audio stream # 2. It is a figure which shows the example of the bitmap syntax of Combination_of_Primary_and_Secondary. It is a figure explaining the other example of the stream number table showing the relationship between the audio | voice signal provided to a user, and a caption signal. It is a figure explaining the other example which mixes and reproduces two audios. It is a figure which shows the other example of the syntax of STN_table () which defines the combination of audio stream # 1 and audio stream # 2. It is a figure which shows the further another example of the syntax of STN_table () which defines the combination of audio stream # 1 and audio stream # 2. It is a figure explaining a picture in picture technique. It is a figure which shows the example of the syntax of STN_table () in the case of defining the combination of a secondary video stream, a primary audio stream, a secondary audio stream, and a subtitle stream reproduced | regenerated simultaneously in combination with a primary video stream. It is a figure which shows the example of the syntax of STN_table () in the case of defining the combination of a secondary video stream, a primary audio stream, a secondary audio stream, and a subtitle stream reproduced | regenerated simultaneously in combination with a primary video stream. It is a figure which shows the example of the stream number table showing the relationship which can combine the audio | voice signal provided to a user, the video signal as a secondary video stream, and a caption signal. It is a figure explaining the selection operation of the user for selecting the combination of a secondary video stream, a primary audio stream, a secondary audio stream, and a subtitle stream reproduced | regenerated simultaneously in combination with a primary video stream. In the case of realizing the picture-in-picture method, an example of a PlayList generated using the SubPath_type in the example of FIG. 11 is shown. It is a figure explaining the example different from FIG. 11 of SubPath_type. It is a figure which shows the 2nd example of the syntax of stream_entry (). An example of a PlayList generated using the SubPath_type in the example of FIG. 45 when the picture-in-picture method is realized is shown. FIG. 45 shows another example of the PlayList generated using the SubPath_type in the example of FIG. 45 when the picture-in-picture method is realized. FIG. 26 is a block diagram showing an example of the configuration of a playback apparatus to which the present invention is applied, which is different from FIG. 25. 50 is a flowchart for describing secondary video switching processing in the playback device of FIG. 49. 50 is a flowchart for describing secondary video switching processing in the playback device of FIG. 49. It is a figure explaining the example of the kind of application_type. It is a figure for demonstrating a sub path | pass restriction | limiting technique, Comprising: It is a figure which shows the relationship of the combination of SubPath which the PlayList can have with respect to the kind of PlayList. It is a figure for demonstrating a sub path | pass restriction | limiting technique, Comprising: It is a figure which shows the relationship of the combination of the number of PlayItem and SubPath in PlayList. FIG. 56 is a diagram illustrating a specific example of a PlayList created by the sub-path restriction method described with reference to FIGS. 53 to 55. FIG. 56 is a diagram illustrating a specific example of a PlayList created by the sub-path restriction method described with reference to FIGS. 53 to 55. FIG. 56 is a diagram illustrating a specific example of a PlayList created by the sub-path restriction method described with reference to FIGS. 53 to 55. FIG. 56 is a diagram illustrating a specific example of a PlayList created by the sub-path restriction method described with reference to FIGS. 53 to 55. 50 is a flowchart describing PlayList playback processing of application_type = 3 in the playback device of FIG. 49. 50 is a flowchart describing PlayList playback processing of application_type = 3 in the playback device of FIG. 49. 50 is a flowchart for describing PlayList playback processing with application_type = 1or2 in the playback device of FIG. 49. 50 is a flowchart for describing PlayList playback processing with application_type = 1or2 in the playback device of FIG. 49. 50 is a flowchart for describing PlayList playback processing with application_type = 1or2 in the playback device of FIG. 49. 50 is a flowchart for describing PlayList playback processing with application_type = 1or2 in the playback device of FIG. 49. It is a figure for demonstrating manufacture of the recording medium which recorded the data which can be reproduce | regenerated with a reproducing | regenerating apparatus. It is a figure for demonstrating manufacture of the recording medium which recorded the data which can be reproduce | regenerated with a reproducing | regenerating apparatus. It is a figure which shows the structure of a personal computer.

  Embodiments of the present invention will be described below. The correspondence relationship between the invention described in this specification and the embodiments of the invention is exemplified as follows. This description is intended to confirm that the embodiments supporting the invention described in this specification are described in this specification. Therefore, even if there is an embodiment that is described in the embodiment of the invention but is not described here as corresponding to the invention, the fact that the embodiment is not It does not mean that it does not correspond to the invention. Conversely, even if an embodiment is described herein as corresponding to an invention, that means that the embodiment does not correspond to an invention other than the invention. Absent.

  Further, this description does not mean all the inventions described in this specification. In other words, this description is for the invention described in the present specification, which is not claimed in this application, that is, for the invention that will be applied for in the future or that will appear and be added by amendment. It does not deny existence.

The playback device according to the first aspect of the present invention (the playback device 401 in FIG. 49)
Main playback path (for example, Main path: main path in FIG. 58) indicating the position on the time axis of a main stream group (for example, Video or Audio of Clip-AV Stream 0 in FIG. 58), the first substream A first sub reproduction path (for example, Sub path-1: sub path-1 in FIG. 58) indicating a position on the time axis of a group (for example, 2nd Video, 2nd Audio, etc.), and the first sub stream group Are second sub-reproduction paths indicating positions on the time axis of different second sub-stream groups (for example, Audio) (for example, Sub path-2: sub-path-2 referring to Clip AV Stream -1 in FIG. 58) And a predetermined type (for example, FIG. 53) of a plurality of types that are defined in advance according to the type of the main stream group, including Sub path-3 (subpath-3) referring to Clip AV Stream-2. Among the types divided by application_type (of Main TS), Movie Type / Time-based Slides Reproduction management information (for example, PlayList in FIG. 58) classified into a type described as “how”, ie, a synchronous type described later), and the first sub reproduction path and the second sub Each of the reproduction paths is a type permitted for the predetermined type (for example, in the case of FIG. 53, SubPath_type = any one of 5, 6, and 7. However, the SubPath_type here is shown in FIG. 45). In order to execute the playback processing of the PlayList with application_type = 1or2 in FIGS. 61 to 64, for example, to acquire the playback management information satisfying at least the first condition that the playback path is according to the example) The controller 34 in FIG. 49 for executing the processing in step S11 in FIG.
Based on the reproduction management information acquired by the acquisition means, the selection of a stream to be reproduced is performed by a first combination of the main stream group and the first substream group, or the main stream group and the first stream. Receiving means (for example, the controller 34 in FIG. 49 that executes the processes in steps S163YES, S164 to S169 in FIG. 64) that is received as the second combination with the two substream groups;
When the first combination is received by the receiving unit, the main stream group is read with reference to the main reproduction path, and the first substream group is referred to the first sub reproduction path. When the second combination is received by the receiving unit, the main stream group is read with reference to the main reproduction path, and the second substream group is read by the second sub stream. Reading means for referring to the reproduction path (for example, in the case of reproduction of the PlayList in the example of FIG. 58, processing such as step S151 in FIG. 62, step S155 in FIG. 63, and steps S161 and S162 in FIG. 64 is executed. Before, the controller 34 in FIG. 49 for executing processing equivalent to step S108 in FIG.
Along with the main stream group read by the reading means, one of the first substream group and the second substream group read by the reading means is reproduced (for example, FIG. 58). In the case of playing the PlayList in the example of FIG. 49, the AV decoder unit 33 in FIG. 49 for executing the processing of step S151 in FIG. 62, step S155 in FIG. 63, steps S161 and S162 in FIG.
The types of reproduction management information include a synchronous type and an asynchronous type,
The first condition is that when the type of the reproduction management information is the asynchronous type, the type of the secondary reproduction path is a non-multiplex type path, and is an audio presentation path of a browsable slide show. Type (SubPath_type = 2) or a non-multiplexed path and a second type (SubPath_type = 3) which is an interactive graphics menu, and the type of the reproduction management information is the synchronous type If there is, the condition is that the type of the secondary playback path is a type other than the first type,
The main stream group, the first substream group, and the second substream group are included in any file (any Clip),
The number of files that can be read at once by the reading means is determined in advance,
In addition to the first condition, a second condition (for example, in FIG. 53) that one or more sub reproduction paths are determined so that the total number of the files read at one time is equal to or less than the read number. The reproduction management information satisfies the conditions shown in the item “number of SubPaths” and the table in FIG.

In the recording medium according to the second aspect of the present invention, the reproduction management information includes a main reproduction path indicating a position on the time axis of the main stream group, and a first position indicating a position on the time axis of the first substream group. A secondary reproduction path and a second secondary reproduction path indicating a position on the time axis of a second substream group different from the first substream group, and depending on the type of the main stream group The first sub-reproduction path and the second sub-reproduction path are classified into predetermined types among a plurality of types defined in advance, and each of the first sub reproduction path and the second sub reproduction path is permitted for the predetermined type ( 53, for example, SubPath_type = any one of 5, 6, and 7. However, SubPath_type here is at least a reproduction path according to the example of FIG. The type of the reproduction management information includes the same If the type of the playback management information is the asynchronous type, the sub playback path type is a non-multiplex type path, and an audio presentation of a browsable slide show Either the first type (SubPath_type = 2) that is a path or the second type (SubPath_type = 3) that is a non-multiplex type path and is an interactive graphics menu, and the type of the reproduction management information Is the condition that the type of the secondary playback path is a type other than the first type, the main stream group, the first substream group, and the second type The substream group is included in one of the files (one of the clips), and the total number of the files read at one time is The first sub reproduction path and the second sub reproduction path are determined so as to be equal to or less than a predetermined number of readouts that can be read at a time.

In the recording medium manufacturing method according to the second aspect of the present invention, the reproduction management information indicates the main reproduction path indicating the position of the main stream group on the time axis, and the position of the first substream group on the time axis. A type of the main stream group including a first sub reproduction path and a second sub reproduction path indicating a position on the time axis of a second substream group different from the first substream group And the first sub reproduction path and the second sub reproduction path are permitted for the predetermined type. 53 (for example, SubPath_type = 5, 6, or 7 in the case of the example in FIG. 53, where SubPath_type follows the example in FIG. 45). and, kind of the reproduction management information There are synchronous type and asynchronous type, and the condition is that when the type of the playback management information is the asynchronous type, the secondary playback path type is a non-multiplex type path, and the browserable Either the first type (SubPath_type = 2), which is an audio presentation path of a slide show, or the second type (SubPath_type = 3), which is a non-multiplex type path and an interactive graphics menu. When the type of management information is the synchronous type, the sub playback path type is a condition other than the first type, and the main stream group, the first sub stream group, and The second substream group is included in one of the files (one of the Clips), and the total number of the files read at one time is the playback device. Data having a data structure for determining the first sub reproduction path and the second sub reproduction path is generated so that the number of read data can be read at a time is less than or equal to a predetermined number. Recording the generated data on a recording medium.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

  FIG. 5 shows an example of an application format on a recording medium mounted on a playback apparatus to which the present invention is applied, for example, the playback apparatus 20 described later with reference to FIG. 25 or the playback apparatus 401 described later with reference to FIG. FIG. The recording medium may be a magnetic disk or a semiconductor memory in addition to the optical disk described later.

  The application format has two layers, PlayList and Clip, for managing AV (Audio Visual) streams. Here, a pair of one AV stream and Clip information as information accompanying the AV stream is considered as one object, and these are collectively referred to as a Clip. Hereinafter, the AV stream is also referred to as an AV stream file. In addition to audio data and video data, the AV stream file may include various stream files that are reproduced together with these data. Clip information is also referred to as a Clip information file.

  In general, a file used in a computer or the like is handled as a byte string, but the content of an AV stream file is expanded on the time axis, and an access point of a clip is mainly designated by a PlayList with a time stamp. That is, PlayList and Clip are layers for AV stream management.

  When the access point in the Clip is indicated by the PlayList with a time stamp, the Clip Information file is used to find address information to start decoding in the AV stream file from the time stamp.

  The PlayList is a collection of AV stream playback sections. One playback section in a certain AV stream is called PlayItem, which is represented by a pair of an IN point (playback start point) and an OUT point (playback end point) of the playback section on the time axis. Therefore, the PlayList is composed of one or a plurality of PlayItems as shown in FIG.

  In FIG. 5, the first PlayList from the left is composed of two PlayItems, and the two PlayItems refer to the first half and the second half of the AV stream included in the left Clip, respectively. Also, the second PlayList from the left is composed of one PlayItem, whereby the entire AV stream included in the right Clip is referenced. Further, the third PlayList from the left is composed of two PlayItems, and the two PlayItems refer to a portion of the AV stream included in the left Clip and a portion of the AV stream included in the right Clip, respectively. ing.

  For example, when the disc navigation program of FIG. 5 specifies the left PlayItem included in the first PlayList from the left as the information indicating the playback position at that time, it is included in the left Clip referred to by the PlayItem. The first half of the AV stream is played back. Thus, the PlayList is used as playback management information for managing playback of AV stream files.

  The disc navigation program has a function of controlling the playback order of the PlayList and interactive playback of the PlayList. The disc navigation program also has a function of displaying a menu screen for the user to instruct execution of various reproductions. This disc navigation program is described in a programming language such as Java (registered trademark), and is prepared on a recording medium.

  In this embodiment, a playback path created by a sequence of one or more PlayItems (by successive PlayItems) in the PlayList is called a main path (Main Path), and in the PlayList, parallel to the Main Path (in parallel) ) And a playback path created by a sequence of one or more sub paths (which may be discontinuous or may be continuous) is referred to as a sub path. That is, the application format on the recording medium mounted on the playback device 20 (described later with reference to FIG. 25) or the playback device 401 (described later with reference to FIG. 49) is associated with the main path (accordingly). ) The PlayList has a sub path to be played (Sub Path).

  FIG. 6 is a diagram for explaining the structure of the main path and the sub path. The PlayList can have one main path and one or more sub paths. One main path is created by a sequence of one or more PlayItems, and one sub-path is created by a sequence of one or more SubPlayItems.

  In the case of the example in FIG. 6, the PlayList has one main path made up of a sequence of three PlayItems and three sub paths. Each PlayItem constituting the main path is assigned ID (Identification) in order from the top. Specifically, the main path is made up of PlayItems with PlayItem_id = 0, PlayItem_id = 1, and PlayItem_id = 2. Subpaths are also assigned IDs in order from the top, Subpath_id = 0, Subpath_id = 1, and Subpath_id = 2. A subpath of Subpath_id = 0 includes one SubPlayItem, a subpath of Subpath_id = 1 includes two SubPlayItems, and a subpath of Subpath_id = 2 includes one SubPlayItem.

  The stream referred to by the SubPlayItem included in the subpath of Subpath_id = 0 is assumed to be, for example, a Japanese dubbing sound of a movie, and may be played in place of the audio stream of the AV stream file referenced by Mainpath. Also, the stream referred to by SubPlayItem included in the subpath of Subpath_id = 1 is assumed to be a director's cut of a movie, for example, and a comment such as a movie director is included only in a predetermined part of the AV stream file referred to by Main Path. The case is considered.

  A Clip AV stream file referred to by one PlayItem includes at least video stream data (main image data). In addition, the Clip AV stream file may or may include one or more audio streams that are played back at the same timing (synchronously) as the video stream (main image data) included in the Clip AV stream file. It does not have to be. Furthermore, the Clip AV stream file may or may not include one or more bitmap subtitle streams that are played back at the same timing as the video stream included in the Clip AV stream file. The Clip AV stream file may or may not include one or more interactive graphics streams that are played back at the same timing as the video stream included in the Clip AV stream file. The video stream included in the Clip AV stream file and the audio stream, bitmap subtitle stream file, or interactive graphics stream reproduced at the same timing as the video stream are multiplexed. That is, a Clip AV stream file referred to by one PlayItem includes video stream data, zero or more audio streams to be played in accordance with the video stream, zero or more bitmap subtitle stream data, and zero or more. The interactive graphics stream data is multiplexed.

  That is, a Clip AV stream file referred to by one PlayItem includes a plurality of types of streams such as a video stream, an audio stream, a bitmap subtitle stream file, or an interactive graphics stream.

  One SubPlayItem refers to audio stream data and subtitle data of a stream (different stream) different from the Clip AV stream file referred to by the PlayItem.

  When playing a PlayList that has only the main path, the user can select audio and subtitles only from the audio stream and sub-picture stream multiplexed in the clip referenced by the main path. Can not do it. On the other hand, when playing a PlayList having a main path and a sub path, in addition to the audio stream and sub picture stream multiplexed in the Clip AV stream file referenced by the main path, the clip audio stream referenced by the SubPlayItem Or a sub-picture stream.

  In this way, since a plurality of SubPaths are included in one PlayList and each SubPath refers to each SubPlayItem, an AV stream with high expandability and high flexibility can be realized. In other words, in addition to the Clip AV stream referred to by MainPath, a SubPlayItem can be added later.

  FIG. 7 is a diagram illustrating an example of a main path and a sub path. In FIG. 7, an audio playback path that is played back at the same timing as the main path (synchronized with AV) is represented using sub-paths.

  The PlayList in FIG. 7 includes one PlayItem with PlayItem_id = 0 as a main path and one SubPlayItem as a sub path. One PlayItem () with PlayItem_id = 0 in the main path refers to the main AV stream in FIG. SubPlayItem () includes the following data. First, SubPlayItem () includes Clip_Information_file_name for designating a Clip referred to by a Sub Path in the PlayList. In the case of the example in FIG. 7, an auxiliary audio stream (audio stream) with SubClip_entry_id = 0 is referred to by SubPlayItem. Also, SubPlayItem () includes SubPlayItem_IN_time and SubPlayItem_OUT_time for specifying the playback section of the Sub Path in the stream (Auxiliary audio stream in this case) included in the specified Clip. Further, SubPlayItem () includes sync_PlayItem_id and sync_start_PTS_of_PlayItem for specifying the time at which the Sub Path starts to be reproduced on the time axis of the Main path. In the example of FIG. 7, sync_PlayItem_id = 0 and sync_start_PTS_of_PlayItem = t1. As a result, it is possible to specify the time t1 at which the Sub Path starts playback on the time axis of PlayItem_id = 0 of the main path. That is, in the case of the example in FIG. 7, the reproduction start time t1 of the main path and the start time t1 of the sub path are the same time.

  Here, the Clip AV stream of the audio referred to by the Sub Path must not include STC discontinuities (system time base discontinuities). The audio sample clock of Clip used for the sub path is locked to the audio sample clock of the main path.

  In other words, in SubPlayItem (), information specifying the Clip referred to by the Sub Path, information specifying the playback section of the Sub Path, and information specifying the time when the Sub Path starts playback on the time axis of the Main path It is included. Since the Clip AV stream used in Sub Path does not include STC, information included in SubPlayItem () (information specifying Clip referred to by Sub Path, information specifying Sub Path playback section, and time axis of Main path) Based on the above (information specifying the time when the Sub Path starts playback), the audio stream of the Clip AV stream different from the Clip AV stream (main AV stream) referred to by the main path can be referred to and reproduced. it can.

  In this way, PlayItem and SubPlayItem manage Clip AV stream files, respectively, and here, the Clip AV stream file (main AV stream) managed by PlayItem and the Clip AV stream file managed by SubPlayItem are different from each other. Become.

  Similar to the example of FIG. 7, the subtitle stream playback path that is played back at the same timing as the main path can also be expressed using the sub path.

  FIG. 8 is a diagram for explaining another example of the main path and the sub path. In FIG. 8, a playback path of audio that is played back at the same timing as the main path (synchronized with AV) is represented using sub paths. Here, the main AV stream file referred to by PlayItem in the main path is the same as in FIG.

  For example, a clip AV stream referred to by the main path is set as one movie content (AV content), an auxiliary audio stream referenced by the sub-path audio path is set as a director's comment for the movie, and a clip AV stream referred to by the main path is used. Such a configuration is used when the auxiliary audio stream referred to in the sub-path audio path is mixed (overlaid) with the audio stream. That is, when the user inputs a command to listen to the director's comment together with the movie while watching the movie, for example, the audio of the Clip AV stream referenced in the main path and the audio path of the sub path This is used when the Auxiliary audio stream referenced in the above is mixed and played back.

  In FIG. 8, three PlayItems are arranged in the main path as PlayItem_id = 0, 1, 2, and two SubPlayItems are arranged in the subpath (Subpath_id = 0). SubPlayItem (described later in FIG. 12) called by SubPath with Subpath_id = 0 (described later in FIG. 10) is an Auxiliary audio stream (English Auxiliary audio stream clip with SubClip_entry_id = 0 and SubClip_entry_id = 1 Japanese) SubPlayItem_IN_time and SubPlayItem_out_time for specifying the subpath playback section of (Auxiliary audio stream clip).

  Comparing FIG. 8 and FIG. 7, in FIG. 8, Auxiliary audio stream (English or Japanese audio stream) of SubClip_entry_id = 0, 1 can be referred to by SubPlayItem. That is, it has a structure that refers to a plurality of audio stream files using SubPlayItem, and when reproducing this SubPlayItem, one audio stream file is selected and reproduced from the plurality of audio storm files. In the case of the example in FIG. 8, one audio stream file is selected and reproduced from the English audio stream file and the Japanese audio stream file. Specifically, one of subClip_entry_id = 0 and 1 is selected (for example, based on a user instruction), and an auxiliary audio stream referred to by the ID is reproduced. In addition to this, when playback by mixing with an audio stream referred to by the main path is selected (when two audio streams are selected as audio streams to be played back), for example, the playback is referred to by the main path. The audio stream file to be referred to and the audio stream file referred to by the sub-path audio path are mixed and reproduced.

  Next, a data structure (syntax) for specifically realizing the main path and sub path structures described with reference to FIGS. 6 to 8 will be described.

  FIG. 9 is a diagram illustrating the syntax of PlayList ().

  length is a 32-bit unsigned integer indicating the number of bytes from immediately after this length field to the end of PlayList (). That is, this field indicates the number of bytes from reserved_for_future_use to the end of the Playlist. After this length, 16-bit reserved_for_future_use is prepared. number_of_PlayItems is a 16-bit field indicating the number of PlayItems in the PlayList. For example, in the example of FIG. 6, the number of PlayItems is three. The value of PlayItem_id is assigned from 0 in the order in which PlayItem () appears in the PlayList. For example, as shown in FIGS. 6 and 8, PlayItem_id = 0, 1, and 2 are allocated.

  number_of_SubPaths is a 16-bit field indicating the number of SubPaths (number of entries) in the PlayList. For example, in the example of FIG. 6, the number of Sub Paths is three. The value of SubPath_id is assigned from 0 in the order in which SubPath () appears in the PlayList. For example, as shown in FIG. 6, Subpath_id = 0, 1, and 2 are allocated. In the subsequent for statement, PlayItems are referenced by the number of PlayItems, and Sub Paths are referenced by the number of Sub Paths.

  FIG. 10 is a diagram illustrating the syntax of SubPath ().

  length is a 32-bit unsigned integer indicating the number of bytes from immediately after this length field to the end of Sub Path (). That is, this field indicates the number of bytes from reserved_for_future_use to the end of the Playlist. After this length, 16-bit reserved_for_future_use is prepared. SubPath_type is an 8-bit field indicating the application type of SubPath. SubPath_type is used, for example, when the type indicates whether the Sub Path is audio, bitmap subtitle, text subtitle, or the like. This SubPath_type will be described later with reference to FIG. 15-bit reserved_for_future_use is prepared after SubPath_type. is_repeat_SubPath is a 1-bit field that designates the SubPath playback method, and indicates whether the SubPath playback is repeated during the main path playback or the SubPath playback is performed only once. For example, it is used when the playback timing of the clip specified by the main AV stream and the sub path is different (for example, when the main path is used as a still image slide show and the sub path audio path is used as the main path BGM). . 8-bit reserved_for_future_use is prepared after Is_repeat_SubPath. number_of_SubPlayItems is an 8-bit field indicating the number of SubPlayItems (number of entries) in one SubPath. For example, number_of_SubPlayItems has one SubPlayItem with SubPath_id = 0 in FIG. 6 and two SubPlayItems with SubPath_id = 1. In subsequent for statements, SubPlayItems are referenced by the number of SubPlayItems.

  By the way, SubPath_type (subpath type) is disclosed, for example, in FIG. 42 of JP-A No. 2002-158965 and JP-A No. 2002-158972. However, in these publications, only one type of SubPath_type is defined, and even if the definition is used as it is, the above-described various processing and various methods described later cannot be realized. Therefore, in the present embodiment, for example, the type of SubPath_type shown in FIG. 11 or FIG. 45 is adopted according to various types of processing or various types of methods.

  Therefore, first, the type of SubPath_type in the example of FIG. 11 will be described. The SubPath_type in the example of FIG. 45 will be described later.

  That is, FIG. 11 is a diagram illustrating an example of the type of SubPath_type. That is, in the example of FIG. 11, the type of SubPath can be defined as follows.

  In FIG. 11, SubPath_type = 0, 1 is reserved. SubPath_type = 2 is an Audio presentation path of the Browsable slideshow. For example, SubPath_type = 2 indicates that in the playlist, the audio presentation path referred to by the sub path and the main path referred to by the play item are asynchronous.

  SubPath_type = 3 is an interactive graphics presentation menu (interactive graphics presentation menu). For example, SubPath_type = 3 indicates that in the playlist, the interactive graphics menu referenced by the sub path and the main path referenced by the play item are asynchronous.

  SubPath_type = 4 is a Text subtitle presentation path (a text subtitle presentation path). For example, SubPath_type = 4 indicates that in the playlist, the text subtitle presentation path referenced by the sub path and the main path referenced by the play item are synchronized.

  SubPath_type = 5 is 2nd Audio Presentation path (second audio presentation path) (path for referring to the 2nd audio stream). Specifically, SubPath_type = 5 indicates that in the playlist, the second audio presentation path referred to by the sub path and the main path referred to by the play item are synchronized. For example, the (second) audio stream referenced in this subpath is a director's comment (voice) on the movie. In the case of the SubPath indicated by Subpath_id = 0 in FIG. 8, the SubPath_type in FIG. 10 is SubPath_type = 5.

  SubPath_type = 6 is 2nd Video Presentation path (second video presentation path) (path for referring to the 2nd video stream). Specifically, SubPath_type = 6 indicates that in the playlist, the second video presentation path referred to by the sub path and the main path referred to by the play item are synchronized. For example, the (second) video stream referred to in this sub-path is a director's comment (moving image) for the movie.

  SubPath_type = 7 to 255 is reserved.

  FIG. 12 is a diagram illustrating the syntax of SubPlayItem (i).

  The length is a 16-bit unsigned integer indicating the number of bytes from immediately after this length field to the end of Sub playItem ().

  In FIG. 12, it is divided into a case where SubPlayItem refers to one Clip and a case where it refers to a plurality of Clips.

  First, a case where SubPlayItem refers to one Clip will be described.

  SubPlayItem includes Clip_Information_file_name [0] for specifying Clip. In addition, Clip_codec_identifier [0] that specifies the codec method of Clip, reserved_for_future_use, is_multi_Clip_entries that is a flag indicating the presence or absence of multi-clip registration, and ref_to_STC_id [0] that is information about STC discontinuity points (system time base discontinuity points) including. When the is_multi_Clip_entries flag is set, the syntax when SubPlayItem refers to a plurality of Clips is referenced. In addition, SubPlayItem_IN_time and SubPlayItem_OUT_time for specifying the playback section of the Sub Path in the Clip are included. Furthermore, sync_PlayItem_id and sync_start_PTS_of_PlayItem are included to specify the time at which the Sub Path starts to be reproduced on the time axis of the main path. As described above, this sync_PlayItem_id and sync_start_PTS_of_PlayItem are used in the case of FIGS. 7 and 8 (when the playback timing of the file indicated by the main AV stream and the sub path is the same), and the file indicated by the main AV stream and the sub path. Are not used (for example, when the still image referred to by the main path and the audio referred to by the sub path are not synchronized as in the BGM of a slide show composed of still images). Also, SubPlayItem_IN_time, SubPlayItem_OUT_time, sync_PlayItem_id, and sync_start_PTS_of_PlayItem are used in common in the Clip referred to by SubPlayItem.

  Next, a case where SubPlayItem refers to a plurality of Clips (if (is_multi_Clip_entries == 1b), that is, multi-clip registration is performed) will be described. Specifically, as shown in FIG. 8, a case where SubPlayItem refers to a plurality of Clips is shown.

  num_of_Clip_entries indicates the number of Clips, and Clip_Information_file_name [SubClip_entry_id] specifies Clips excluding Clip_Information_file_name [0]. That is, Clips such as Clip_Information_file_name [1] and Clip_Information_file_name [2] are specified excluding Clip_Information_file_name [0]. Further, SubPlayItem includes Clip_codec_identifier [SubClip_entry_id] that specifies the codec method of Clip, ref_to_STC_id [SubClip_entry_id] that is information on STC discontinuity points (system time base discontinuity points), and reserved_for_future_use.

  Note that SubPlayItem_IN_time, SubPlayItem_OUT_time, sync_PlayItem_id, and sync_start_PTS_of_PlayItem are used in common among a plurality of clips. In the example of FIG. 8, SubPlayItem_IN_time, SubPlayItem_OUT_time, sync_PlayItem_id, and sync_start_PTS_of_PlayItem are used in common between SubClip_entry_id = 0 and SubClip_entry_id = 1, and the Text based subtitle for the selected SubClip_entry_id is _ Playback is based on SubPlayItem_OUT_time, sync_PlayItem_id, and sync_start_PTS_of_PlayItem.

  Here, the value of SubClip_entry_id is assigned from 1 in the order in which Clip_Information_file_name [SubClip_entry_id] in SubPlayItem appears. Also, SubClip_entry_id of Clip_Information_file_name [0] is 0.

  FIG. 13 is a diagram illustrating the syntax of PlayItem ().

  length is a 16-bit unsigned integer indicating the number of bytes from immediately after this length field to the end of PlayItem (). Clip_Information_file_name [0] is a field for designating a Clip referred to by PlayItem. In the example of FIG. 7, the main AV stream is referred to by Clip_Information_file_name [0]. PlayItem () includes Clip_codec_identifier [0], reserved_for_future_use, is_multi_angle, connection_condition, and ref_to_STC_id [0], which is information on STC discontinuity points (system time base discontinuity points) that specify the codec method of Clip. ing. Furthermore, PlayItem () includes IN_time and OUT_time for designating the playback section of the PlayItem in the Clip. In the example of FIG. 7, the playback range of the main ClipAV stream file is represented by IN_time and OUT_time. PlayItem () includes UO_mask_table (), PlayItem_random_access_mode, and still_mode. Note that the case where there are a plurality of is_multi_angles is not directly related to the present invention, and thus the description thereof is omitted.

  STN_table () in PlayItem () refers to the PlayItem's operations such as audio switching and subtitle switching when the target PlayItem and one or more SubPaths to be played back are prepared. It provides a mechanism that allows the user to select from Clips and Clips referenced by one or more of these SubPaths. STN_table () provides a mechanism that allows selection of mixing reproduction of two audio streams.

  FIG. 14 is a diagram illustrating a first example (first STN_table ()) of syntax of STN_table (). STN_table () is set as an attribute of PlayItem.

  length is a 16-bit unsigned integer indicating the number of bytes from immediately after this length field to the end of STN_table (). 16-bit reserved_for_future_use is prepared after the length. number_of_video_stream_entries indicates the number of streams to which video_stream_id entered (registered) in STN_table () is given. video_stream_id is information for identifying a video stream, and video_stream_number is a video stream number that is used for video switching and is visible to the user.

  number_of_audio_stream_entries indicates the number of streams of the first audio stream to which audio_stream_id entered in STN_table () is given. audio_stream_id is information for identifying an audio stream. audio_stream_number is an audio stream number that can be seen by the user, and is used for audio switching. number_of_audio_stream2_entries indicates the number of streams of the second audio stream to which audio_stream_id2 entered in STN_table () is given. audio_stream_id2 is information for identifying an audio stream. audio_stream_number is an audio stream number that can be seen by the user, and is used for audio switching. Specifically, the number_of_audio_stream_entries audio stream entered in STN_table () is an audio stream decoded by the 1st audio decoder 75-1 of the playback device 20 of FIG. 25 or the playback device 401 of FIG. 49 described later, and STN_table The audio stream of number_of_audio_stream2_entries entered in () is an audio stream decoded by the 2nd audio decoder 75-2 of the playback device 20 in FIG. 25 or the playback device 401 in FIG. As described above, in STN_table () in FIG. 14, audio streams to be decoded by the two audio decoders can be entered.

  In the following, the audio stream of number_of_audio_stream_entries decoded by the 1st audio decoder 75-1 of the playback device 20 in FIG. 25 or the playback device 401 in FIG. 49 is referred to as audio stream # 1 (further, the audio stream # 1 will be described later). The audio stream of number_of_audio_stream2_entries decoded by the 2nd audio decoder 75-2 of the playback device 20 in FIG. 25 or the playback device 401 in FIG. 49 is referred to as audio stream # 2. (Further, the audio stream # 2 may be referred to as a secondary audio stream, as will be described later). Also, it is assumed that the audio stream # 1 is an audio stream that has priority over the audio stream # 2.

  number_of_PG_txtST_stream_entries indicates the number of streams to which PG_txtST_stream_id entered in STN_table () is given. In this, a stream (PG, Presentation Graphics stream) obtained by run-length encoding bitmap subtitles such as a DVD sub-picture and a text subtitle file (txtST) are entered. PG_txtST_stream_id is information for identifying a subtitle stream, and PG_txtST_stream_number is a subtitle stream number (text subtitle stream number) that can be seen by the user used for subtitle switching.

  number_of_IG_stream_entries indicates the number of streams to which IG_stream_id entered in STN_table () is given. In this, an interactive graphics stream is entered. IG_stream_id is information for identifying an interactive graphics stream, and IG_stream_number is a graphics stream number that is visible to the user used for graphics switching.

  Here, the syntax of stream_entry () will be described with reference to FIG.

  length is an 8-bit unsigned integer indicating the number of bytes from immediately after this length field to the end of stream_entry (). type is an 8-bit field indicating the type of information necessary to uniquely specify the stream to which the above-described stream number is given.

  In type = 1, a 16-bit packet ID (PID) is used to identify one elementary stream from among a plurality of elementary streams multiplexed in a Clip (Main Clip) referenced by PlayItem. Is specified. ref_to_stream_PID_of_mainClip indicates this PID. That is, with type = 1, the stream is determined simply by specifying the PID in the main ClipAV stream file.

  In type = 2, when SubPath refers to a plurality of Clips at a time and each Clip multiplexes a plurality of elementary streams, a plurality of elementary streams of one Clip (SubClip) referenced by SubPath SubPath_id, Clip id, and packet ID (PID) of the SubPath are specified in order to identify one elementary stream from the list. ref_to_SubPath_id indicates this SubPath_id, ref_to_SubClip_entry_id indicates this Clip id, and ref_to_stream_PID_of_SubClip indicates this PID. This is used when a plurality of Clips are referred to in SubPlayItem and a plurality of elementary streams are referred to this Clip.

  In this way, by using type (two types of type = 1 and type = 2), when PlayItem and one or more SubPaths to be played back are prepared, this PlayItem refers to One elementary stream can be identified from Clips referenced by Clip and one or more SubPaths. Note that type = 1 indicates a clip (main clip) referred to by the main path, and type = 2 indicates a clip (sub clip) referred to by the sub path.

  Returning to the description of the first STN_table () in FIG. 14, video_stream_id from 0 to video_stream_id is assigned to one video elementary stream specified for each stream_entry () in order in the for loop of the video stream ID (video_stream_id). Given. Note that a video stream number (video_stream_number) may be used instead of the video stream ID (video_stream_id). In this case, video_stream_number is given from 1 instead of 0. That is, video_stream_number is obtained by adding 1 to the value of video_stream_id. The video stream number is defined from 1 since it is a video stream number that is used for video switching and is visible to the user.

  Similarly, in the for loop of the audio stream ID (audio_stream_id), audio_stream_id is given from 0 to one audio elementary stream that is specified for each stream_entry () in order. As in the case of the video stream, an audio stream number (audio_stream_number) may be used instead of the audio stream ID (audio_stream_id). In this case, audio_stream_number is given from 1 instead of 0. That is, audio_stream_number is obtained by adding 1 to the value of audio_stream_id. The audio stream number is an audio stream number that can be seen by the user and is used for audio switching.

  Similarly, audio_stream_id2 is given from 0 to one audio elementary stream specified for each stream_entry () in order in the for loop of the audio stream ID2 (audio_stream_id2). As in the case of the video stream, audio stream number 2 (audio_stream_number2) may be used instead of audio stream ID2 (audio_stream_id2). In this case, audio_stream_number2 is given from 1 instead of 0. That is, audio_stream_number2 is obtained by adding 1 to the value of audio_stream_id2. The audio stream number 2 is an audio stream number 2 that can be seen by the user and used for audio switching.

  That is, in the first STN_table () of FIG. 14, an audio stream of number_of_audio_stream_entries (audio stream # 1) and an audio stream of number_of_audio_stream2_entries (audio stream # 2) are defined. In other words, since the audio stream # 1 and the audio stream # 2 can be entered using the STN_table (), the user can select two audio streams to be reproduced in synchronization.

  Similarly, PG_txtST_stream_id is assigned from 0 to one bitmap subtitle elementary stream or text subtitle identified in order for each stream_entry () in the for loop of the subtitle stream ID (PG_txtST_stream_id). As in the case of the video stream, a subtitle stream number (PG_txtST_stream_number) may be used instead of the subtitle stream ID (PG_txtST_stream_id). In this case, PG_txtST_stream_number is given from 1 instead of 0. That is, PG_txtST_stream_number is obtained by adding 1 to the value of PG_txtST_stream_id. The subtitle stream number is a subtitle stream number (text subtitle stream number) that can be seen by the user used for subtitle switching, and is defined from 1.

  Similarly, in the for loop of the graphics stream ID (IG_stream_id), IG_stream_id is assigned from 0 to one interactive graphics elementary stream that is identified for each stream_entry () in order. As in the case of the video stream, a graphics stream number (IG_stream_number) may be used instead of the graphics stream ID (IG_stream_id). In this case, IG_stream_number is given from 1 instead of 0. That is, IG_stream_number is obtained by adding 1 to the value of IG_stream_id. Since the graphics stream number is a graphics stream number that can be seen by the user used for switching graphics, it is defined from 1.

  Next, stream_attribute () of the first STN_table () in FIG. 14 will be described.

  In the for statement after reserved_for_future_use, the video stream is referenced by the amount of the video stream, the audio stream is referenced by the disc manufacturer who has set the Main Path and Sub Path for the audio stream, and the PG textST stream Only the PG textST stream is referred to, and the IG stream is referred to by the amount of the IG stream.

  The stream_attribute () in the for loop of the video stream ID (video_stream_id) gives stream attribute information of one video elementary stream specified for each stream_entry (). That is, in this stream_attribute (), stream attribute information of one video elementary stream specified for each stream_entry () is described.

  Similarly, stream_attribute () in the for loop of the audio stream ID (audio_stream_id) gives stream attribute information of one audio elementary stream specified for each stream_entry (). That is, in this stream_attribute (), stream attribute information of one audio elementary stream specified for each stream_entry () is described. For example, since there is one audio elementary stream specified by type = 1 or type = 2 of stream_entry () in FIG. 15, stream_attribute () gives stream attribute information of that one audio elementary stream.

  Similarly, stream_attribute () in the for loop of audio stream ID2 (audio_stream_id2) gives stream attribute information of one audio elementary stream specified for each stream_entry (). That is, in this stream_attribute (), stream attribute information of one audio elementary stream specified for each stream_entry () is described. For example, since there is one audio elementary stream specified by type = 1 or type = 2 of stream_entry () in FIG. 15, stream_attribute () gives stream attribute information of that one audio elementary stream.

  Similarly, stream_attribute () in the for loop of the subtitle stream ID (PG_txtST_stream_id) gives stream attribute information of one bitmap subtitle elementary stream or text subtitle elementary stream specified for each stream_entry (). That is, in this stream_attribute (), stream attribute information of one bitmap subtitle elementary stream specified for each stream_entry () is described.

  Similarly, stream_attribute () in the for loop of the graphics stream ID (IG_stream_id) gives stream attribute information of one interactive graphics elementary stream specified for each stream_entry (). That is, stream attribute information of one interactive graphics elementary stream specified for each stream_entry () is described in the stream_attribute ().

  Here, the syntax of stream_attribute () will be described with reference to FIG.

  The length is a 16-bit unsigned integer indicating the number of bytes from immediately after this length field to the end of stream_attribute ().

  stream_coding_type indicates the encoding type of the elementary stream as shown in FIG. The encoding type of elementary stream includes video codec format such as MPEG-2 video stream, audio codec format such as HDMV LPCM audio, Dolby AC-3 audio, and dts audio, Presentation graphics stream, Interactive graphics stream, And a Text subtitle stream are described.

  video_format indicates the video format of the video elementary stream as shown in FIG. 480i, 576i, 480p, 1080i, 720p, and 1080p are described as the video format of the video elementary stream.

  As shown in FIG. 19, frame_rate indicates the frame rate of the video elementary stream. As frame rates of the video elementary stream, 24000/1001, 24, 25, 30000/1001, 50, and 60000/1001 are described.

  The aspect_ratio indicates the aspect ratio information of the video elementary stream as shown in FIG. As aspect ratio information of the video elementary stream, 4: 3 display aspect ratio and 16: 9 display aspect ratio are described.

  audio_presentation_type indicates presentation type information of an audio elementary stream as shown in FIG. As the presentation type information of the audio elementary stream, single mono channel, dual mono channel, stereo (2-channel), and multi-channel are described.

  Sampling_frequency indicates the sampling frequency of the audio elementary stream as shown in FIG. As sampling frequencies of audio elementary streams, 48 kHz and 96 kHz are described.

  audio_language_code indicates the language code of the audio elementary stream (Japanese, Korean, Chinese, etc.).

  PG_language_code indicates the language code (Japanese, Korean, Chinese, etc.) of the bitmap subtitle elementary stream.

  IG_language_code, which indicates the language code (Japanese, Korean, Chinese, etc.) of the interactive graphics elementary stream.

  textST_language_code indicates the language code (Japanese, Korean, Chinese, etc.) of the text subtitle elementary stream.

  The character_code indicates the character code of the text subtitle elementary stream as shown in FIG. The character code of the text subtitle elementary stream includes Unicode V1.1 (ISO 10646-1), Shift JIS (Japanese), KSC 5601-1987 including KSC 5653 for Roman character (Korean), GB 18030-2000 (Chinese), GB2312 (Chinese) and BIG5 (Chinese) are described.

  Hereinafter, the syntax of stream_attribute () in FIG. 16 will be specifically described with reference to FIGS. 16 and 17 to 23.

  When the encoding type of the elementary stream (stream_coding_type in FIG. 16) is an MPEG-2 video stream (FIG. 17), the stream_attribute () includes the video format (FIG. 18) and frame rate (FIG. 19) of the elementary stream. ), And aspect ratio information (FIG. 20).

  When the encoding type of the elementary stream (stream_coding_type in FIG. 16) is HDMV LPCM audio, Dolby AC-3 audio, or dts audio (FIG. 17), the stream_attribute () includes the presentation type information of the audio elementary stream. (FIG. 21), sampling frequency (FIG. 22), and language code.

  When the encoding type of the elementary stream (stream_coding_type in FIG. 16) is a Presentation graphics stream (FIG. 17), the stream_attribute () includes the language code of the bitmap subtitle elementary stream.

  When the encoding type of the elementary stream (stream_coding_type in FIG. 16) is Interactive graphics stream (FIG. 17), stream_attribute () includes the language code of the interactive graphics elementary stream.

  When the encoding type of the elementary stream (stream_coding_type in FIG. 16) is a Text subtitle stream (FIG. 17), stream_attribute () includes the character code (FIG. 23) and language code of the text subtitle elementary stream. .

  Note that the attribute information is not limited to this.

  As described above, when a PlayItem and one or more SubPaths to be played back are prepared, a stream_entry () is used to select a Clip referred to by the PlayItem and a Clip referred to by one or more SubPaths. The attribute information of one identified elementary stream can be known by stream_attribute ().

  By examining this attribute information (stream_attribute ()), the playback device can check whether or not it has the function of playing back the elementary stream. Further, the playback device can select the elementary stream corresponding to the initial information of the language setting of the playback device by examining the attribute information.

  For example, it is assumed that the playback apparatus has only a playback function for a bitmap subtitle elementary stream and does not have a playback function for a text subtitle elementary stream. When the user instructs the playback apparatus to switch the language, the playback apparatus sequentially selects and reproduces only the bitmap subtitle elementary stream from the for loop of the subtitle stream ID (PG_txtST_stream_id).

  Also, for example, assume that the initial information of the language setting of the playback device is Japanese. When the user instructs the playback apparatus to switch to the secondary audio (that is, secondary audio) or to mix and reproduce the main audio and the secondary audio, the playback apparatus uses the audio stream ID ( Only audio elementary streams whose language code is Japanese are sequentially selected from the for loop of (Audio stream id) and played.

  Further, for example, when playing back an AV stream (movie) made up of a video stream and an audio stream that is referenced by the main path, the user instructs the playback device to switch the audio, and the audio stream # 1 (normal When the audio output in the movie) and the audio stream # 2 (comment by the director or performer) are designated (selected) as the audio to be played back, the playback device selects the audio stream # 1 and the audio stream # 2. Mix (superimpose) and play along with the video stream.

  Note that both the audio stream # 1 and the audio stream # 2 may be audio streams included in the Clip referred to by the main path, as can be seen by referring to the STN_table () in FIG. 14 and FIG. Also, one of the audio stream # 1 and the audio stream # 2 may be an audio stream included in a clip referred to by the main path, and the other may be an audio stream included in the clip referenced by the sub path. As described above, it is possible to select two audio streams superimposed on the main AV stream referred to by the main path, mix them, and reproduce them.

  In this way, by providing STN_table () in PlayItem (), when PlayItem and one or more SubPaths to be played back are prepared, operations such as audio switching and subtitle switching by the user In response to this, a stream to be played can be selected from the Clip referred to by this PlayItem and the Clip referred to by one or more SubPaths. As a result, an interactive operation can be performed on a stream or data file in which the main AV stream is recorded and which is different from the AV stream to be reproduced.

  Also, since a plurality of SubPaths are used in one PlayList and each SubPath refers to a SubPlayItem, an AV stream with high expandability and high flexibility can be realized. That is, a configuration in which SubPlayItem can be added later in the content played back by this playlist. For example, when there is a ClipAV stream file referenced by the Main Path and a PlayList associated with the ClipAV stream file, and this PlayList is rewritten to a PlayList with a new Sub Path added, the ClipAV referenced by the Main Path is based on the new PlayList. Along with the stream file, reproduction can be performed by referring to a ClipAV stream file different from the ClipAV stream file referred to by the Main Path. Thus, it can be set as the structure which has an expandability.

  Furthermore, by providing STN_table () in PlayItem (), audio stream # 1 and 2nd audio decoded by the first audio decoder 75-1 of the playback device 20 shown in FIG. 25 or the playback device 401 shown in FIG. The audio stream # 2 decoded by the decoder 75-2 can be mixed and reproduced. For example, when PlayItem () and one or more SubPaths to be played back are prepared, the clip audio stream referenced by PlayItem is defined as audio stream # 1, and the clip audio stream referenced by SubPath is defined as audio stream # 1. An audio stream # 2 is provided, and a mechanism capable of mixing and reproducing these is provided. Also, for example, two audio streams included in a clip (main clip) referred to by PlayItem are set as an audio stream # 1 and an audio stream # 2, respectively, and these can be mixed and played back. did. Accordingly, it is possible to perform superimposed reproduction of an audio stream (for example, a director's comment stream) that is different from the main audio stream to be reproduced, in which the main AV stream is recorded. Also, the two audio streams # 1 and # 2 superimposed on the main AV stream can be superimposed (mixed) and reproduced.

  A specific example will be described with reference to FIG. FIG. 24 is a diagram illustrating an example of a stream number table representing a relationship between an audio signal provided to a user and a caption signal.

  In FIG. 24, the audio numbers are referred to as A_SN (Audio Stream Number) and A_SN2, and the subtitle numbers are referred to as S_SN (SubPicture Stream Number). In FIG. 24, A_SN is given to each of the audio streams # 1 (audio streams entered with audio_stream_id) entered in STN_table () of PlayItems constituting the Main Path of the PlayList, and PlayItems constituting the Main Path of the PlayList. A_SN2 is given to each of the audio streams # 2 (audio streams entered with audio_stream_id2) entered in STN_table ().

  Specifically, audio 2 is given to A_SN = 1, audio 1 is given to A_SN = 2, and audio 3 is given to A_SN = 3. Audio 4 is given to A_SN2 = 1, and audio 5 is given to A_SN2 = 2. The user selects the audio stream # 1 to be reproduced from the audio streams to which A_SN is given, and selects the audio stream # 2 to be mixed with the selected audio stream # 1 in the audio stream to which A_SN2 is given. Select from. For example, the user selects audio 1 with A_SN = 2 and audio 5 with A_SN2 = 2 as playback audio streams.

  As a specific example, when the audio 2 with A_SN = 1 is selected and the user commands the audio switching, the audio is switched to the audio 1 with A_SN = 2, and the user further selects the audio. When switching is instructed, the audio is switched to audio 3 with A_SN = 3. Further, when the user instructs to switch audio further, the audio is switched to audio 2 with A_SN = 1. Also, for example, when the audio 4 with A_SN2 = 1 is selected and the user commands to switch audio, the audio is switched to audio 5 with A_SN2 = 2, and the user further commands to switch audio. In this case, the audio is switched to the audio 4 with A_SN2 = 1. As described above, A_SN for selecting the audio stream # 1 and A_SN2 for selecting the audio stream # 2 are independent from each other. That is, the user selects one audio stream from A_SN = 1 to A_SN = 3, and selects one audio stream from A_SN2 = 1 and A_SN2 = 2.

  Also, S_SN is assigned to each of a plurality of sub-picture streams of the main AV stream referred to by the main path of the PlayList. That is, sub-picture 3 is given to S_SN = 1, sub-picture 1 is given to S_SN = 2, and sub-picture 2 is given to S_SN = 3.

  Note that here, the smaller the numbers of A_SN, A_SN2, and S_SN, the higher the priority of the audio signal and sub-picture provided to the user. Further, the stream given by A_SN has higher priority than the stream given by A_SN2. That is, A_SN = 1 is an audio stream reproduced by default, and S_SN = 1 is a sub-picture stream reproduced by default.

  Specifically, the audio reproduced based on the initial information of the language setting of the reproduction apparatus corresponds to audio 2 (FIG. 24) with A_SN = 1, and the audio reproduced after the audio is switched. Corresponds to audio 1 (FIG. 24) with A_SN = 2.

  In order to provide such a stream number table, in the STN table () (FIG. 14) in PlayItem () referenced by PlayList (), first, audio_stream_id = 0 for entry to audio stream # 1 Audio 2 is given by (A_SN = 1), audio 1 is given by audio_stream_id = 1 (A_SN = 2), and audio 3 is given by audio_stream_id = 2 (A_SN = 3). Next, in STN table () (FIG. 14), audio 4 is given by audio_stream_id2 = 0 (A_SN2 = 1) for entry into audio stream # 2, and audio 5 is given by audio_stream_id2 = 1 (A_SN2 = 2).

  That is, by defining two audio streams to be reproduced (audio stream # 1 and audio stream # 2) separately, the user can arbitrarily select two audio streams to be reproduced from the defined ones. Can do. In other words, the user can freely select two audio streams to be played (freely defined by the audio stream # 1 and the audio stream # 2). High selection can be made. For example, the user can select a combination of audio 2 + audio 4 (a combination of A_SN = 1 and A_SN2 = 1) or a combination of audio 2 + audio 5 (a combination of A_SN = 1 and A_SN2 = 2).

  As described above, the stream_entry () (FIG. 15) of the STN table () (FIG. 14) in the PlayItem () is configured so that two audio streams can be entered. Can be mixed and played back. That is, two streams of the same type (in this example, an audio stream) can be mixed (superimposed or combined) from a plurality of types of streams and simultaneously played back. In addition, the user can instruct mixing reproduction of two desired types of streams.

  Next, a playback apparatus to which the present invention is applied will be described. FIG. 25 is a block diagram illustrating a configuration example of the playback device 20 to which the present invention has been applied. The playback device 20 is a playback device 20 that plays the PlayList having the main path and the sub path described above.

  The playback device 20 is provided with a storage drive 31, a switch 32, an AV decoder unit 33, and a controller 34.

  In the case of the example of FIG. 25, first, the controller 34 reads out the PlayList file via the storage drive 31, and based on the information of the PlayList file, the HDD, Blu-Ray Disc (trademark) or DVD via the storage drive 31. AV streams and AV data are read from a recording medium such as The user can instruct the controller 34 to switch between audio and subtitles using the user interface. Further, the controller 34 is supplied with initial information of the language setting of the playback device 20 from a storage unit (not shown).

  The PlayList file includes STN_table () in addition to Main Path and Sub Path information. The controller 34 receives the main Clip AV stream file referenced by the PlayItem included in the PlayList file, the sub Clip AV stream file referenced by the SubPlayItem, the text subtitle data referenced by the SubPlayItem, and the like from the recording medium via the storage drive 31. read out. Here, the main Clip AV Stream referred to by PlayItem and the sub Clip AV Stream referred to by SubPlayItem may be recorded on different recording media. For example, the main Clip AV Stream may be recorded on a recording medium, and the corresponding sub Clip AV Stream may be supplied via a network (not shown) and stored in the HDD. In addition, the controller 34 selects an elementary stream corresponding to the playback function of itself (the playback device 20) and controls playback, or selects an elementary stream corresponding to the initial information of the language setting of the playback device 20. And control to play. In general, since a stream referred to by PlayItem includes a video stream, a Clip including a stream referred to by PlayItem (main path) is referred to as a main Clip. A stream other than the main clip is called a sub clip. That is, the sub clip is a stream that is referred to in the sub path, and is a video stream that replaces the video stream that is referred to by PlayItem, a video stream that is combined with a video stream that is referenced by PlayItem, or an audio stream that does not include a video stream, , PG streams, etc., a clip including at least one stream is called a sub clip.

  The AV decoder unit 33 includes buffers 51 to 54, PID filter 55, PID filter 56, switches 57 to 59, background decoder 71, video decoder 72, presentation graphics decoder 73, interactive graphics decoder 74, 1st audio decoder 75. -1 and 2nd audio decoder 75-2, Text-ST composition 76, switch 77, background plane generation unit 91, video plane generation unit 92, presentation graphics plane generation unit 93, interactive graphics plane generation unit 94, buffer 95, a video data processing unit 96, a mixing processing unit 97, and a mixing processing unit 98 are provided. The 1st audio decoder 75-1 is for decoding the audio stream # 1, and the 2nd audio decoder 75-2 is for decoding the audio stream # 2. Specifically, in STN_table () of FIG. 14, the 1st audio decoder 75-1 is for decoding the audio stream given by audio_stream_id, and the 2nd audio is for decoding the audio stream given by audio_stream_id2. This is a decoder 75-2.

  Thus, the playback device 20 has two audio decoders (1st audio decoder 75-1 and 2nd audio decoder 75-2) in order to decode two audio streams. In the following, the 1st audio decoder 75-1 and the 2nd audio decoder 75-2 are referred to as audio decoders 75 when not distinguished from each other.

  The file data read by the controller 34 is demodulated by a demodulation / ECC decoding unit (not shown), and error correction is performed on the demodulated multiplexed stream. The switch 32 selects the demodulated and error-corrected data for each stream type based on the control from the controller 34 and supplies the selected data to the corresponding buffers 51 to 54. Specifically, the switch 32 supplies the background image data to the buffer 51, supplies the main Clip AV Stream data to the buffer 52, and buffers the sub Clip AV Stream data based on the control from the controller 34. The switch 32 is switched so that the Text-ST data is supplied to the buffer 54. The buffer 51 buffers background image data, the buffer 52 buffers main Clip AV Stream data, the buffer 53 buffers sub Clip AV Stream data, and the buffer 54 Text-ST. Buffer data.

  The main Clip AV Stream is a stream (for example, a transport stream) obtained by multiplexing one or more streams in addition to video among video, audio, bitmap subtitle (Presentation Graphics stream), and interactive graphics. The sub Clip AV Stream is a stream obtained by multiplexing one or more streams among audio, bitmap subtitle (Presentation Graphics stream), interactive graphics, and audio. Note that the data of the text subtitle data file (Text-ST) may or may not be in the form of a multiplexed stream such as a transport stream.

  Further, when reading the main Clip AV Stream, the sub Clip AV Stream, and the text subtitle data from the storage drive 31 (recording medium), the respective files may be alternately read in a time division manner, or the sub Clip AV Stream may be read. Alternatively, all the text subtitle data may be preloaded into the buffer (buffer 53 or buffer 54) before being read from the main Clip AV Stream.

  The playback device 20 reads the data of these files from the recording medium via the storage drive 31, and plays back video, bitmap subtitles, interactive graphics, and audio.

  Specifically, the stream data read from the buffer 52 that is the main Clip AV Stream read buffer is output to the PID (packet ID) filter 55 in the subsequent stage at a predetermined timing. The PID filter 55 distributes the input main Clip AV Stream to the decoders of the elementary streams in the subsequent stage according to the PID (packet ID) and outputs the result. That is, the PID filter 55 supplies the video stream to the video decoder 72, supplies the presentation graphics stream to the switch 57 serving as the supply source to the presentation graphics decoder 73, and supplies the interactive graphics stream to the interactive graphics decoder 74. The audio stream is supplied to the switch 59 serving as the supply source to the 1st audio decoder 75-1 and the 2nd audio decoder 75-2.

  The presentation graphics stream is, for example, bitmap subtitle data, and the text subtitle data is, for example, text subtitle data.

  The stream data read from the buffer 53, which is a sub-Clip AV Stream read buffer, is output to a PID (packet ID) filter 56 at the subsequent stage at a predetermined timing. The PID filter 56 distributes the input sub clip to the decoder of each elementary stream in the subsequent stage according to the PID (packet ID) and outputs the result. That is, the PID filter 56 supplies the presentation graphics stream to the switch 57 serving as the supply source to the presentation graphics decoder 73 and supplies the interactive graphics stream to the switch 58 serving as the supply source to the interactive graphics decoder 74. The audio stream is supplied to the switch 59 serving as a supply source to the 1st audio decoder 75-1 and the 2nd audio decoder 75-2.

  Data read from the buffer 51 that buffers background image data is supplied to the background decoder 71 at a predetermined timing. The background decoder 71 decodes the background image data and supplies the decoded background image data to the background plane generation unit 91.

  The video stream distributed by the PID filter 55 is supplied to the video decoder 72 at the subsequent stage. The video decoder 72 decodes the video stream and outputs the decoded video data to the video plane generation unit 92.

  The switch 57 selects one of the presentation graphics stream included in the main Clip AV Stream supplied from the PID filter 55 and the presentation graphics stream included in the sub Clip AV Stream, and selects the selected presentation graphic. The stream is supplied to the presentation graphics decoder 73 at the subsequent stage. The presentation graphics decoder 73 decodes the presentation graphics stream, and supplies the decoded presentation graphics stream data to the switch 77 serving as a supply source to the presentation graphics plane generation unit 93.

  The switch 58 selects one of the interactive graphics stream included in the main Clip AV Stream supplied from the PID filter 55 and the interactive graphics stream included in the sub Clip AV Stream. The interactive graphics stream is supplied to the subsequent interactive graphics stream decoder 74. That is, the interactive graphics stream input to the interactive graphics decoder 74 at the same time is a stream separated from either the main Clip AV Stream or the sub Clip AV Stream. The interactive graphics decoder 74 decodes the interactive graphics stream and supplies the decoded interactive graphics stream data to the interactive graphics plane generating unit 94.

  Further, the switch 59 selects one of the audio stream included in the main Clip AV Stream and the audio stream included in the sub Clip AV Stream supplied from the PID filter 55, and selects the selected audio stream. The signal is supplied to the 1st audio decoder 75-1 or the 2nd audio decoder 75-2 at the subsequent stage. Here, the audio stream simultaneously input to the 1st audio decoder 75-1 is a stream separated from either the main clip or the sub clip. Similarly, the audio stream input to the 2nd audio decoder 75-2 at the same time is a stream separated from either the main clip or the sub clip. For example, when the audio stream # 1 and the audio stream # 2 are included in the main clip, the PID filter 55 filters the audio stream # 1 and the audio stream # 2 based on the PID of the audio stream, and switches 59.

  For example, the switch 59 selects the switch so that the audio stream # 1 supplied from the PID filter 55 is supplied to the 1st audio decoder 75-1, and the audio stream # 2 supplied from the PID filter 55 is selected as the 2nd audio decoder. Select the switch to supply to 75-2.

  The 1st audio decoder 75-1 decodes the audio stream and supplies the decoded audio stream data to the mixing processing unit 101. The 2nd audio decoder 75-2 decodes the audio stream and supplies the decoded audio stream data to the mixing processing unit 101.

  Here, when audio stream # 1 and audio stream # 2 are superimposed and reproduced (when two audio streams are selected as audio streams to be reproduced by the user), decoding is performed by the 1st audio decoder 75-1. The audio stream # 1 and the audio stream # 2 decoded by the 2nd audio decoder 75-2 are supplied to the mixing processing unit 101.

  The mixing processing unit 101 mixes (superimposes) the audio data from the 1st audio decoder 75-1 and the audio data from the 2nd audio decoder 75-2, and outputs the mixed data to the subsequent mixing processing unit 97. In the present embodiment, mixing (superimposing) the audio data output from the 1st audio decoder 75-1 and the audio data output from the 2nd audio decoder 75-2 is also referred to as synthesis. In other words, synthesis means to mix two audio data.

  The sound data selected by the switch 32 is supplied to the buffer 95 and buffered. The buffer 95 supplies the sound data to the mixing processing unit 97 at a predetermined timing. The sound data is, for example, sound effect data by menu selection or the like. The mixing processing unit 97 is the audio data mixed by the mixing processing unit 101 (audio data obtained by mixing the audio data output from the 1st audio decoder 75-1 and the audio data output from the 2nd audio decoder 75-2). The sound data supplied from the buffer 95 is mixed (superimposed or synthesized) and output as an audio signal.

  The data read from the buffer 54 that is a text subtitle read buffer is output to the text subtitle composition (decoder) 76 at the subsequent stage at a predetermined timing. The text subtitle composition 76 decodes the Text-ST data and supplies it to the switch 77.

  The switch 77 selects either the presentation graphics stream decoded by the presentation graphics decoder 73 or Text-ST (text subtitle data), and supplies the selected data to the presentation graphics plane generating unit 93. . That is, the subtitle image simultaneously supplied to the presentation graphics plane generating unit 93 is an output of either the presentation graphics decoder 73 or the text subtitle (Text-ST) composition 76. The presentation graphics stream that is simultaneously input to the presentation graphics decoder 73 is a stream separated from either the main Clip AV Stream or the sub Clip AV Stream (selected by the switch 57). Therefore, the subtitle image that is simultaneously output to the presentation graphics plane generating unit 93 is a decoding output of the presentation graphics stream from the main Clip AV Stream, the presentation graphics stream from the sub Clip AV Stream, or the text subtitle data. It is.

  Based on the background image data supplied from the background decoder 71, for example, the background plane generation unit 91 generates a background plane that becomes a wallpaper image when the video image is displayed in a reduced size, and this is converted into the video data. This is supplied to the processing unit 96. The video plane generation unit 92 generates a video plane based on the video data supplied from the video decoder 72 and supplies this to the video data processing unit 96. The presentation graphics plane generation unit 93 generates, for example, a presentation graphics plane which is a rendering image based on the data (presentation graphics stream or text subtitle data) selected by the switch 77 and supplied, and this is converted into a video. The data is supplied to the data processing unit 96. The interactive graphics plane generating unit 94 generates an interactive graphics plane based on the data of the interactive graphics stream supplied from the interactive graphics decoder 74, and supplies this to the video data processing unit 96.

  The video data processing unit 96 includes a background plane from the background plane generation unit 91, a video plane from the video plane generation unit 92, a presentation graphics plane from the presentation graphics plane generation unit 93, and an interactive graphics plane generation unit. The interactive graphics plane from 94 is synthesized and output as a video signal. The mixing processing unit 97 also receives audio data from the mixing processing unit 101 (audio data obtained by mixing audio data decoded by the first audio decoder 75-1 and audio data decoded by the second audio decoder 75-2). The sound data from the buffer 95 is mixed (synthesized or superimposed) and output as an audio signal.

  The switches 57 to 59 and the switch 77 are switched based on selection from the user via the user interface or the file side including the target data. For example, when the audio stream is included only in the sub clip AV stream file, the switch 59 switches the switch to the sub side.

  Next, playback processing in the playback apparatus 20 of FIG. 25 will be described with reference to the flowcharts of FIGS. Note that this process is started when a reproduction of a predetermined AV stream is instructed by the user via the user interface.

  In step S <b> 11, the controller 34 reads out a PlayList file recorded on a recording medium or an HDD (Hard Disk Drive) (not shown) via the storage drive 31. For example, the PlayList file described with reference to FIG. 9 is read.

  In step S12, the controller 34 reads out the main Clip AV Stream, the sub Clip AV Stream, and the text subtitle data (Text-ST data). Specifically, the controller 34 reads out the main Clip AV Stream based on the PlayItem included in the PlayList described with reference to FIG. Further, the controller 34 reads the sub Clip AV Stream and the text subtitle data based on the SubPlayItem described with reference to FIGS. 10 to 12 that is referred to by the SubPath included in the PlayList.

  In step S <b> 13, the controller 34 controls the switch 32 to supply the read data (main Clip AV Stream, sub Clip AV Stream, and text subtitle data) to the corresponding buffers 51 to 54. Specifically, the controller 34 supplies the background image data to the buffer 51, supplies the main Clip AV Stream data to the buffer 52, supplies the sub Clip data to the buffer 53, and converts the Text-ST data to the buffer 53. Switch 32 is switched to supply to buffer 54.

  In step S <b> 14, the switch 32 switches the switch 32 based on control from the controller 34. As a result, the background image data is supplied to the buffer 51, the main Clip AV Stream data is supplied to the buffer 52, the sub Clip AV Stream data is supplied to the buffer 53, and the text subtitle data is supplied to the buffer 54. .

  In step S15, each of the buffers 51 to 54 buffers the supplied data. Specifically, the buffer 51 buffers background image data, the buffer 52 buffers the data of the main Clip AV Stream, the buffer 53 buffers the data of the sub Clip AV Stream, and the buffer 54 Buffers Text-ST data.

  In step S <b> 16, the buffer 51 outputs the background image data to the background decoder 71.

  In step S <b> 17, the buffer 52 outputs the stream data of the main Clip AV Stream to the PID filter 55.

  In step S18, the PID filter 55 distributes to the decoders of the elementary streams based on the PIDs attached to the TS (Transport Stream) packets constituting the main Clip AV stream file. Specifically, the PID filter 55 supplies the video stream to the video decoder 72, supplies the presentation graphics stream to the switch 57 serving as the supply source to the presentation graphics decoder 73, and converts the interactive graphics stream to the interactive graphics. The audio stream is supplied to the switch 58 serving as the supply source to the decoder 74, and the audio stream is supplied to the switch 59 serving as the supply source to the 1st audio decoder 75-1. That is, different PIDs are attached to the video stream, presentation graphics stream, interactive graphics stream, and audio stream, respectively.

  In step S <b> 19, the buffer 53 outputs the sub Clip AV Stream stream data to the PID filter 56.

  In step S20, the PID filter 56 distributes each elementary stream to a decoder corresponding to each elementary stream based on the PID. Specifically, the PID filter 56 supplies the presentation graphics stream to the switch 57 serving as the supply source to the presentation graphics decoder 73, and the switch 58 serving as the supply source to the interactive graphics decoder 74. And the audio stream is supplied to the switch 59 serving as a supply source to the 1st audio decoder 75-1 and the 2nd audio decoder 75-2.

  In step S21, the switches 57 to 59 subsequent to the PID filter 55 and the PID filter 56 select either the main Clip AV Stream or the sub Clip AV Stream based on the control from the controller 34 via the user interface. Specifically, the switch 57 selects the presentation graphics stream of the main Clip AV Stream or the sub Clip AV Stream supplied from the PID filter 55 and supplies it to the presentation graphics decoder 73 at the subsequent stage. The switch 58 selects the main clip AV stream or sub clip AV stream interactive graphics stream supplied from the PID filter 55 and supplies the selected interactive graphics stream to the subsequent interactive graphics stream decoder 74. Further, the switch 59 is an audio stream of the main Clip AV Stream supplied from the PID filter 55 or the sub Clip AV Stream supplied from the PID filter 56 (in this case, since the audio is not switched, the audio stream # 1) is selected and supplied to the first audio decoder 75-1. Note that when the user issues a command to switch the audio, the switch 59 supplies the audio stream of the main Clip AV Stream to the 2nd audio decoder 75-2 or the audio stream of the sub Clip AV Stream 1st. Although it is supplied to the audio decoder 75-1 and the 2nd audio decoder 75-2, here, since the reproduction process before the sound is switched is described, the description thereof is omitted.

  In step S <b> 22, the buffer 54 outputs the text subtitle data to the text subtitle composition 76.

  In step S <b> 23, the background decoder 71 decodes the background image data and outputs it to the background plane generation unit 91.

  In step S <b> 24, the video decoder 72 decodes the video stream and outputs it to the video plane generation unit 92.

  In step S25, the presentation graphics decoder 73 decodes the presentation graphics stream selected and supplied by the switch 57, and outputs it to the subsequent switch 77.

  In step S <b> 26, the interactive graphics decoder 74 decodes the interactive graphics stream selected and supplied by the switch 58, and outputs this to the subsequent interactive graphics plane generating unit 94.

  In step S27, the 1st audio decoder 75-1 decodes the supplied audio stream (audio stream # 1) selected by the switch 59 and outputs it to the subsequent mixing processing unit 101. In the reproduction process (reproduction process of FIGS. 26 to 28) in a state where the user has not instructed to switch the audio, the audio data is not output from the 2nd audio decoder 75-2. The audio data output from the 1st audio decoder 75-1 is supplied to the subsequent mixing processing unit 97 as it is.

  In step S28, the Text-ST composition 76 decodes the text subtitle data and outputs it to the switch 77 at the subsequent stage.

  In step S29, the switch 77 selects either the presentation graphics decoder 73 or the data from the Text-ST composition 76. Specifically, the switch 77 selects any one of the presentation graphics stream decoded by the presentation graphics decoder 73 and Text-ST (text subtitle data), and the selected data is displayed in the presentation graphics sprayer. Supplied to the generator 93.

  In step S <b> 30, the background plane generation unit 91 generates a background plane based on the background image data supplied from the background decoder 71.

  In step S31, the video plane generating unit 92 generates a video plane based on the video data supplied from the video decoder 72.

  In step S32, the presentation graphics plane generating unit 93 selects the presentation based on the data from the presentation graphics decoder 73 or the data from the Text-ST composition 76 selected and supplied by the switch 77 in the process of step S29. Generate a graphics plane.

  In step S <b> 33, the interactive graphics plane generating unit 94 generates an interactive graphics plane based on the interactive graphics stream data supplied from the interactive graphics decoder 74.

  In step S34, the buffer 95 buffers the sound data selected and supplied in step S14 and supplies the sound data to the mixing processing unit 97 at a predetermined timing.

  In step S35, the video data processing unit 96 combines and outputs the data of each plane. Specifically, the data from the background plane generation unit 91, the video plane generation unit 92, the presentation graphics plane generation unit 93, and the interactive graphics plane generation unit 94 are combined and output as video data.

  In step S36, the mixing processing unit 97 mixes (synthesizes) audio data (audio data output from the mixing processing unit 101) and sound data, and outputs the result.

  26 to 28, the main Clip AV Stream, the sub Clip AV Stream, and the text subtitle data associated with the main path and the sub path included in the PlayList are referred to and reproduced. Since the main path and sub path are provided in the PlayList, and the Clip AV Stream file different from the Clip AV Stream file specified by the main path can be specified by the sub path, the main Clip AV Stream indicated by the PlayItem of the main path is The data of the sub clip, which is a different clip from the included clip, and the data of the main clip can be reproduced together (at the same timing).

  In FIG. 26 to FIG. 28, the processing of step S16 and step S17 may be performed in reverse order or in parallel. In addition, the processes in step S18 and step S20 may be performed in reverse order or in parallel. Further, the order of steps S23 to S28 may be reversed or may be executed in parallel. In addition, the processes in steps S30 to S33 may be executed in reverse order or in parallel. Furthermore, the order of steps S35 and S36 may be reversed, or may be executed in parallel. That is, in FIG. 25, the processing of buffers 51 to 54 in the same vertical layer, the processing of switches 57 to 59, the processing of decoders 71 to 76, the processing of plane generation units 91 to 94, the video data processing unit 96, and the mixing processing unit The processes 97 may be executed in parallel, and the order thereof is not limited.

  Next, processing in the playback device 20 when an instruction to switch audio is given will be described with reference to the flowcharts of FIGS. First, with reference to the flowchart of FIG. 29, a process when the user instructs to switch audio for the audio stream # 1 will be described. The audio switching process for the audio stream # 2 will be described later with reference to FIG. Note that the processes in FIGS. 29 and 30 are processes executed during the execution of the reproduction processes in FIGS. 26 to 28, for example.

  In step S51, the controller 34 acquires an order list of audio stream numbers (or IDs). Specifically, the controller 34 refers to the STN_table () of the PlayItem described with reference to FIG. 13, and further changes the order of the audio stream numbers (ID) entered in the STN_table () described with reference to FIG. Get a list. This process is a process executed when the reproduction process of FIGS. 26 to 28 is started.

  When a voice switching command is issued by the user via the user interface, in step S52, the controller 34 receives a voice switching command for the audio stream # 1 from the user. At this time, the controller 34 may control a display unit (not shown) to display a stream number table as shown in FIG. 24 on the display unit. Since the process in FIG. 29 is a process in the case where the audio switching for the audio stream # 1 is instructed, the user, for example, in the stream number table as shown in FIG. 24, A_SN = 1, A_SN = 2, Command to switch audio of A_SN = 3. Specifically, the user commands audio switching from audio 2 with A_SN = 1 to audio 1 with A_SN = 2. The controller 34 accepts this audio switching for the audio stream # 1 from the user. In FIG. 29, the process of step S51 is a process that is executed in advance, and the process after step S52 is performed when the user issues a voice switching command for audio stream # 1.

  In step S53, the controller 34 acquires the audio stream number next to the audio stream number being reproduced. For example, if the audio stream file corresponding to the audio 2 with A_SN = 1 in FIG. 24 is played in the process of step S52, the controller 34 determines that the audio 1 with A_SN = 2 is next. Get the number of the corresponding audio stream.

  In step S54, the controller 34 determines whether or not it has a function of playing back an audio stream (audio stream # 1) corresponding to the acquired number. Specifically, the controller 34 itself reproduces the audio stream (audio stream # 1) corresponding to the acquired number based on the contents described in stream_attribute () (FIG. 16). It is determined whether or not it has a function to perform. For example, if type = 1 in Stream_entry () of FIG. 15, the controller 34 determines whether or not the playback device 20 has a function of playing back the audio stream of the main Clip AV Stream corresponding to the acquired number. To do. Since the audio stream corresponding to the acquired number is the audio stream # 1 decoded by the audio decoder 75-1, this process determines whether or not the audio decoder 75-1 can decode the audio stream # 1. In other words, it can be determined.

  If it is determined in step S54 that the audio stream corresponding to the acquired number is not reproduced, the process proceeds to step S55, and the controller 34 acquires the stream number next to the current stream number. That is, when the audio stream having the current stream number is not reproduced, the stream number is skipped (not subject to reproduction), and the next stream number is acquired. And after the process of step S55, a process returns to step S54 and the process after it is repeated. That is, the process is repeated until the controller 34 obtains the number of the audio stream (audio stream # 1) that has the function of reproducing itself.

  If it is determined in step S54 that the audio stream corresponding to the acquired number is played back, in step S56, the controller 34 checks the clip including the audio stream corresponding to the acquired number. For example, the controller 34 checks whether the audio stream corresponding to the acquired number is in the main clip (type = 1) or the sub clip (type = 2) based on the type in FIG. Further, for example, in the case of type = 1 in Stream_entry () in FIG. 15, the controller 34 confirms that the audio stream corresponding to the acquired number is in the main clip. Further, for example, when type = 2, the controller 34 confirms that the audio stream corresponding to the acquired number is in the sub clip.

  In step S57, the controller 34 specifies a desired audio stream. Specifically, a desired audio stream in the main clip or sub clip to which the stream corresponding to the acquired number belongs is specified. For example, when it is confirmed in step S54 that the audio stream corresponding to the acquired number is included in the main clip, the controller 34 stores the information (ref_to_stream_PID_of_mainClip) entered with type = 1 in FIG. Based on this, a predetermined audio stream in the main clip is specified. Also, for example, when it is confirmed in step S54 that the audio stream corresponding to the acquired number is included in the sub clip, the controller 34 receives the information (ref_to_SubPath_id, Based on ref_to_SubClip_entry_id and ref_to_stream_PID_of_SubClip), a predetermined audio stream in the sub Clip is specified. This identified audio stream becomes the audio stream # 1 decoded by the audio decoder 75-1.

  In step S58, the controller 34 instructs the storage drive 31 to read out the Clip AV Stream (main Clip AV Stream or sub Clip AV Stream) corresponding to the Clip in which the desired audio stream is multiplexed. Based on this instruction, the storage drive 31 reads the target Clip AV Stream. Specifically, the controller 34 instructs the storage drive 31 to read out the Clip AV Stream specified in the process of step S57.

  In step S59, the controller 34 instructs the AV decoder unit 33 to reproduce an audio stream from the read Clip AV Stream. Specifically, the controller 34 instructs the audio decoder 75-1 of the AV decoder unit 33 to decode a desired audio stream (that is, audio stream # 1) included in the read Clip AV Stream.

  In step S60, the AV decoder 33 decodes the audio stream (audio stream # 1) and outputs the audio. Specifically, the audio decoder 75-1 of the AV decoder unit 33 decodes the audio stream # 1 (the audio stream that is instructed to be read in the process of step S58), and outputs the decoded audio stream # 1 to the mixing processing unit 101. Here, when the audio stream # 2 is decoded by the audio decoder 75-2 in the process of FIG. 30 to be described later, the mixing processing unit 101 uses the decoded audio stream # 1 and audio stream # 2 as a result. Mixing and supplying to the subsequent mixing processing unit 97. On the other hand, when the audio stream # 2 has not been decoded by the audio decoder 75-2 in the processing of FIG. 30 described later, the mixing processing unit 101 uses the decoded audio stream # 1 as it is, and the subsequent mixing processing unit. 97. The mixing processing unit 97 decodes the audio data supplied from the mixing processing unit 101 (audio data obtained by decoding only the audio stream # 1 or audio stream # 1 and audio stream # 2 respectively) Data) and sound data output from the buffer 95 are mixed and output as an audio signal. A detailed example of the process in step S60 in FIG. 29 will be described later with reference to the flowchart in FIG.

  Next, with reference to the flowchart of FIG. 30, processing when the user instructs to switch audio for the audio stream # 2 will be described. This process is, for example, a process executed during the execution of the reproduction process of FIGS. In general, when the audio stream # 2 is reproduced, the audio stream # 1 is also reproduced. Therefore, the process of FIG. 30 is performed in parallel with the process of FIG. 29, for example. (When the audio stream # 2 is not selected by the user, the process of FIG. 30 is not executed, and only the process of FIG. 29 is executed).

  In step S71, the controller 34 obtains an order list of audio stream numbers (may be IDs). Specifically, the controller 34 refers to the STN_table () of the PlayItem described with reference to FIG. 13, and further changes the order of the audio stream numbers (ID) entered in the STN_table () described with reference to FIG. Get a list. This process is a process executed when the reproduction process of FIGS. 26 to 28 is started.

  When a voice switching command is issued by the user via the user interface, in step S72, the controller 34 receives a voice switching command for the audio stream # 2 from the user. At this time, the controller 34 may control a display unit (not shown) to display a stream number table as shown in FIG. 24 on the display unit. Since the process in FIG. 30 is a process in the case where the audio switching for the audio stream # 2 is instructed, for example, the user sets A_SN2 = 1 and A_SN2 = 2 in the stream number table as shown in FIG. Command to switch audio corresponding to. Specifically, the user commands audio switching from audio 4 with A_SN2 = 1 to audio 5 with A_SN2 = 2. The controller 34 accepts the audio switching for the audio stream # 2 from the user. In FIG. 30, the process of step S71 is a process that has been executed in advance, and the process after step S72 is performed when the user issues a voice switching command for audio stream # 2.

  In step S73, the controller 34 acquires the audio stream number next to the audio stream number being reproduced. For example, when an audio stream (Auxiliary audio stream) of SubClip_entry_id = 0 in FIG. 8 is being reproduced, the number of the audio stream file corresponding to the next SubClip_entry_id = 1 is acquired. For example, if the audio stream file corresponding to the audio 4 with A_SN2 = 1 in FIG. 24 is played in the process of step S72, the controller 34 changes the audio 5 to A5 because the next is the audio 5 with A_SN2 = 2. Get the number of the corresponding audio stream.

  In step S74, the controller 34 determines whether or not it has a function of playing back an audio stream (audio stream # 2) corresponding to the acquired number. Specifically, the controller 34 itself reproduces the audio stream (audio stream # 2) corresponding to the acquired number based on the contents described in stream_attribute () (FIG. 16). It is determined whether or not it has a function of For example, in the case of type = 1 in Stream_entry () in FIG. 15, the controller 34 determines whether or not the playback device 20 has a function of playing back the audio stream of the main clip corresponding to the acquired number. Since the audio stream corresponding to the acquired number is the audio stream # 2 decoded by the audio decoder 75-2, this process determines whether or not the audio decoder 75-2 can decode the audio stream # 2. In other words, it can be determined.

  When it is assumed that the audio stream # 1 decoded by the 1st audio decoder 75-1 is more scalable than the audio stream # 2 decoded by the 2nd audio decoder 75-2 and is a multistream. For example, when the 1st audio decoder 75-1 has a higher performance than the 2nd audio decoder 75-2 (audio stream # 1 is a normal movie sound, audio stream # 2 is a director's comment on the movie, etc. ), The processing of step S74 and step S75 may be omitted.

  Assuming that this is the case, the audio stream # 2 decoded by the 2nd audio decoder 75-2 is likely to be restricted to some extent so that the audio decoder 75-2 cannot decode it. This is because it is assumed that there is a high possibility that such a stream is not set as the audio stream # 2.

  If it is determined in step S74 that the audio stream corresponding to the acquired number is not reproduced, the process proceeds to step S75, and the controller 34 acquires the stream number next to the current stream number. That is, when the audio stream having the current stream number is not reproduced, the stream number is skipped (not subject to reproduction), and the next stream number is acquired. Then, after the process of step S75, the process returns to step S74, and the subsequent processes are repeated. That is, the process is repeated until the number of the audio stream (audio stream # 2) having the function of reproducing itself is acquired.

  If it is determined in step S74 that the audio stream corresponding to the acquired number is played back, in step S76, the controller 34 checks the clip including the audio stream corresponding to the acquired number. For example, the controller 34 checks whether the audio stream corresponding to the acquired number is in the main clip (type = 1) or the sub clip (type = 2) based on the type in FIG. In the case of type = 1 in the Stream_entry () of FIG. 15, the controller 34 confirms that the audio stream corresponding to the acquired number is in the main clip. In the case of type = 2 in the Stream_entry () of FIG. 15, the controller 34 confirms that the audio stream corresponding to the acquired number is in the sub clip.

  In step S77, the controller 34 specifies a desired audio stream. Specifically, a desired audio stream in the main clip or sub clip to which the stream corresponding to the acquired number belongs is specified. For example, when it is confirmed in step S74 that the audio stream corresponding to the acquired number is included in the main clip, the controller 34 stores the information (ref_to_stream_PID_of_mainClip) entered with type = 1 in FIG. Based on this, a predetermined audio stream in the main clip is specified. Also, for example, when it is confirmed in step S74 that the audio stream corresponding to the acquired number is included in the sub clip, the controller 34 receives the information (ref_to_SubPath_id, Based on ref_to_SubClip_entry_id and ref_to_stream_PID_of_SubClip), a predetermined audio stream in the sub Clip is specified. This identified audio stream becomes the audio stream # 2 decoded by the audio decoder 75-1.

  In step S78, the controller 34 instructs the storage drive 31 to read out the Clip AV Stream (main Clip AV Stream or sub Clip AV Stream) corresponding to the Clip in which the desired audio stream is multiplexed. Based on this instruction, the storage drive 31 reads the target Clip AV Stream. Specifically, the controller 34 instructs the storage drive 31 to read out the Clip AV Stream specified in the process of step S77.

  In step S79, the controller 34 instructs the AV decoder unit 33 to reproduce the audio stream from the read Clip AV Stream. Specifically, the controller 34 instructs the audio decoder 75-2 of the AV decoder unit 33 to decode a desired audio stream (that is, audio stream # 2) included in the read Clip AV Stream.

  In step S80, the AV decoder 33 decodes the audio stream (audio stream # 2) and outputs the audio. Specifically, the audio decoder 75-2 of the AV decoder unit 33 decodes the audio stream # 2 (the audio stream that is instructed to be read in the process of step S78), and outputs the decoded audio stream # 2 to the mixing processing unit 101. In the process of FIG. 30, the audio stream # 2 is decoded by the audio decoder 75-2. This means that the audio stream # 1 is also decoded by the audio decoder 75-1 in the process of FIG. Since the stream # 1 is output to the mixing processing unit 101, the mixing processing unit 101 mixes the decoded audio stream # 1 and audio stream # 2, and supplies the mixed audio stream # 1 to the subsequent mixing processing unit 97. The mixing processing unit 97 receives the audio data supplied from the mixing processing unit 101 (audio data obtained by decoding the audio stream # 1 and the audio stream # 2 respectively) and the sound data output from the buffer 95. , Mixing processing and output as an audio signal. A detailed example of the process in step S80 in FIG. 30 (and step S60 in FIG. 20) will be described with reference to the flowchart in FIG.

  FIG. 31 is a flowchart illustrating an example of processing for outputting an audio stream in the AV decoder unit 33. That is, FIG. 31 is a flowchart for explaining a detailed example of the process of step S60 of FIG. 29 and the process of step S80 of FIG.

  In step S <b> 81, the AV decoder unit 33 determines whether there is one audio stream to be reproduced based on the control from the controller 34. Specifically, the AV decoder unit 33 determines whether or not the audio stream # 1 and the audio stream # 2 are selected by the user based on the control from the controller 34. For example, when the user instructs switching between the audio stream # 1 and the audio stream # 2 by the process of step S52 of FIG. 29 and the process of step S72 of FIG. 30, or the audio stream # 1 and the audio stream # 2 are designated by the user. When the playback is instructed, the AV decoder unit 33 determines that the number of audio streams to be played back is not one, that is, two. Further, for example, when the user instructs the reproduction of the audio stream # 1 or when the user instructs the switching of the audio stream # 1 in the process of step S52 of FIG. 29, the user selects the audio stream # 2. Is not instructed to reproduce, the AV decoder 33 determines that there is one audio stream to be reproduced.

  If it is determined in step S81 that the number of audio streams to be reproduced is one, in step S82, the 1st audio decoder 75-1 decodes the one audio stream to be reproduced, that is, audio stream # 1. For example, when the audio stream to be played is a main Clip AV Stream audio stream, the main Clip AV Stream audio stream selected and supplied by the PID filter 55 is selected by the switch 59, and the 1st audio decoder 75-1. Therefore, the 1st audio decoder 75-1 decodes the audio stream of the main Clip AV Stream (here, the audio stream # 1). The 1st audio decoder 75-1 supplies the decoded audio stream # 1 to the mixing processing unit 101 at the subsequent stage. The mixing processing unit 101 outputs the supplied audio data as it is to the subsequent mixing processing unit 97.

  If it is determined in step S81 that there is not one audio stream to be played back, that is, if it is determined that there are two audio streams to be played back, audio stream # 1 is the 1st audio decoder of the two audio streams. 75-1 and the audio stream # 2 is supplied to the 2nd audio decoder 75-2. In other words, the switch 59 selects the switch to supply the audio stream # 1 supplied via the PID filter 55 or the PID filter 56 to the audio decoder 75-1, and supplies the audio stream # 1 via the PID filter 55 or the PID filter 56. The switch is selected to supply the audio stream # 2 thus transmitted to the audio decoder 75-2.

  Therefore, in step S83, the 1st audio decoder 75-1 decodes the audio stream # 1. For example, when the audio stream # 1 to be reproduced is a main Clip AV Stream audio stream, the main Clip AV Stream audio stream selected and supplied by the PID filter 55 is selected by the switch 59, and the 1st audio decoder 75 is selected. The first audio decoder 75-1 decodes the main clip audio stream (audio stream # 1). The 1st audio decoder 75-1 supplies the decoded audio stream # 1 to the mixing processing unit 101 at the subsequent stage.

  In step S84, the 2nd audio decoder 75-2 decodes the audio stream # 2. For example, when the audio stream # 2 to be reproduced is a sub-Clip AV Stream audio stream, the sub-Clip AV Stream audio stream selected and supplied by the PID filter 56 is selected by the switch 59 and the 2nd audio decoder 75. 2nd audio decoder 75-2 decodes the audio stream (audio stream # 2) of the sub Clip AV Stream. The 2nd audio decoder 75-2 supplies the decoded audio stream # 2 to the mixing processing unit 101 at the subsequent stage. Note that the processes of step S83 and step S84 may be executed in parallel or in reverse order.

  In step S85, the mixing processing unit 101 mixes the audio stream # 1 decoded by the 1st audio decoder 75-1 and the audio stream # 2 decoded by the 2nd audio decoder 75-2. That is, the mixing processing unit 101 mixes (synthesizes and superimposes) the audio data supplied from the 1st audio decoder 75-1 and the audio data supplied from the 2nd audio decoder 75-2, thereby obtaining one audio data. And The mixing processing unit 101 supplies the audio data obtained as a result of mixing to the mixing processing unit 97 at the subsequent stage.

  After the process of step S82 or the process of step S85, the process proceeds to step S86, and the mixing processing unit 97 mixes the supplied audio data with the sound data supplied from the buffer 95, and Output. For example, when the process of step S86 is after the process of step S82, the mixing processing unit 97 mixes the audio data obtained by decoding the audio stream # 1 and the sound data. For example, when the process of step S86 is after the process of step S85, the mixing processing unit 97 mixes the audio data obtained by decoding the audio stream # 1 and the audio data obtained by decoding the audio stream # 2. Sound data is mixed with the recorded audio data. Thereafter, the process ends.

  According to the process of FIG. 31, two audio streams can be mixed and reproduced.

  31 determines the selection of the switch 59 in FIG. 25 in step S21 in FIG. That is, when there is one audio stream to be reproduced in FIG. 31, the switch 59 selects to supply the audio stream supplied from the PID filter 55 or PID filter 56 to the 1st audio decoder 75-1 (FIG. 31). If the number of audio streams to be reproduced is two, the switch 59 selects to supply the audio stream supplied from the PID filter 55 or PID filter 56 to the 2nd audio decoder 75-2 (FIG. 31). YES in step S81).

  As described above, the controller 34 can control switching of audio (audio) based on STN_table () of PlayItem. In addition, the controller 34 can control playback switching by selecting only a stream having a playback function by referring to the stream_attribute of the STN_table (). Furthermore, the controller 34 can control to superimpose and reproduce two audio streams based on STN_table (). The controller 34 mixes and reproduces one main audio stream # 1 and an audio stream # 2 that is the same type of stream (audio stream) that is reproduced in synchronization with the audio stream # 1. Can be controlled.

  In the process of FIG. 29, the audio is switched based on the audio stream number, but the audio may be switched based on the audio stream ID (audio_stream_id). In this case, the audio stream ID is obtained by subtracting 1 from the audio stream number.

  In the above example, an example in which two audio streams are mixed (synthesized) and reproduced has been described. However, the present invention is not limited to an audio stream, and the same applies to various stream files as long as the stream types are the same. Can be applied. For example, this configuration can also be applied to the case of picture-in-picture display in which two video streams (streams whose stream types are video streams) are combined and played back. As described above, according to the present invention, two stream files of the same type can be synthesized and reproduced.

  For example, an AV stream video stream (video stream # 1) referred to in the main path is displayed as the main screen, and a video stream (video stream) referred to in the sub-path (or main path) video path is displayed in the main screen. It can also be used for so-called PinP (picture-in-picture) display in which # 2) is combined and displayed as a sub-screen.

  As described above, when audio or subtitles are prepared in a stream or data file different from the main AV stream, the PlayList has a main path and sub-path, so the user can perform operations such as audio switching and subtitle switching. However, it can be selected from a different stream or a different data file from the main AV stream.

  In addition, the Stream Number Definition Table that defines the type of the data multiplexed in the AV stream file and the data referenced by the Sub Path is provided in the PlayItem of the main path. A high stream can be realized.

  Furthermore, the playback device 20 can sequentially select and play back only the streams corresponding to the functions of itself by referring to the stream_attribute of the STN_table ().

  The above processing is summarized as follows.

  The playback device 20 has a main path that is a main playback path indicating the position of the main Clip AV stream file including at least one stream, and each of the sub Clip AV stream files that are played back in accordance with the timing of the main Clip AV stream file. A PlayList is obtained as reproduction management information composed of a plurality of sub-paths indicating positions. Then, the playback device 20 accepts selection of a stream to be played based on STN_table () for selecting a stream to be played back included in the PlayList. STN_table () is a predetermined type of stream (for example, audio stream # 1) of the main Clip AV stream file and the same type of stream file (for example, audio stream #) that is played in accordance with the playback timing of the stream. 2), the selection of the stream to be played back is accepted based on this STN_table () (STN_table () indicated by PlayItem).

  Here, when two streams of the same type (audio stream) are selected as playback streams, the playback device 20 reads the two selected stream files, and the mixing processing unit 101 of the playback device 20 Two audio streams (decoded audio streams) are synthesized, and the mixing processing unit 97 (AV decoder unit 33) reproduces (outputs) the synthesized stream.

  As a result, as shown in FIG. 32, when two of audio 1 with A_SN = 2 and audio 5 with A_SN2 = 2 are selected by the user as a stream to be played (when switched and selected), audio 1 and The audio 5 can be mixed (synthesized) and reproduced.

  In addition, since the PlayList includes the Main Path and the Sub Path and refers to the Clip having a different Main Path and Sub Path, the stream can have extensibility. Further, since a plurality of files can be referred to by one Sub Path (for example, FIG. 8), the user can select from a plurality of different streams.

  Further, in the Play Item of the Main Path, attached data (for example, an audio stream included in the AV stream file referenced by the Main Path) multiplexed (included) in the AV stream file referenced by the Main Path, and Since STN_table () in FIG. 14 is provided as a table for defining auxiliary data referred to by Sub Path, a stream with higher expandability can be realized. In addition, Sub Path can be easily extended by entering STN_table ().

  In addition, since stream_attribute () of FIG. 16 that is stream attribute information is provided in STN_table (), the playback apparatus 20 can determine whether or not the selected stream can be played back. Furthermore, by referring to stream_attribute (), it is possible to select and reproduce only a stream having a reproduction function.

  Furthermore, in STN_table () (FIG. 14), two same type stream files (here, audio stream files) are defined, and two audio decoders (1st audio decoder 75) are added to the playback apparatus 20 of FIG. -1 and 2nd audio decoder 75-2) and a mixing processing unit 101 for synthesizing (mixing) audio data decoded by the two audio decoders are provided. Can be played at the same time.

  Further, as shown in FIG. 11, the Sub Path includes a SubPath_type that indicates the type of Sub Path (type such as audio or text subtitle), a Clip_Information_file_name in FIG. 12 includes the SubPlayItem_IN_time in FIG. 12 indicating the clip In point and the SubPlayItem_OUT_time in FIG. 12 indicating the Out point, so that the data referred to by the Sub Path can be accurately identified.

  Furthermore, in the Sub Path, sync_PlayItem_id in FIG. 12 (for example, sync_PlayItem_id in FIGS. 7 and 8), which is designation information for designating an AV stream file on the Main Path for reproducing the Main Path at the same timing as the Sub Path. And sync_start_PTS_of_PlayItem (for example, sync_start_PTS_of_PlayItem in FIGS. 7 and 8) that is the time on the main path at which the in point of the data referred to by the sub path starts synchronously on the time axis of the main path. 7 and FIG. 8, the data (file) referred to by the Sub Path can be reproduced in synchronization with the main ClipAV stream file referred to by the Main Path.

  Note that the data read by the storage drive 31 in FIG. 25 may be data recorded on a recording medium such as a DVD (Digital Versatile Disc), or data recorded on a hard disk, Data downloaded via a network (not shown) may be used, or data obtained by combining these may be used. For example, playback may be performed based on a clip including a PlayList and a sub Clip AV Stream downloaded and recorded on a hard disk, and a Clip including a main Clip AV Stream recorded on a DVD. Also, for example, when a PlayList that recognizes a clip recorded on a DVD as a clip including a sub clip AV stream and a clip including a main clip AV stream are recorded on the hard disk, the clip is recorded on the hard disk. Based on the PlayList, the main Clip AV Stream and the sub Clip AV Stream may be read from the hard disk and the DVD and reproduced.

  Further, when two audio streams are mixed and reproduced, the 1st audio decoder 75-1 of the reproduction apparatus 20 decodes the audio stream # 1, and the 2nd audio decoder 75-2 decodes the audio stream # 2. Further, whether or not the codec (encoding) system is the same (for example, Dolby AC-3 audio in FIG. 17) may be further determined by the processing in step S54 in FIG. 29 and step S74 in FIG. This is because even if the 1st audio decoder 75-1 and the 2nd audio decoder 75-2 are in the middle of decoding the audio stream, if the codec system is the same, the data system in the middle of decoding is the same. This is because the mixing processing unit 101 can perform mixing.

  In the above example, the audio stream # 1 decoded by the 1st audio decoder 75-1 and the audio stream # 2 decoded by the 2nd audio decoder 75-2 of the playback apparatus 20 in FIG. However, when a combination of the audio stream # 1 and the audio stream # 2 is defined and mixed reproduction is performed, the user may select a combination from the combinations. That is, a combination of the audio stream # 1 and the audio stream # 2 that can be mixed and reproduced may be defined. Two examples of this case will be described with reference to FIGS.

  First, the first example will be described. FIG. 33 is a diagram illustrating a second example (second STN_table ()) of the syntax of STN_table () when a combination of the audio stream # 1 and the audio stream # 2 is defined. In addition, about the part same as FIG. 14 in the figure, since it becomes repetition, the description is abbreviate | omitted.

  In FIG. 33, audio_stream_id2 is given from 0 to one audio elementary stream identified for each stream_entry () in order in the for loop of audio stream ID2 (audio_stream_id2). At this time, Combination_of_Primary_and_Secondary is newly defined for audio_stream_id2 in addition to stream_entry () and stream_attribute (). Combination_of_Primary_and_Secondary is information defining a combination of audio stream # 1 (Primary) and audio stream # 2 (Secondary). That is, information indicating which audio stream # 1 can be combined with the audio stream # 2 specified by the audio stream ID2 (audio_stream_id2) is defined in the for loop of the audio stream ID2. In other words, the audio stream # 2 that can be combined with the audio stream # 1 is defined by Combination_of_Primary_and_Secondary. Thus, instead of defining the audio stream ID2 to be combined in the for loop of the audio stream ID1, the audio stream ID1 and the audio stream ID2 are individually defined, and the combination is defined in the for loop of the audio stream ID2. Thus, even when the registerable number of the audio stream ID1 is determined, the registerable number can be used effectively by not counting the combination registration as one.

  As described with reference to FIG. 14, audio stream number 2 (audio_stream_number2) may be used instead of audio stream ID2 (audio_stream_id2). In this case, audio_stream_number2 is given from 1 instead of 0. That is, audio_stream_number2 is obtained by adding 1 to the value of audio_stream_id2. The audio stream number 2 is an audio stream number 2 that can be seen by the user and used for audio switching.

  Here, an example of the bitmap syntax of Combination_of_Primary_and_Secondary in FIG. 33 will be described with reference to FIG.

  In the example of FIG. 34, the bitmap syntax of Combination_of_Primary_and_Secondary is 32 bits. 34, audio_stream_id indicates audio_stream_id (audio stream ID) defined in the second STN_table () of FIG. 33, and the value of the indicator is the audio stream # 2 of the corresponding audio_stream_id2 (audio stream ID2). This is a flag indicating audio_stream_id for specifying a combinable audio stream # 1. That is, the audio_stream_id of the audio stream # 1 that can be combined with the audio stream # 2 given by the audio_stream_id2 can be specified by the audio_stream_id and the indicator.

  That is, as shown in FIG. 37, Combination_of_Primary_and_Secondary is defined in the for_loop of audio_stream_id2, there is a combination_of_Primary_and_Secondary for each audio_stream_id2, and a combination is possible for audio stream # 2 specified by a specific audio_stream_id2 Audio stream # 1 is specified by audio_stream_id indicated by the indicator.

  Specifically, when the value of the Indicator is 0, it indicates that the audio_stream_id2 does not correspond to the audio stream # 1 specified by the audio_stream_id. When the value of the indicator is 1, the audio_stream_id2 is specified by the audio_stream_id It corresponds to the audio stream # 1.

  That is, in the case of the example in FIG. 34, audio stream # 2 specified by audio_stream_id2 (audio stream ID 2) is combined with two audio streams (audio stream # 1) specified by audio_stream_id = 0 and audio_stream_id = 1. It has been shown to be possible.

  In FIG. 34, audio stream # 1 that can be combined is specified by audio_stream_id, but an audio stream number may be used instead of audio_stream_id. In this case, audio_stream_id is given from 1 instead of 0.

  As shown in FIG. 34, Combination_of_Primary_and_Secondary is defined with 32-bit bitmap syntax, so that it is possible to define audio streams # 2 that can be combined with 32 audio streams # 1.

  According to FIGS. 33 and 34, a combination of audio stream # 1 and audio stream # 2 can be defined using audio_stream_id, audio_stream_id2, and Combination_of_Primary_and_Secondary. That is, since it is not necessary to define a contradictory combination of audio streams, the user can always select reproduction of a reproducible combination of audio streams.

  A specific example will be described with reference to FIG. FIG. 35 is a diagram illustrating an example of a stream number table representing a relationship between an audio signal provided to a user and a caption signal. In the figure, the description of the caption signal (sub-picture) is the same as in FIG.

  In FIG. 35, A_SN is given to each of the audio streams # 1 (audio streams entered with audio_stream_id) entered in PlayItem STN_table () constituting the Main Path of the PlayList.

  Specifically, audio 2 is given to A_SN = 1, audio 1 is given to A_SN = 2, and audio 3 is given to A_SN = 3.

  At this time, audio 4 (audio_stream_id2 = 0, that is, audio 4 with audio stream number 2 being 1) can be combined with audio 2 with A_SN = 1. For audio 1 with A_SN = 2, audio 4 (audio_stream_id2 = 0, ie, audio 4 with audio stream number 2 being 1) and audio 5 (audio_stream_id2 = 1, ie, audio stream number 2 being 2) It can be combined with some audio 5).

  Specifically, in the case of audio 4 (audio 4 with audio_stream_id2 = 0), the combination_of_Primary_and_Secondary bitmap syntax in FIG. The values of the indicators up to = 31 are all 0. Further, in the case of audio 5 (audio 5 with audio_stream_id2 = 1), the bitmap syntax of Combination_of_Primary_and_Secondary in FIG. The values of the indicators up to are all 0.

  The user selects the audio stream # 1 to be reproduced from the audio streams to which A_SN is given, and the audio stream # 2 to be mixed with the selected audio stream # 1 is an audio stream to which A_SN is given. Select from the ones displayed on the right side. In the example of FIG. 35, when audio 2 (A_SN = 1) is selected as audio stream # 1, audio stream # 2 that can be mixed is only audio 4 displayed on the right side of audio 2 in the drawing. Therefore, it is selected whether or not to perform mixing reproduction. In FIG. 35, the audio 4 corresponding to the audio 2 with A_SN = 1 is “x”, that is, the state in which mixing reproduction is not selected. The user can set the state where mixing reproduction of the audio 2 and the audio 4 is selected by switching the “x” to “◯”.

  Also, for example, in the case of the example of FIG. 35, when audio 1 (A_SN = 2) is selected as audio stream # 1, audio stream # 2 that can be mixed is audio displayed on the right side of audio 2 in the figure. 4 and audio 5 can be selected. Therefore, the user selects with which audio stream # 2 the audio stream # 1 is to be reproduced. In FIG. 35, the audio 4 and the audio 5 corresponding to the audio 1 with A_SN = 2 are “x”, that is, the state where the mixing reproduction is not selected. The user can set the state of selecting the audio 2 and audio 4 or the audio 2 and audio 5 mixing reproduction by switching the “x” to “o”.

  As a specific example of this playback selection, when audio 2 with A_SN = 1 is selected and the user commands switching of audio, a combination of audio 2 with audio 4 and audio 4 is selected. When the playback is switched to the audio (the audio 4 corresponding to the audio 2 in FIG. 35 becomes “◯”) and the user instructs to switch the audio further, the audio is reproduced from the audio 1 of A_SN = 2 (in FIG. 35). When the audio 4 corresponding to the audio 2 is switched to “x”) and the user further instructs to switch the audio, the audio is reproduced by the combination of the audio 1 and the audio 4 of A_SN = 2 (FIG. 35). The audio 4 corresponding to the audio 1 is switched to “◯”. Further, when the user instructs to switch the audio further, the audio is reproduced as a combination of the audio 1 and the audio 5 of A_SN = 2 (the audio 4 corresponding to the audio 1 in FIG. Becomes “○”.

  The user can switch the selection of the audio stream to be reproduced in this way, and select reproduction of a combination of audio 1 and audio 5 of A_SN2 as shown in FIG. 36, for example. Thereby, audio 1 (audio stream # 1) and audio 5 (audio stream # 2) can be mixed and reproduced.

  Thus, by defining the combination of audio stream # 1 and audio stream # 2 using audio_stream_id and audio_stream_id2 and Combination_of_Primary_and_Secondary, it is not necessary to define conflicting combinations, and the user can , A combination of audio streams that can always be played back can be selected. That is, in the example of FIG. 35, only the selectable ones can be provided to the user by not defining the audio 2 + audio 5 that is a combination of contradictory audio streams.

  In other words, in the second STN table () (FIG. 33) in PlayItem (), the audio stream # 1 decoded by the 1st audio decoder 75-1 in FIG. 25 and the 2nd audio decoder 75-2 are decoded. Since the combination with the audio stream # 2 is defined using Combination_of_Primary_and_Secondary, the two audio streams can be mixed and reproduced. That is, streams of the same type (in this example, an audio stream) can be mixed (superimposed or combined) from a plurality of types of streams and played back simultaneously.

  With this configuration, the user does not need to individually select two audio streams to be mixed, and can select an audio stream to be reproduced from a combination of the two audio streams. .

  Next, when a combination of audio stream # 1 and audio stream # 2 is defined and mixed reproduction is performed, that is, the user is allowed to select reproduction from a combination of audio stream # 1 and audio stream # 2. A second example of the case will be described with reference to FIG. In other words, FIG. 37 is a second example in the case where an audio stream that can be mixed with audio stream # 1 is defined as audio stream # 2.

  FIG. 37 is a diagram illustrating a third example (third STN_table ()) of syntax of STN_table () when defining a combination of audio stream # 1 and audio stream # 2. In addition, about the part same as FIG. 14 in the figure, since it becomes repetition, the description is abbreviate | omitted.

  In FIG. 37, audio_stream_id2 is given from 0 to one audio elementary stream identified for each stream_entry () in order in the for loop of audio stream ID2 (audio_stream_id2). At this time, number_of_combinations is given to audio_stream_id2, and in the for statement after number_of_combinations, audio_stream_id is given by the number of number_of_combinations. The for statement after number_of_combinations and number_of_combinations is information defining the combination of audio stream # 1 and audio stream # 2, and the number of audio streams # 1 that can be combined with audio stream # 2 specified by audio_stream_id2 is , Number_of_combinations. Then, audio_stream_id that identifies the audio stream # 1 that can be combined with the audio stream # 2 identified by audio_stream_id2 is defined in a for statement after number_of_combinations.

  That is, unlike FIG. 33, in the for statement of the audio stream ID2 (audio_stream_id2) of the third STN_table () of FIG. 37, the audio stream # 2 is compared with the audio stream # 2 specified by the audio stream ID2. The number that can be combined with 1 is defined by number_of_combinations, and the audio_stream_id that identifies the target audio stream # 1 is directly defined in the for statement after number_of_combinations.

  As described above, in the third STN_table (), information indicating which audio stream # 1 can be combined with the audio stream # 2 specified by the audio stream ID2 (audio_stream_id2) is the audio stream ID2. Defined in the for statement after number_of_combinations. That is, the audio stream # 2 that can be combined with the audio stream # 1 is defined by the for statement after the number_of_combinations of the audio stream ID2.

  As described with reference to FIG. 14, audio stream number 2 (audio_stream_number2) may be used instead of audio stream ID2 (audio_stream_id2). Also, an audio stream number (audio_stream_number) may be used instead of the audio stream ID (audio_stream_id). That is, audio_stream_id defined in the for statement after number_of_combinations may be an audio stream number.

  According to FIG. 37, the combination of audio stream # 1 and audio stream # 2 can be defined using number_of_combinations and the subsequent for statement. That is, since it is not necessary to define a contradictory combination of audio streams, the user can always select reproduction of a reproducible combination of audio streams.

  A specific example of the stream number table in this case is the same as that described with reference to FIGS.

  That is, when FIG. 35 corresponds to the third STN_table () of FIG. 37, audio stream # 1 that can be combined with audio 4 (audio_stream_id2 = 0, that is, audio 4 with audio stream number 2 being 1). As described above, audio 2 (audio_stream_id = 0) with A_SN = 1 and audio 1 (audio_stream_id = 1) with A_SN = 2 are defined. In this case, the number of number_of_combinations corresponding to audio_stream_id2 = 0 in FIG. 37 is two, and the defined audio_stream_id is audio_stream_id = 0 and audio_stream_id = 1. Also, for audio 5 (audio_stream_id2 = 1, ie, audio 5 with audio stream number 2 being 2), audio 1 (audio_stream_id = 1) with A_SN = 2 is defined as an audio stream # 1 that can be combined. Will be. In this case, the number_of_combinations corresponding to audio_stream_id2 = 1 in FIG. 37 is one, and the defined audio_stream_id is audio_stream_id = 1.

  The user switches the audio reproduction selection shown in FIG. 35, and selects the reproduction of the combination of audio 1 and audio 5 with A_SN = 2 as shown in FIG. 36, for example. Thereby, audio 1 (audio stream # 1) and audio 5 (audio stream # 2) can be mixed and reproduced.

  In this way, by defining the combination of audio stream # 1 and audio stream # 2 using number_of_combinations in FIG. 37 and the subsequent for statement, it is not necessary to define contradictory combinations. The user can select a combination of audio streams that can always be reproduced. That is, in the example of FIG. 35, only the selectable ones can be provided to the user by not defining the audio 2 + audio 5 that is a combination of contradictory audio streams.

  In other words, in STN table () (FIG. 37) in PlayItem (), audio stream # 1 decoded by 1st audio decoder 75-1 in FIG. 25 and audio stream # decoded by 2nd audio decoder 75-2. Since the combination with 2 is defined by using number_of_combinations and the subsequent for statement, it is possible to mix and reproduce the two combinable audio streams. That is, streams of the same type (in this example, an audio stream) can be mixed (superimposed or combined) from a plurality of types of streams and played back simultaneously.

  Further, by using the third STN_table (), the user does not need to individually select two audio streams to be mixed, and can select an audio stream to be reproduced from a combination of the two audio streams. Can do.

  Next, FIG. 38 is a diagram illustrating a fourth example (fourth STN_table ()) of the syntax of STN_table () when defining a combination of audio stream # 1 and audio stream # 2. In the figure, the same parts as those in FIGS. 14 and 37 are repeated, and the description thereof is omitted.

  In the for loop of the audio stream ID (audio_stream_id) in the example of FIG. 38, audio_stream_id is assigned from 0 to one audio stream # 1 that is specified for each stream_entry () in order, and is arranged thereafter. The stream_attribute () is the same as the example in FIG.

  However, the following information defining the combination of the audio stream # 1 and the audio stream # 2 is further included in the for loop of the audio stream ID (audio_stream_id) in the example of FIG. That is, if number_of_audio_stream2_entries is not 0 (if (number_of_audio_stream2_entries! = 0)), number_of_audio_stream2_ref_entries is given. number_of_audio_stream2_ref_entries indicates the number of audio streams # 2 that can be combined with the audio stream # 1 specified by audio_stream_id. Then, audio_stream2_id_ref for specifying an audio stream # 2 that can be combined with the audio stream # 1 specified by audio_stream_id is defined by a for statement after number_of_audio_stream2_ref_entries.

  As in the other examples, the audio stream number 2ref (audio_stream_number2_ref) may be used instead of the audio stream ID2ref (audio_stream_id2_ref). Also, an audio stream number (audio_stream_number) may be used instead of the audio stream ID (audio_stream_id).

  As described above, the combination of the audio stream # 1 and the audio stream # 2 can also be defined using the STN_table () of the fourth example described with reference to FIG. That is, since it is not necessary to define a contradictory combination of audio streams, the user can always select reproduction of a reproducible combination of audio streams.

  By the way, as described above, the application format on the recording medium mounted on the playback apparatus of the present invention has two layers, PlayList and Clip. A PlayList is composed of a main path (Main Path) created by a sequence of one or more PlayItems (by consecutive PlayItems) and a parallel (in parallel) to the Main Path (by a sequence of one or more SubPlayItems (non- One or more sub paths (sub paths) created by sub play items that may be continuous or may be continuous may be included.

  Therefore, for example, as shown in FIG. 39, an AV stream (a video stream 311 described as Primary Video in FIG. 39 and a primary audio) is displayed on the entire screen 301 of one screen. The AV stream (described as Secondary Video in FIG. 39) is reproduced on the sub-screen 302 arranged on the full screen 301. A video stream 313, and a movie director corresponding to an audio stream 314) described as Secondary Audio, or the like, such as playing a content other than the AV stream referenced in the main path. It becomes possible to apply.

  In other words, it is possible to apply a technique in which different contents are simultaneously reproduced on the full screen (parent screen) 301 and the child screen 302, respectively.

  Such a technique is called a picture-in-picture technique.

  Note that the reproduction of the audio corresponding to the audio stream 312 described as Primary Audio and the audio corresponding to the audio stream 314 described as Secondary Audio is performed on these two audio streams as described above. This is the sound output obtained as a result of the mixing process.

  In addition, as described above, the actual reproduction target is audio or an image corresponding to each stream. However, in the following description, there are cases where each stream is used for the sake of simplicity. That is, an expression such as playing a predetermined stream may be used.

  In order to realize an application to which the picture-in-picture method is applied, for example, the following syntax is required. That is, for example, a video stream specified by video_stream_id in the STN_definition table in the examples of FIGS. 40 and 41 (hereinafter referred to as a primary video stream. For example, in the example of FIG. 39, a video stream 311 described as Primary Video is described. Is called a primary video stream 311). In view of this, if there is a Syntax that defines a combination of streams that can be reproduced simultaneously with the primary video stream, it is possible to realize a picture-in-picture technique, that is, an application that applies picture-in-picture. .

  Here, for example, in the present embodiment, the following first to fourth types of streams are prepared as streams that can be reproduced simultaneously with the primary video stream by the picture-in-picture method.

  The first type of stream is an audio stream specified by audio_stream_id in the STN_definition table in the examples of FIGS. 40 and 41, that is, the above-described audio stream # 1 (hereinafter referred to as a primary audio stream. For example, FIG. 39 In this example, the audio stream 312 described as Primary Audio is referred to as a primary audio stream 312).

  The second type of stream is a video stream specified by video_stream_id2 in the STN_definition table of the example of FIGS. 40 and 41 (hereinafter referred to as a secondary video stream. For example, in the example of FIG. 39, it is described as Secondary Video. The video stream 313 is referred to as a secondary video stream 313).

  The third type of stream is an audio stream specified by audio_stream_id2 in the STN_definition table in the examples of FIGS. 40 and 41, that is, the above-described audio stream # 2 (hereinafter referred to as a secondary audio stream. For example, FIG. In the example, the audio stream 314 described as Secondary Audio is referred to as a secondary audio stream 314).

  The fourth type of stream is a text subtitle stream (txtST) referenced in a sub-path, a stream (PG, Presentation Graphics stream) obtained by run-length encoding a bitmap subtitle such as a DVD sub-picture, and the like. Say. For example, in the example of FIG. 39, the stream 315 described as Subtitle is an example of the fourth type of stream. Hereinafter, the fourth type of stream, that is, txtST, PG, and the like are collectively referred to as a subtitle stream. Note that the playback position of the subtitle stream (the display position of subtitles and the like) does not have to be in the child screen, and may be any position on the entire screen. For this reason, in the example of FIG. 39, the subtitle stream 315 is drawn so as to protrude from the child screen 302.

  As described above, in order to realize an application to which the picture-in-picture method is applied, for example, the definition of the combination of streams that can be simultaneously reproduced in combination with the primary video stream, that is, the secondary video stream, the primary audio stream, the secondary It is necessary to define a combination of an audio stream and a subtitle stream.

  Furthermore, by defining a plurality of combinations, not only switching of the primary video stream and primary audio stream reproduced on the full screen 301 but also switching of the secondary video stream reproduced on the sub-screen 302 and switching of the subtitle stream are performed. Can also be easily performed. However, an example of the switching operation will be described later with reference to FIGS. 42 and 43.

  The definition of the combination of streams that can be simultaneously reproduced in combination with the primary video stream can be performed in STN_table as shown in FIGS. 40 and 41, for example. That is, FIG. 40 and FIG. 41 show the syntax of STN_table () when defining a combination of a secondary video stream, a primary audio stream, a secondary audio stream, and a subtitle stream that can be simultaneously reproduced in combination with the primary video stream. It is a figure which shows the 5th example (5th STN_table ()). In the figure, the same parts as those in FIG. 37 described above are repeated, and the description thereof is omitted.

  In the example of FIGS. 40 and 41, the combination of the secondary video stream, the primary audio stream, the secondary audio stream, and the subtitle stream that can be simultaneously reproduced in combination with the primary video stream is defined as follows. That is, first, one or more secondary video streams that can be simultaneously reproduced in combination with the primary video stream are defined. Then, for each of the one or more secondary video streams, an audio stream (primary audio stream and secondary audio stream) and a subtitle stream that can be reproduced simultaneously are defined.

  Specifically, in FIG. 40, number_of_video_stream2_entries indicates the number of streams to which video_stream_id2 entered (registered) in STN_table () is given. video_stream_id2 is information for identifying a secondary video stream, and video_stream_number2 is a secondary video stream number that is used for video switching and is visible to the user.

  In FIG. 41, in a for loop of video stream ID2 (video_stream_id2), video_stream_id2 from 0 to one video elementary stream (video elementary stream to be a secondary video stream) specified for each stream_entry () in order. Given.

  At this time, number_of_Audio_combinations_for_video2 is given to video_stream_id2, and in the subsequent for statement, audio_stream_id and audio_stream_id2 are given by the number of number_of_Audio_combinations_for_video2. number_of_Audio_combinations_for_video 2 and the subsequent for statement are information defining a combination of audio streams that are simultaneously played with the secondary video stream, that is, a combination of a primary audio stream specified by audio_stream_id and a secondary audio stream specified by audio_stream_id2 It is. The number of sets of audio streams (a set of primary audio streams and secondary audio streams) that can be combined with the secondary video stream specified by video_stream_id2 is number_of_Audio_combinations_for_video2. Then, as a set of audio streams that can be combined with the secondary video stream identified by video_stream_id2, audio_stream_id that identifies the primary audio stream and audio_stream_id2 that identifies the secondary audio stream are defined in a for statement after number_of_Audio_combinations_for_video2, respectively. .

  Also, number_of_Subtitle_combinations_for_video2 is given to the same video_stream_id2, and PG_textST_stream_id is given to the same video_stream_id2 by the number of number_of_Subtitle_combinations_for_video2. The number_of_Subtitle_combinations_for_video2 and the subsequent for sentence are information defining a combination of subtitle streams that are reproduced simultaneously with the secondary video stream, that is, a combination of subtitle streams specified by PG_textST_stream_id. The number of subtitle streams that can be combined with the secondary video stream specified by video_stream_id2 is number_of_Subtitle_combinations_for_video2. Then, PG_textST_stream_id for specifying a subtitle stream that can be combined with the secondary video stream specified by video_stream_id2 is defined in a for sentence after number_of_Subtitle_combinations_for_video2.

  As described with reference to FIGS. 14 and 37, each number is used instead of each ID, for example, an audio stream number (audio_stream_number) is used instead of audio_stream_id, or an audio stream number 2 is used instead of audio_stream_id2. (Audio_stream_number2) may be used. The same applies to the video stream and the subtitle stream.

  According to the fifth STN_table () described with reference to FIGS. 40 and 41, the video_stream_id2 can be used to define a secondary video stream that is played back simultaneously with the primary video stream, and the video_stream_id2 and Using audio_stream_id, audio_stream_id2, and PG_textST_stream_id, a combination of a primary audio stream, a secondary audio stream, and a subtitle stream that can be played back simultaneously with the secondary video stream can be defined. That is, a combination of a secondary video stream, a primary audio stream, a secondary audio stream, and a subtitle stream that can be reproduced simultaneously with the primary video stream can be defined.

  Therefore, since it is not necessary to define a conflicting combination as a combination of the secondary video stream, the primary audio stream, the secondary audio stream, and the subtitle stream that can be played back simultaneously with the primary video stream, the user can It is possible to easily select a combination of streams that can be reproduced simultaneously.

  A specific example will be described with reference to FIGS. FIG. 42 corresponds to the audio signal and secondary video stream corresponding to the primary audio stream or secondary audio stream provided to the user when the fifth STN_table () described with reference to FIGS. 40 and 41 is used. It is a figure which shows the example of the stream number table showing the relationship between the combination of a video signal and a subtitle signal corresponding to a subtitle stream. In the figure, the description of the audio signal (audio) and the subtitle signal (sub-picture) is the same as in FIG. 24 or FIG.

  In FIG. 42, V2_SN (Video2 Stream Number: also referred to as secondary video stream number) is assigned to each secondary video stream (secondary video stream entered with video_stream_id2) entered in STN_table () of PlayItem that constitutes the main path of the PlayList. Is given.

  Specifically, video 2 is given to V2_SN = 1, and video 1 is given to V2_SN = 2.

  At this time, for video 2 with V2_SN = 1, audio 2 (primary audio stream entered with audio_stream_id in the for statement when video_stream_id2 in FIG. 41 is an ID for identifying video 2) is the primary audio stream. As a secondary audio stream, audio 4 (secondary audio stream entered with audio_stream_id2 in the for sentence when video_stream_id2 in FIG. 41 is an ID for identifying video 2) is sub-picture 3 (video_stream_id2 in FIG. The subtitle streams entered by PG_textST_stream_id in the for sentence in the case of the ID for specifying the video 2 can be combined.

  For video 1 with V2_SN = 2, audio 1 (primary audio stream entered with audio_stream_id in the for statement when video_stream_id2 in FIG. 41 is an ID for identifying video 1) is used as a primary audio stream. As the secondary audio stream, audio 4 or audio 5 (each of the two secondary audio streams entered in each of the two audio_stream_id2 in the for statement when video_stream_id2 in FIG. 41 is an ID for identifying video 1) is a subtitle stream. Sub-picture 1 (sub-title stream entered with PG_textST_stream_id in the for sentence when video_stream_id2 in FIG. 41 is an ID for identifying video 1) can be combined.

  That is, in the for statement in the case where video_stream_id2 in FIG. 41 is an ID for identifying video 1, the following first set and second set are set as audio stream sets that can be played back simultaneously with video 1 with V2_SN = 2. Are defined respectively. The first set is a set in which the primary audio stream is audio 1 and the secondary audio stream is audio 4. The second set refers to a second set in which the primary audio stream is audio 1 and the secondary audio stream is audio 5.

  When the stream number table shown in FIG. 42 is presented to the user (in this case, the presentation form is not particularly limited), the user first starts the secondary video stream corresponding to the video of the director's commentary or the like to be played back simultaneously with the main movie or the like. That is, one of video 2 (VS_SN = 1) and video 1 (VS_SN2) is selected as the secondary video stream corresponding to the video to be displayed on the sub-screen (sub-screen 302 in the example of FIG. 39). (Such selection operation is possible).

  For example, when video 2 (VS_SN = 1) is selected as the secondary video stream, as shown in FIG. 42, the combinations that can be reproduced simultaneously with video 2 (VS_SN = 1) are audio 2 and audio 4 , And a combination of sub-pictures 3 only. Therefore, when the video 2 (VS_SN = 1) is selected as the secondary video stream, the user's selection operation ends.

  On the other hand, as shown in FIG. 43, when video 1 (VS_SN = 2) is selected as the secondary video stream, the combinations that can be reproduced simultaneously with video 1 (VS_SN = 2) include audio 1 and audio 4 and a first combination of sub-picture 1 and two combinations of audio 1, audio 5 and a second combination of sub-picture 1 exist. Therefore, when video 1 (VS_SN = 2) is selected as the secondary video stream, the user can further select one of the first combination and the second combination (such as such Select operation).

  Specifically, for example, here, as in the selection operation described with reference to FIG. 35, the user selects the right side of the corresponding display on the stream number table with respect to the stream to be simultaneously played with video 1 (VS_SN = 2). On the other hand, when selecting playback, it is possible to execute a selection operation such as switching from “×” to “◯” while holding “X” when not selecting playback. As in the case described with reference to FIG. 35, “◯” is a symbol indicating a state in which reproduction is selected, while “x” is a symbol indicating a state in which reproduction is not selected. .

  In the example of FIG. 43, the result of selecting the first combination of audio 1, audio 4, and sub-picture 1 by such a selection operation is shown. That is, as shown in FIG. 43, since audio 1 is selected as the primary audio stream that is played back simultaneously with video 1 of V2_SN = 2, the selection state is switched from “×” to “◯”. Also, since audio 4 is selected as the secondary audio stream that is played back simultaneously with video 1 of V2_SN = 2 (because audio 5 is not selected), the selection state of audio 4 is switched from “×” to “◯”. On the other hand, the selection state of the audio 5 is kept “x”. Further, since sub-picture 1 is selected as a subtitle stream that is played back simultaneously with video 1 with V2_SN = 2, the selection state is switched from “×” to “◯”.

  That is, by the series of selection operations described above, the video 1, audio 1, audio 4, and sub-picture 1 can be reproduced together with the main video (which is a primary video stream) (such reproduction is selected. State).

  In the above-described example, as an operation for selecting a stream to be reproduced simultaneously with the primary video stream, an operation with the secondary video stream as an axis, that is, after selecting the secondary video stream first, the primary audio stream, the secondary audio stream, and The operation for selecting a subtitle stream has been described.

  However, as such a selection operation, not only the above-described example, but also, for example, in the same manner as the operation described above using FIG. 35 and the like, after the primary audio stream is first selected, the secondary video stream, the secondary audio stream, and An operation for selecting a subtitle stream may be employed. That is, any one of the audio 2 with A_SN = 1, the audio 1 with A_SN = 2, and the audio 3 with A_SN = 3 in the stream number table of FIG. 43 is set as the primary audio stream. A selection operation of selecting the secondary video stream, the secondary audio stream, and the subtitle stream by the same operation as that described above with reference to FIG.

  Further, for example, an operation of selecting a primary audio stream, a secondary video stream, and a secondary audio stream after selecting a subtitle stream first may be employed. That is, any one of sub-picture 3 with S_SN = 1, sub-picture 1 with S_SN = 2, and sub-picture 2 with S_SN = 3 in the stream number table of FIG. As shown in FIG. 43, a selection operation such as selecting a secondary video stream, a primary audio stream, and a secondary audio stream may be employed.

  In this embodiment, the user switching operation of the secondary audio stream is prohibited. For this reason, in the example of FIG. 43, A_SN for the secondary audio stream is not given. That is, in the example of FIG. 43, audios 1 to 3 are used as primary audio streams, and audios 4 and 5 are used as secondary audio streams.

  By the way, when the picture-in-picture method (see FIG. 39) is realized by the syntax using the fifth STN_table () of FIG. 40 and FIG. 41 described above, the type of Subpath_type has been described with reference to FIG. When the type of the first example is used, for example, a PlayList shown in FIG. 44 is generated.

  In the example of FIG. 44, Video (video stream) and Audio (audio stream) included in clip AV Stream-0 (main Clip AV Stream) referred to by play item 1 (PlayItem 1) constituting the main path (Main path). Are a primary video stream and a primary audio stream. That is, in correspondence with the example of FIG. 39, each of Video and Audio included in clip AV Stream-0 is reproduced and displayed on the full screen 301 as a primary video stream 311 and a primary audio stream 312, respectively. Sound is output.

  On the other hand, 2ndVideo (video stream) included in Clip AV Stream-1 referred to by a sub play item (SubPlayItem: SPI) included in sub path 1 (Sub path-1) is a secondary video stream, and sub path 2 ( 2ndAudio (audio stream) included in Clip AV Stream-1 as a sub play item (SPI) included in Sub path-2) is a secondary audio stream. That is, in correspondence with the example of FIG. 39, each of 2ndVideo and 2ndAudio included in Clip AV Stream-1 is reproduced and displayed on the child screen 302 as a secondary video stream 313 and a secondary audio stream 314, respectively. Sound is output.

  However, the audio included in Clip AV Stream-0 is played back (sound output) as a primary audio stream in association with the entire screen 301, and the 2ndAudio included in Clip AV Stream-1 is displayed as a secondary audio stream on the child screen 302. Playing in correspondence (sound output) means that, as described above, the sound obtained as a result of performing the mixing process on these two audio streams is played back and output.

  As described above, in the example of FIG. 44, the type of the example of FIG. 11 is used as the type of SubPath_type, and as a result, the picture-in-picture technique is realized by the main path, the subpath 1, and the subpath 2.

  However, in the example of FIG. 44, each of the secondary video stream and the secondary audio stream that are played back simultaneously with the video of the clip AV Stream-0 that is the primary video stream is the 2ndVideo included in the same Clip AV Stream-1. In spite of being 2ndAudio, it is defined in each of separate subpath 1 and subpath 2. This is because, in the example of FIG. 11, only the SubPath_type type that only one ES (elementary stream) is referenced is prepared. That is, in the example of FIG. 11, each type of ES such as Audio, IG, Text, and Video is included in each Clip AV Stream file included in the same Clip. This is because only the SubPath_type type that SubPath_type is always assigned one by one is prepared.

  That is, regardless of the example of FIG. 44, when the type of SubPath_type in the example of FIG. 11 is used, when there are two or more ESs to be reproduced, apart from the ES referenced in the main path, these two are used. The above ESs are divided into separate sub-paths, that is, defined as two different SPIs.

  In this case, if these two or more ESs are included in the Clip AV Stream file included in the same Clip, the above-mentioned SubPlayItem_IN_time, SubPlayItem_OUT_time, etc. are the same even though they are the same. As a result, there is a need for multiple times), and the first problem is that the information of SubPlayItem_IN_time and SubPlayItem_OUT_time becomes redundant.

  Further, in this case, since a sub path must be defined for each ES to be reproduced, a second problem that the structure of the PlayList becomes more complicated than necessary occurs.

  Furthermore, from the second problem, the following third problem also occurs.

  That is, for example, in this embodiment, the playback device (for example, the playback device 20 in FIG. 25 described above or the playback device 401 in FIG. 49 described later) can read only up to two TSs (two clips) at the same time. Restrictions are imposed. In a playback apparatus with such restrictions, when a PlayList having such a complicated structure is to be played, it is immediately determined which of the main path defined in the PlayList and which sub-path should be combined for playback. The third problem arises that it is not possible, that is, it is necessary to execute complicated processing to determine such a combination.

  Therefore, in order to solve the first to third problems, the present inventor adds, as the type of SubPath_type, a type that can handle a plurality of ESs, that is, a type that can simultaneously reference a plurality of ESs. The method was further invented. A second example of the type of SubPath_type to which such a technique is applied is shown in FIG.

  That is, FIG. 45 is a second example of the type of SubPath_type (subpath type) and is different from the example of FIG. 11, that is, for the types existing in the example of FIG. It is a figure explaining the example which added the kind which can solve 3 problems. That is, in order to solve the first to third problems, for example, the second SubPath_type shown in FIG. 45 may be adopted.

  In FIG. 45, SubPath_type = 0 to 4 are the same as the corresponding types in FIG. 11, so the description thereof is omitted here.

  However, in “Meaning” with SubPath_type = 2 to 4, the description “Out-of-mux” is described in the example of FIG. 45, but the description is omitted in the example of FIG. The description will be described. That is, the description of “Out-of-mux” is the type (type) of the subpath when the TS including the ES referred to in the subpath is different from the TS including the ES referred to in the main path. That is, the ES referred to in the sub path indicates that the sub path type is not multiplexed in the TS including the play item referred to in the main path. Hereinafter, this type is referred to as a main path TS non-multiplex type path.

  Further, when the ES referred to in the main path and the ES referred to in the sub path are asynchronous as in SubPath_type = 2, 3, the type (kind) of the sub path is hereinafter referred to as an asynchronous path. On the other hand, when the ES referenced by the main path and the ES referenced by the sub path are synchronized as in SubPath_type = 4, 5, the type (type) of the sub path is hereinafter referred to as a synchronous path.

  SubPath_type = 5 is Out-of-mux and AV Synchronized type of one or more elementary streams path (Primary audio / PG / IG / Secondary audio path) .Out of mux and AV synchronized type of Picture-in-Picture presentation path which Contains one or more elementary streams paths. In other words, SubPath_type = 5 is a main path TS non-multiplexed and synchronous path, and includes one or more ES (Primary audio / PG / IG / Secondary audio) paths, picture-in-picture presentation paths, and so on. Has been.

  That is, SubPath_type = 5 includes the types indicated by SubPath_type = 5 and 6 in the example described with reference to FIG. 11, and refers to the 2nd Audio Presentation path (second audio presentation path) (refers to the 2nd audio stream). For example) and a 2nd Video Presentation path (second video presentation path) (path for referring to the 2nd video stream).

  Here, the picture-in-picture presentation path refers to a primary audio stream stream, a secondary video stream, a secondary audio stream, and a subtitle stream for a predetermined primary video stream in the above-described picture-in-picture method (see FIG. 39). One or more paths (types of such sub paths).

  SubPath_type = 6 is Out-of-mux and AV non-Synchronized type of Picture-in-Picture presentation path which contains one or more elementary streams paths. That is, SubPath_type = 8 is a main path TS non-multiplexed and asynchronous path, and is a picture-in-picture presentation path (one or more ES paths).

  SubPath_type = 7 is In-mux type and AV Synchronized type of Picture-in-Picture presentation path which contains one or more elementary streams paths.

  Here, the description “In-mux” is a subpath type (kind) when a TS including an ES referenced by a subpath is the same as a TS including one or more ESs referenced by a main path. That is, it is indicated that the ES referred to in the subpath is the type (kind) of the subpath multiplexed in the TS including the ES referred to in the main path. In other words, the main TS included in the main clip includes the main Clip AV stream and the sub Clip AV stream. Hereinafter, this type is referred to as a main path TS multiplexing type path.

  That is, SubPath_type = 7 is a main path TS multiplexing type and synchronous type path, which is a picture-in-picture presentation path (one or more ES paths).

SubPath_type = 8 to 255 is reserved.

  By using the type of Subpath_type in the example of FIG. 45, it is possible to use newly added SubPath_type = 5, 6, and 7. Here, in SubPath_type = 7, MainTS referred to by PlayItem, that is, the main Clip AV stream includes SubST referred to by SubPath. Therefore, instead of Stream_entry () described with reference to FIG. 17, second Stream_entry () shown in FIG. 46, in which type = 3 is defined, is used.

  That is, type = 3 defined in the second Stream_entry () is for identifying an elementary stream that is included in the MainClip and referenced by the SubPath when SubPath_type = 7 in the corresponding PlayItem. .

  46, the description of the same parts as those in FIG. 15 will be omitted as appropriate.

  That is, when type = 3, SubPath refers to the same Clip as MainPath, in other words, a plurality of elementary streams are multiplexed on MainClip, and both MainPath and SubPath are multiplexed on MainClip. When referring to any one of the streams, Ref_to_stream_PID_of_MainClip is used to specify one elementary stream referenced by SubPath from among a plurality of elementary streams of MainClip when SubPath_type = 7. MainClip packet ID (PID) is specified. Also, SubPath_id is indicated by ref_to_SubPath_id, the SubPath defined in PlayList () of FIG. 9 is specified by this SubPath_id, and the corresponding SubPlayItem is called from this Subpath (FIG. 10), so that the elementary stream playback time ( (In_time, Out_time) and the like can be grasped (FIG. 12).

  As described above, when a type (three types of type 1 to type 3) is used and a PlayItem and one or more SubPaths to be played back are prepared, a clip referred to by the PlayItem is further Even if it is referenced from SubPath, one elementary stream can be specified from the Clip.

  By using the type of Subpath_type in the example of FIG. 45 as described above, that is, using the newly added SubPath_type = 5, 6, 7 together with SubPath_type = 1 to 4 that also existed in the example of FIG. Thus, as a PlayList for realizing the picture-in-picture method (see FIG. 39), a PlayList having a simple structure as shown in FIG. 47, for example, instead of a PlayList having a complicated structure as in the example of FIG. Can be created / used.

  That is, also in the example of FIG. 47, each of Video (video stream) and Audio (audio stream) included in Clip AV Stream-0 referred to by PlayItem 1 (PlayItem1) included in the main path is Each is a primary video stream and a primary audio stream. That is, in correspondence with the example of FIG. 39, each of Video and Audio included in clip AV Stream-0 is reproduced on the full screen 301 as a primary video stream 311 and a primary audio stream 312, respectively. Is output.

  Also in the example of FIG. 47, each of the secondary video stream and the secondary audio stream that are played back simultaneously with the video of Clip AV Stream-0 that is the primary video stream is the same as the 2nd Video included in the same Clip AV Stream-1. Each with 2ndAudio.

  Therefore, in the example of FIG. 47, as the type of sub path A (Sub path-A), the above-mentioned SubPath_type = 7, that is, the main path TS non-multiplexed type and synchronous type path, which is a picture-in-picture presentation path. Is adopted. As a result, two ESs of 2ndVideo and 2ndAudio included in Clip AV Stream-1 can be referenced simultaneously as a sub play item (SPI) referred to by sub path A (Sub path-A). As a result, when corresponding to the example of FIG. 39, 2ndVideo and 2ndAudio included in Clip AV Stream -1 referred to by only one sub path A (Sub path-A) are represented by secondary video stream 313 and secondary audio, respectively. Each of the streams 314 is reproduced on the child screen 302 and output as audio.

  At this time, when the above-described SubPath_type = 5 is adopted as the type of sub path A (Sub path-A), the secondary video stream 313 and the secondary audio stream 314 are reproduced in synchronization with the primary video stream 311. It will be.

  In other words, when it is desired to reproduce the secondary video stream 313 and the secondary audio stream 314 asynchronously with the primary video stream 311 (or when it is necessary to reproduce asynchronously), the sub path A (Sub path-A) As the type, SubPath_type = 8, that is, a main path TS non-multiplexed type and asynchronous type path, which is a picture-in-picture presentation path (one or more ES paths) may be adopted.

  As described above, in the example of FIG. 47, as a result of adopting the type of Subpath_type in the example of FIG. 45, the picture-in-picture technique is realized only by one subpath other than the main path.

  That is, when realizing a picture-in-picture method in which 2ndvideo and 2ndAudio included in the same Clip AV Stream-1 are used as the secondary video stream 313 and the secondary audio stream 314, the type of Subpath_type in the example of FIG. 44, the 2ndvideo and 2ndAudio included in the same Clip AV Stream-1 are referred to by two separate subpaths. On the other hand, in the PlayList of the example of FIG. 47 generated using the Subpath_type of the example of FIG. 45, each of 2ndvideo and 2ndAudio included in the same Clip AV Stream-1 can be referred to by only one subpath. It is like that.

  As a result, it is only necessary to have one SubPlayItem_IN_time, SubPlayItem_OUT_time, etc. for each of 2ndvideo and 2ndAudio, so that the first problem described above can be solved.

  Further, as apparent from comparing the PlayList in the example of FIG. 44 and the example of FIG. 47, the structure can be simplified, so that the second problem and the third problem can be solved.

  Furthermore, SubPath_type = 7 of the types of Subpath_type in the example of FIG. 45, that is, a main path TS multiplexing type and synchronous type path, which uses a picture-in-picture presentation path (one or more ES paths). Thus, for example, a PlayList as shown in FIG. 48 can be created. As a result, the picture-in-picture technique using the PlayList in the example of FIG. 48 can be easily realized.

  In the example of FIG. 48, the Main TS referred to by the play item 1 (PlayItem 1) referred to by the main path (Main path) includes Clip AV Stream-0 (however, a Clip different from the Clip AV Stream-0 of FIG. 47). AV Stream) is included, and Clip AV Stream-0 includes a Main TS referenced by PlayItem1 and a Sub TS referenced by SubPlayItem. Then, each of Video (video stream) and Audio (audio stream) included in the Main TS becomes a primary video stream and a primary audio stream, respectively. That is, in correspondence with the example of FIG. 39, each of Video and Audio included in Clip AV Stream-0 is reproduced on the full screen 301 as the primary video stream 311 and the primary audio stream 312, respectively, and the sound is output. Is done.

  Also, in the example of FIG. 48, each of the secondary video stream and the secondary audio stream that are played back simultaneously with the video of the Main TS that is the primary video stream is the second video included in the Clip AV Stream-0 included in the same Main TS. Each with 2ndAudio.

  In this case, in order to create the PlayList in the example of FIG. 48, SubPath_type = 7 may be adopted as the type of sub path A (Sub path-A). As a result, as the ES that the sub play item (SPI) included in the sub path A (Sub path-A) refers to, the SubST 2nd Video and 2nd Audio included in the Clip AV Stream-0 of the Main TS that is also referenced in the main path The two ESs can be referred to. As a result, according to the example of FIG. 39, 2ndVideo and 2ndAudio included in Clip AV Stream-0 referred to by sub path A (Sub path-A) are respectively converted into secondary video stream 313 and secondary audio stream 314. Respectively, it is possible to reproduce on the sub-screen 302 and output sound. That is, by configuring so that one file includes a main path and a sub path, a stream included in another file can be reproduced simultaneously.

  In addition, as shown in FIG. 48, as a sub-path for PlayItem-1, in addition to sub-path A, Clip AV Stream -1 included in a TS different from MainTS including Clip AV Stream-0 (however, 44, when a subpath B (Subpath-B) that refers to a Clip AV Stream that is different from Clip AV Stream -1 in FIG. 44 is defined, as described above, the playback device according to the present embodiment has 2 Since one TS (Clip) can be read out simultaneously, it is possible to read out the PG included in the SubTS as well as the Video, Audio, 2ndVideo, and 2ndAudio included in the MainTS.

  Heretofore, the syntax example necessary for applying the picture-in-picture method (see FIG. 39), that is, the STN_table () in the example of FIGS. 40 and 41 and the type of SubPath_type in the example of FIG. 45 have been described.

  In order to realize such a picture-in-picture method, it is necessary to apply a playback device having a function of simultaneously playing a primary video stream and a secondary video stream, for example, a playback device as shown in FIG.

  That is, FIG. 49 is a block diagram showing a configuration example of a playback apparatus to which the present invention is applied, which is different from the example of FIG.

  The playback device 401 in the example of FIG. 49 is a PlayList having the above-described main path and sub path, and performs playback according to the PlayList (for example, the PlayList in the examples of FIGS. 47 and 48) for realizing the picture-in-picture method. It can be carried out.

  49, parts (blocks) corresponding to those in FIG. 25 are denoted by the corresponding reference numerals, and description thereof will be omitted as appropriate.

  The playback device 401 in the example of FIG. 49 is provided with a storage drive 31, a switch 32, and a controller 34 that have the same functions and configurations as the playback device 20 in the example of FIG. However, the function of the controller 34 is added. Details of the added function will be described later. Further, the playback device 401 in the example of FIG. 49 is provided with an AV decoder unit 403 having a slightly different function and configuration from the AV decoder unit 33 of the playback device 20 in the example of FIG.

  That is, the AV decoder unit 403 in the example of FIG. 49 is provided with a buffer 51 through a mixing processing unit 101 having substantially the same functions and configurations as the AV decoder unit 33 in the example of FIG. The AV decoder unit 33 in this example is provided with a PID filter 411 to a preload buffer 414 which are not provided.

  Note that “almost” is described as “having substantially the same function and configuration as the AV decoder unit 33 in the example of FIG. 25” because the name of the video decoder 72 and the video plane generating unit 92 is the 1st video decoder. This is because there are some differences such as a change to 72 or the 1st video plane generation unit 92, or an increase in the output of the PID filter 56 by one.

  In the following, portions of the AV decoder unit 403 in the example of FIG. 49 that are different from the AV decoder unit 33 in the example of FIG. 25 will be described.

  The AV decoder unit 403 needs to decode the primary video stream and the secondary video stream in order to realize the picture-in-picture method. Therefore, each of the video decoder 72 and the video plane generation unit 92 provided in the AV decoder unit 33 in the example of FIG. 25 is the first video for the primary video stream in the AV decoder unit 403 in the example of FIG. Each of the decoder 72 and the 1st video plane generating unit 92 is provided. Further, the AV decoder unit 403 in the example of FIG. 49 includes a 2nd video decoder 412 and a 2nd video plane for secondary video streams having the same functions and configurations as the 1st video decoder 72 and the 1st video plane generating unit 92, respectively. Each of the generators 413 is provided.

  Also, the main ClipAV Stream video stream or the sub Clip AV Stream video stream is input to the PID filter 411 via the PID filter 55 or the PID filter 56. Therefore, the PID filter 411 distributes the input video stream to the subsequent 1st video decoder 72 or 2nd video decoder 412 according to the PID (packet ID) and outputs the result. That is, when a primary video stream is input, the PID filter 411 supplies the primary video stream to the 1st video decoder 72. In contrast, when a secondary video stream is input, the PID filter 411 supplies the secondary video stream to the 2nd video decoder 412.

  In the present embodiment, only the main ClipAV Stream video stream is used as the primary video stream, and the sub Clip AV Stream video stream is not used. However, the sub-Clip AV Stream video stream supplied from the PID filter 56 can also be supplied to the 1st video decoder 72 in consideration that the sub-Clip AV Stream video stream may be used as the primary video stream in the future. The PID filter 411 is configured.

  The preload buffer 414 temporarily stores the interactive graphics stream supplied from the interactive graphics decoder 74. That is, as described above, in the present embodiment, the playback device 401 is restricted to read only two TSs (Clip) at a time. Therefore, the playback device 401 obtains the interactive graphics stream in advance (by preloading), so that when the video stream or the audio stream is played back, the preloaded interactive graphics stream is simultaneously received. It has the function of being able to reproduce. In order to realize such a function, that is, to store the preloaded interactive graphics stream until the timing when the video stream or the audio stream is reproduced, the preload buffer 414 includes the AV decoding unit in the example of FIG. 403 is provided.

  49 is an example of processing by the picture-in-picture method executed by the playback apparatus 401 in the example of FIG. 49, and is a series necessary for switching the secondary video stream to be played back during playback of the primary video stream or the like. An example of this process (hereinafter referred to as secondary video switching process) will be described with reference to the flowcharts of FIGS.

  In step S101 of FIG. 50, the controller 34 displays an order list of secondary video stream numbers (which may be IDs) and a list of primary audio streams, secondary audio streams, and subtitle streams combined with the secondary video streams. 40 and the fifth STN_table () described with reference to FIG.

  Here, after the processing of step S101, based on the acquired list, for example, the above-described stream number table as shown in FIG. 42 is generated, and the stream number table itself or a predetermined GUI (Graphical) generated based on the stream number table is generated. User Interface) image is presented to the user. As a result, the process proceeds to step S102.

  In step S102, the controller 34 receives a switching instruction for the secondary video stream from the user.

  That is, the content of the user switching operation described with reference to FIGS. 42 and 43 described above is received by the controller 34 as a switching command for the secondary video stream from the user in step S102.

  In step S103, the controller 34 acquires the number of the next combination of the secondary video stream, the primary audio stream, the secondary audio stream, and the subtitle that are being reproduced.

  Specifically, for example, if the stream number table in the example of FIG. 42 is presented to the user, the following first combination to the following are combinations of secondary video streams, primary audio streams, secondary audio streams, and subtitles. There will be 3 combinations. That is, the first combination is a combination of video 2, audio 2, audio 4, and sub-picture 3. The second combination is a combination of video 1, audio 1, audio 4, and sub-picture 1. The third combination is a combination of video 1, audio 1, audio 5, and sub-picture 1. Thus, for example, here, it is assumed that 1, 2-1, and 2-2 are given as the numbers of the combinations of the first combination, the second combination, and the third combination, respectively. For example, here, it is assumed that the selection operation result of the example of FIG. 43, that is, the selection command for the second combination is received by the controller 34 in the process of step S <b> 102. In this case, in the process of step S103, 2-1 is acquired as the next combination number.

  In step S104, the controller 34 refers to stream_attribute () and has a function of reproducing a plurality of streams corresponding to the acquired number (in the first step S104, the number acquired in the immediately previous step S103). It is determined whether or not it has.

  Specifically, for example, focusing on the audio stream playback function, if the controller 34 accepts the selection operation result in the example of FIG. 43, that is, the second combination selection command, the audio 1 and audio 4 The determination as to whether or not each has a reproduction function is executed as the process of step S104.

  Note that a specific example of the method for determining whether or not the audio stream has a playback function has been described in conjunction with the description of step S74 in FIG.

  In addition, it is also possible to determine whether or not there is a playback function for other streams such as a video stream as the processing in step S104.

  If it is determined in step S104 that the plurality of streams corresponding to the acquired number are not played back, the controller 34 acquires the number of the next combination after the current combination number in step S105. That is, when the audio stream or the like specified by the current combination number is not reproduced, the combination number is skipped (not subject to reproduction), and the next combination number is acquired. Thereafter, the process returns to step S104, and the subsequent processes are repeated. That is, the loop processing of steps S104 and S105 is repeated until a combination number specifying an audio stream or the like having a function to be played back by the playback device 401 itself is acquired.

  In step S104, the acquired number (in the case of the first process, the number acquired in the process of the immediately preceding step S103 as described above, and in the process of the second and subsequent processes, it is acquired in the process of the immediately preceding step S105. If it is determined that a function for reproducing a plurality of streams corresponding to the number of the corresponding streams is determined, the process proceeds to step S106 in FIG.

  In step S106, the controller 34 examines a clip including each of the secondary video stream, primary audio stream, secondary audio stream, and subtitle stream corresponding to the acquired number based on the stream_entry () and type information. That is, it is confirmed whether the secondary video stream, primary audio stream, secondary audio stream, and subtitle stream corresponding to the acquired number are in the main clip or the sub clip. Note that a specific example of the confirmation method for each stream has been described in the description of the processing in step S76 in FIG.

  In step S107, the controller 34 specifies each desired secondary video stream, primary audio stream, secondary audio stream, and subtitle stream. That is, in the process of step S107, each desired stream corresponding to the acquired number is specified from the main clip or the sub clip. A specific example of the method for identifying each stream has been described in the description of the processing in step S77 in FIG. 30 described above, and is omitted here.

  In step S108, the controller 34 selects each desired secondary video stream, primary audio stream, secondary audio stream, and subtitle stream from a clip (main clip or sub clip) in which each specified desired stream is multiplexed. To the storage drive 31. Based on this instruction, the storage drive 31 reads a desired stream from the target Clip. Specifically, the controller 34 instructs the storage drive 31 to read the stream identified in step S7 from the Clip identified in step S106.

  In step S109, the controller 34 instructs the AV decoder unit 403 to reproduce each of the read streams, that is, the secondary video stream, the primary audio stream, the secondary audio stream, the subtitle stream, and the like. Specifically, the controller 34 reads so as to cause the 2nd video decoder 412 to decode the read desired secondary video stream, and to cause the 1st audio decoder 75-1 to decode the read desired primary audio stream. Instruct the presentation graphics decoder 73 to decode the read desired subtitle stream so that the desired secondary audio stream is decoded by the second audio decoder 75-2.

  In step S110, the AV decoder unit 403 decodes and outputs the secondary video stream, primary audio stream, secondary audio stream, and subtitle stream, and the process ends.

  At this time, for the primary audio stream and the secondary audio stream, as described in the description of the processing in step S80 in FIG. 30 described above, an audio signal obtained as a result of mixing is output. .

  As a result of such secondary video switching processing, the picture-in-picture method described with reference to FIG. 39 is realized. That is, in the example of FIG. 39, while the video corresponding to the primary video stream 311 is being displayed (reproduced) on the full screen 311, the video corresponding to the secondary video stream 313 output in the process of step S110 is a child. The primary audio that is displayed (reproduced) on the screen 302 and that is displayed (reproduced) at a predetermined position on the entire screen 311 and that corresponds to the subtitle stream 315 that is output in the process of step S110 and that is output in the process of step S110 Audio obtained as a result of mixing the stream 311 and the secondary audio stream 402 is output (reproduced) from a speaker not shown in FIG.

  As described above, the picture-in-picture technique can be realized by adopting the STN_table in the examples of FIGS. 40 and 41.

  When realizing such a picture-in-picture method, it is possible to use a playlist (for example, see FIG. 44) generated by using the type of SubPath_type in the example of FIG. 11, but as described above, the same The secondary video stream, secondary audio stream, etc. included in the Clip AV Stream are divided into separate sub-paths, that is, defined as two different SPIs. As a result, the various problems described above, for example, the problem that the structure of the PlayList becomes more complicated than necessary occurs.

  Therefore, when solving these various problems, two or more ESs (secondary video stream, secondary audio stream, etc.) in the same Clip AV Stream can be referred to by one sub-path, that is, the same Clip AV Stream. The type of SubPath_type that can define two or more ESs as one SPI, that is, the type of SubPath_type described with reference to FIG. 45, for example, may be employed.

  However, from the viewpoint of the creator of the PlayList, it can be said that the PlayList having a simple structure can be created by adopting the type of SubPath_type in the example of FIG. Although the structure has been simplified, the following difficult work is required to create a PlayList with such a simple structure. In other words, the creator of the PlayList decides for himself what kinds of streams can be combined and what kinds of streams cannot be combined in order to realize the picture-in-picture method. In other words, it is necessary to create a PlayList after determining by itself what kind of sub-paths to include. Therefore, it is not difficult to imagine that the creator of PlayList will come up with a desire to easily create PlayList.

  Therefore, in order to respond to such a request in advance, the present inventors further restrict the creation (subpaths that can be included in the PlayList) according to the type of PlayList by the SubPath_type described with reference to FIG. I invented such a technique. Hereinafter, this method is referred to as a sub-path restriction method.

  By applying the sub-path restriction method, the sub-paths that can be included in one PlayList are limited. Therefore, it is easy for the creator to create sub-paths after determining what sub-paths are included in the PlayList. As a result, it is possible to produce an effect that the creation of the PlayList itself becomes easy.

  Hereinafter, the sub path limiting method will be described in detail.

  However, in the sub-path restriction method, the restriction contents vary depending on the number of TSs that can be read by the playback apparatus at one time. In the present embodiment, as described above, the number of TSs that can be read by the playback apparatus at a time is two. Thus, hereinafter, a sub-path limiting method in the case where the playback apparatus can read two TSs at a time will be described.

  In the present embodiment, the PlayList is roughly divided into two types, namely, a Browsable Slideshow (asynchronous type) and a Movie Type / Time-based Slideshow (synchronous type). The distinction as to whether the type of PlayList is asynchronous or synchronous can be made according to application_type (application type) included in the Clip_Information_file of the Clip referenced by the PlayList.

  application_type is described in ClipInfo () in the Clip Information file for a certain Clip. However, here, for simplicity of explanation, description of the syntax of ClipInfo () is omitted, and only the types of application_type that can be described in Clipinfo () will be described below with reference to FIG.

  That is, FIG. 52 is a diagram illustrating an example of the type of application_type.

  As shown in FIG. 52, application_type = 0 is reserved. application_type = 1 is MainTS for movie application. Here, “MainTS” refers to a TS as a PlayItem referred to by the main path in the Playlist, that is, the Main TS. application_type = 2 is a MainTS for Time-based Slideshow, that is, a MainTS for a slide show of still images. application_type = 3 is a TS for Browsable Slideshow, that is, a MainTS for a video slide show. application_type = 4 is a TS of a Browsable Slideshow for SubPath. This is given to Clip Info () for a Clip AV stream that holds BGM (sound) data that is reproduced asynchronously with application_type = 3, for example. application_type = 5 is a TS for SubPath interactive graphics. application_type = 6 is a TS for a SubPath text subtitle (text subtitle data). application_type = 7 is a TS for SubPath including one or more ESs (elementary streams). application_type = 8 to 255 is reserved.

  So, in this Embodiment, when application_type described in ClipInfo () about Main TS indicated by PlayList is application_type = 3, the PlayList is classified as Browsable Slideshow (asynchronous type), and application_type = 1or2 In this case, it is assumed that the PlayList is classified as Movie Type / Time-based Slideshow (synchronous type).

  In this case, the subpath restriction method imposes a restriction that only the combinations shown in FIG. 53 are allowed for combinations of SubPaths that the PlayList can have for the type of PlayList (asynchronous type or synchronous type). Become.

  In FIG. 53, the item “number of SubPaths” describes the number of types of SubPaths indicated by the item “SubPath_type” on the left side that can be possessed by the Main TS indicated by the PlayList.

  Also, each description value (SubPath_type type) of the item “SubPath_type” in FIG. 53 corresponds to the type described with reference to FIG. 45 instead of the type described with reference to FIG. 11 described above.

  As shown in FIG. 53, when the type of PlayList is asynchronous, that is, in the case of a Browsable slideshow with application_type (of Main TS) = 3, (1) zero or more SubPath_type = 2 SubPaths, (2) Only zero or more SubPath_type = 3 SubPaths are allowed. In other words, combinations of SubPaths other than (1) and (2) are prohibited.

  Even in the combination of (2), when IG is included in the Main TS of application_type = 3, SubPath of SubPath_type = 3 is prohibited (see * 2 in FIG. 53). This is due to the following restrictions.

  That is, from the viewpoint of IG, PlayList including IG is roughly divided into the following first type and second type. The first type is a type in which an IG is multiplexed on a referenced Main TS, and Audio / Video and IG of the Main TS are reproduced in synchronization. The second type includes a SubPat of SubPath_type = 3 and IG is preloaded (this IG is used for a pop-up menu). Note that, as described above, the playback device 401 in FIG. 49 is provided with the preload buffer 414 so that the second type PlayList can be played back.

  Further, since the playback device of this embodiment has only one IG decoder, specifically, for example, the playback device 401 of FIG. 49 has only one interactive graphics decoder 74, the following restrictions are imposed. It has been. That is, in the first type PlayList described above, there is a restriction that another IG cannot be preloaded while the IG decoder is decoding the IG of the Main TS. On the other hand, in the second PlayList described above, there is a limitation that when the first IG preloaded in the IG decoder is being input, the second IG from another Main TS cannot be input to the IG decoder. Imposed.

  From the above, the PlayList including STN_table () that refers to the IG multiplexed in the Main TS is subject to the restriction that it cannot have a SubPath of SubPath_type = 3. Because of this restriction, when IG is included in MainTs of application_type = 3, SubPath of SubPath_type = 3 is prohibited (see * 2 in FIG. 53).

  On the other hand, when the type of PlayList is synchronous, that is, in the case of Movie Type / Time-based Slideshow where application_type (of Main TS) = 1 or 2, (3) zero or more SubPath_type = 3 SubPaths, (4 ) 0 or more SubPath_type = 4 SubPaths, (5) 0 or more SubPath_type = 5 SubPaths, (6) 0 or more SubPath_type = 6 SubPaths, (7) 0 or 1 SubPath_type = 7 Only SubPath is allowed. In other words, the combination of SubPaths with SubPath_type = 2 is prohibited.

  Even in the combination of (3), when IG is included in MainTS of application_type = 3, or when IG is included in TS that is referenced by SubPath of SubPath_type = 5, SubPath of SubPath_type = 3 is Prohibited (refer to * 1 in FIG. 53). This is also due to the constraints described above.

  Furthermore, in the present embodiment, as described above, the number of Clips that can be referred to by the PlayList at a time, that is, the TS (Clip) that can be read by the playback device at a time is two. Therefore, it is guaranteed that ESs referenced by the same PlayList are included in at most two Clips, that is, ESs that are played back simultaneously are referenced simultaneously from at most two Clip AV stream files. In order to do this, a restriction that the following condition is satisfied is imposed by the subpath restriction method.

  That is, imposed by the sub path restriction method, referred to by STN_table (), primary_video_stream, primary_audio_stream, PG_textST_stream (excluding Text subtitle stream), IG_stream, secondary_video_stream (excluding IG for Pop-up menu), and secondary_audio_stream The conditions when combining each of these are as follows.

  ES identified by a certain primary_video_stream_number, ES identified by a certain primary_audio_stream_number, ES identified by a certain PG_textST_stream_number, ES identified by a certain IG_stream_number value, ES identified by a certain secondary_video_stream_number In the ES specified by a certain value of secondary_audio_stream_number, combinations that can be reproduced simultaneously need to be stored in at most two Clip AV stream files. The excluded Text subtitle stream and IG for Pop-up menu are preload-type streams, so even if they are played (displayed) at the same time, they are not read from the optical disk, hard disk, etc. at the same time. As described above, it may be stored in a Clip AV stream file different from the two Clip AV stream files.

  Note that the ES referenced by STN_table () may be stored in three or more Clip AV stream files, but ESs that can be played simultaneously are stored in at most two Clip AV stream files. It is made to be done.

  Further, the restriction that the combination of the number of PlayItems (MainPath) and SubPaths in the PlayList needs to be any one of the following (a) to (c) as shown in FIG. Is also imposed by the subpath restriction technique. That is, the combination (a) refers to a combination of only one PlayItem (the combination in the top row in FIG. 54). The combination (b) refers to a combination of one PlayItem and one SubPath (combination of the center row in FIG. 54). The combination of (c) is a combination of one PlayItem, one SubPath, and one SubPath_type = 7, that is, one PlayItem and two SubPaths (provided that one of them is SubPath_type = 7) (The combination of the bottom row in FIG. 54).

  As a result of the various restrictions described above, a technique of restricting SubPath_type that can be created for a predetermined type of PlayList is an example of a subpath restriction technique.

  In other words, the sub-path restriction method is the type of PlayList (in the above example, the type distinguished by the application_type of the Main TS), and the number of TSs that can be read at once by the playback device (in the above example, two). And there is a condition (restriction) such as whether pre-loaded IG is included in the ES referenced by the sub-path (SubPath) of the PlayList (in the above example, whether it is a SubPath such as SubPath_type = 3) Then, it can be said that the SubPath included in the PlayList is determined so as to satisfy these conditions, and in turn, a PlayList including the SubPath determined in this way is created.

  Hereinafter, a specific example of the PlayList created by the subpath restriction method will be described with reference to FIGS. 55 to 58.

  FIG. 55 shows an example of a PlayList for a Browsable Slideshow (asynchronous type), in which there is no SubPath with SubPath_type = 3, that is, a PlayList with only Sub_Path-1 to Sub_Path-3 with SubPath_type = 2. Is shown. The reason why SubPath of SubPath_type = 3 does not exist is that IG is included in Clip AV Stream-0 (ie, Main TS) referenced by PlayItem-1 / PlayItem-2 of PlayList. Since this example represents a Browsable Slideshow in which a plurality of still images are continuously displayed by a user operation, the “video” stream included in the Clip AV stream file-0 includes still image data. keeping.

  The details of the restrictions (conditions) used to create the PlayList in the example of FIG. 55 are as follows. In other words, the restriction “(1) zero or more SubPath_type = 2 SubPaths, (2) only zero or more SubPath_type = 3 SubPaths” described above with reference to FIG. 53, and “(1) The restriction that “SubPath combinations other than (2) are prohibited” and “SubPath of SubPath_type = 3 are prohibited if IG is included in MainTS of application_type = 3 even in the combination of (2). The PlayList in the example of FIG. 55 is created using the limitation “* 2 in FIG. 53”.

  In contrast to the example of FIG. 55, FIG. 56 shows a PlayList in the case of a Browsable Slideshow (asynchronous type) PlayList when SubPath of SubPath_type = 3 exists, that is, Sub_Path-1, 2 of SubPath_type = 2. In addition, an example of a PlayList in the case where Sub_Path-3 of SubPath_type = 3 exists is shown. Note that the SubPath of SubPath_type = 3 exists because the Clip AV Stream-0 referenced by PlayItem-1 / PlayItem-2 of the PlayList does not include IG, that is, the SubPath of SubPath_type = 3 This is because the IG of Clip AV Stream-3 different from AV Stream-0 is referred to.

  The contents of the restrictions (conditions) used to create the PlayList in the example of FIG. 56 are the same as the above-described restrictions of the example of FIG.

  In other words, the restriction indicated by * 2 in FIG. 53, that is, “Subpath of SubPath_type = 3 is prohibited when IG is included in MainTS of application_type = 3 even in the combination of (2)”. An example of the PlayList when it is necessary to impose such a restriction is the example of FIG. 55, and an example of the PlayList when there is no need to impose such a restriction is the example of FIG.

  As described above, FIGS. 55 and 56 illustrate an example of a PlayList of Browsable Slideshow (asynchronous type). On the other hand, FIG. 57 and FIG. 58 show examples of Movie Type Slideshow (synchronous) PlayList.

  That is, FIG. 57 shows the PlayList when the secondary video stream (2ndVideo) and the secondary audio stream (2ndAudio) are not multiplexed on the Main TS (ie, Clip AV Stream-0) in the synchronous PlayList. An example of the PlayList in the case where Sub_Path-1, 2 of SubPath_type = 6 exists is shown.

  The details of the restrictions (conditions) used to create the PlayList in the example of FIG. 57 are as follows. That is, the PlayList in the example of FIG. 57 is created using the restriction “(6) Allow 0 or more SubPath_type = 6 SubPaths” described above with reference to FIG.

  In contrast to the example of FIG. 57, FIG. 58 shows that the Main TS referred to by the synchronous PlayList, that is, the Clip AV stream file-0 includes MainST and SubST, and the secondary video stream (2nd Video) and secondary audio. This is a PlayList when the stream (2ndAudio) is multiplexed with the Main TS (Clip-0), and therefore, Sub_Path-1 with SubPath_type = 7 exists, and Sub_Path-2,3 with SubPath_type = 5 An example of a PlayList in the case where there is is shown.

  The details of the restrictions (conditions) used to create the PlayList in the example of FIG. 58 are as follows. That is, as described above with reference to FIG. 53, the restriction “(5) Allow 0 or more SubPath_type = 5 SubPaths” and “(7) Allow 0 SubPath and SubPath_type = 7 SubPaths”. Using the restriction, the PlayList in the example of FIG. 58 is created.

  Also, the PlayList created by the sub-path restriction method as described above can be reproduced by, for example, the reproduction device 401 in FIG.

  For example, in the case of an asynchronous PlayList, that is, in the case of a PlayList of Browsable slideshow with application_type = 3, the playback device 401 can play back the PlayList as follows. That is, when the PlayList refers to one or more SubPaths of SubPath_type = 3 (Interactive graphics presentation menu,), the clip AV stream for SubPath_type = 3, that is, the interactive graphics stream is converted to the PlayList. It is preloaded before reproduction and accumulated in the preload buffer 414. Further, when the PlayList refers to one or more SubPath_type = 2 SubPaths, the playback apparatus 401 can refer to SubPaths only one by one (two TSs can be read at one time, of which Since one is a MainTS referred to by MainPath), reproduction is performed with reference to a predetermined one of one or more SubPaths with SubPath_type = 2.

  The interactive graphics presentation menu includes a “Pop-up menu” that can be displayed or canceled based on the ON / OFF operation input by the user, and an “Always-on” that is always displayed. ”Is prepared, and a detailed example of playback processing of PlayList with application_type = 3 will be described later with reference to FIGS. 59 and 60.

  For example, in the case of a synchronous PlayList, that is, in the case of a PlayList of Movie Type / Time-based Slideshow where application_type = 1 or 2, the playback device 401 can play back the PlayList as follows. That is, the playback device 401, when the PlayList refers to one or more SubPath_type = 3 or SubPath_type = 4 SubPath, a Clip AV stream for SubPath_type = 3 or SubPath_type-4, that is, an interactive graphics stream or text The subtitle stream is preloaded before the PlayList is played back and stored in the preload buffer 414 or the buffer 54. In addition, when the PlayList refers to one or more SubPaths of SubPath_type = 5or6, the Player can refer to the SubPath only one by one (two TSs can be read at a time, Since one of them is a MainTS referenced by MainPath), playback is performed with reference to a predetermined one of one or more SubPaths with SubPath_type = 5 or 6. However, if one or more SubPaths of SubPath_type = 7 are included, the SubPath refers to the ES included in the Main TS, so the playback device 401 uses one or more SubPaths of SubPath_type = 5 or 6 During playback while referring to a predetermined one of them, playback can be performed with reference to one SubPath of SubPath_type = 7.

  A detailed example of PlayList playback processing with application_type = 1or2 will be described later with reference to FIGS. 61 to 64.

  Next, an example of playback processing of PlayList with application_type = 3 will be described with reference to the flowcharts of FIGS.

  For example, when the controller 34 in FIG. 49 investigates the application_type of the Main TS that is referenced by the MainPath of the PlayList to be played, and recognizes that application_type = 3 by the investigation, the playback process of the PlayList with application_type = 3 Let it begin.

  In step S121 of FIG. 59, the controller 34 determines whether or not the Main TS includes an IG (interactive graphics stream).

  If it is determined in step S121 that the Main TS does not include IG, the process proceeds to step S126 in FIG. However, the processing after step S126 will be described later.

  On the other hand, when it is determined in step S121 that the Main TS includes IG, the process proceeds to step S122.

  In step S122, the controller 34 determines whether or not there is a Sub Path of SubPath_type = 3.

  When it is determined in step S122 that there is no Sub Path of SubPath_type = 3, the process proceeds to step S126 in FIG. However, the processing after step S126 will be described later.

  On the other hand, if it is determined in step S122 that there is a Sub Path of SubPath_type = 3, the process proceeds to step S123. In step S123, the controller 34 examines the Clip referred to by the selected Sub Path (Sub Path of SubPath_type = 3).

  Then, the controller 34 instructs the storage drive 31 to read the stream specified by the Sub Path, that is, the IG from the Clip. Based on this instruction, the storage drive 31 reads the target IG. Then, the controller 34 instructs the interactive graphics decoder 74 to decode the read IG. Accordingly, the process proceeds to step S124.

  In step S124, the interactive graphics decoder 74 decodes the IG in the read Clip. In step S125, the interactive graphics decoder 74 stores the decoded IG in the preload buffer 414.

  When the process of step S125 is completed in this way, or when it is determined NO in step S121 or S122 as described above, the process proceeds to step S126 of FIG.

  In step S126, the controller 34 determines whether there is a Sub Path of SubPath_type = 2.

  If it is determined in step S126 that there is no SubPath of SubPath_type = 2, the process proceeds to step S129. However, the processing after step S129 will be described later.

  On the other hand, if it is determined in step S126 that there is a Sub Path of SubPath_type = 2, the process proceeds to step S127. In step S127, the controller 34 checks the Clip referred to by the selected SubPath (SubPath_type = 2 Sub Path).

  Then, the controller 34 instructs the storage drive 31 to read out the stream specified by the SubPath from the Clip, that is, the audio stream. Based on this instruction, the storage drive 31 reads the target audio stream. Then, the controller 34 instructs the 2nd audio decoder 75-2 to decode the read audio stream, for example. Accordingly, the process proceeds to step S128.

  In step S128, the 2nd audio decoder 75-2 decodes the audio stream in the read Clip.

  In this way, when the process of step S128 ends or when it is determined NO in the process of step S126, the process proceeds to step S129.

  In step S129, the AV decoder unit 403 decodes the main Clip AV Stream. Here, the main Clip AV Stream is a video stream, an audio stream, or the like included in the Main TS referred to by the Main Path of the playlist to be played back. For example, in the picture-in-picture method, a primary video stream, a primary audio stream, etc. I mean. That is, in the AV decoder unit 403, the 1st video decoder 72, the 1st audio decoder 75-1, and the like execute the process of step S129.

  In step S <b> 130, the AV decoder unit 403 combines the decoded main Clip AV Stream and the decoded SubPath stream, and outputs them. Here, the decoded SubPath stream is the IG decoded in step S124 in FIG. 59 and stored in the preload buffer 414 in step S125, or the audio decoded in step S128 in FIG. A stream.

  In step S131, the controller 34 determines whether or not there is a subpath switching instruction (user switching operation) of SubPath_type = 2 while outputting the stream synthesized in the process of step S130.

  If it is determined in step S131 that a subpath switching instruction of SubPath_type = 2 has been issued while outputting the combined stream, the process returns to step S127, and the subsequent processes are repeated. That is, the audio stream to be combined with the main Clip AV Stream is switched.

  On the other hand, if it is determined in step S131 that there is no SubPath_type = 2 SubPath switching instruction while outputting the combined stream, the playback process of the PlayList of application_type = 3 ends.

  Next, an example of PlayList playback processing with application_type = 1or2 will be described with reference to the flowcharts of FIGS.

  For example, when the controller 34 in FIG. 49 checks the application_type of the Main TS that is referenced by the MainPath of the PlayList to be played back and recognizes that application_type = 1or2 as a result of the check, playback of the PlayList with application_type = 1or2 Start processing.

  In step S141 in FIG. 61, the controller 34 determines whether or not the Main TS includes an IG (interactive graphics stream).

  If it is determined in step S141 that the Main TS does not include IG, the process proceeds to step S146. However, the processing after step S146 will be described later.

  On the other hand, when it is determined in step S141 that Main TS includes IG, the process proceeds to step S142.

  In step S142, the controller 34 determines whether or not there is a Sub Path of SubPath_type = 3.

  If it is determined in step S142 that there is no Sub Path of SubPath_type = 3, the process proceeds to step S146. However, the processing after step S146 will be described later.

  On the other hand, if it is determined in step S142 that there is a Sub Path of SubPath_type = 3, the process proceeds to step S143. In step S143, the controller 34 checks the Clip referred to by the selected SubPath (Sub Path of SubPath_type = 3).

  Then, the controller 34 instructs the storage drive 31 to read the stream specified by the SubPath, that is, the IG from the Clip. Based on this instruction, the storage drive 31 reads the target IG. Then, the controller 34 instructs the interactive graphics decoder 74 to decode the read IG. Accordingly, the process proceeds to step S144.

  In step S144, the interactive graphics decoder 74 decodes the IG in the read Clip. In step S145, the interactive graphics decoder 74 stores the decoded IG in the preload buffer 414.

  When the process of step S145 is completed in this way, or when it is determined NO in step S141 or S142 as described above, the process proceeds to step S146.

  In step S146, the controller 34 determines whether or not there is a Sub Path of SubPath_type = 4.

  If it is determined in step S146 that there is no SubPath of SubPath_type = 4, the process proceeds to step S149 in FIG. However, the processing after step S149 will be described later.

  On the other hand, if it is determined in step S146 that there is a Sub Path of SubPath_type = 4, the process proceeds to step S147. In step S147, the controller 34 checks the Clip referred to by the selected SubPath (Sub Path of SubPath_type = 4).

  Then, the controller 34 instructs the storage drive 31 to read the stream specified by the SubPath, that is, the text subtitle stream, from the Clip. Based on this instruction, the storage drive 31 reads the target text subtitle stream. Then, the controller 34 instructs the Text-ST composition 76 to decode the read text subtitle. Accordingly, the process proceeds to step S148.

  In step S148, the Text-ST composition 76 decodes the text subtitle in the read Clip.

  Thus, when the process of step S148 is completed or when it is determined NO in the process of step S146 described above, the process proceeds to step S149 of FIG.

  In step S149, the controller 34 determines whether there is a Sub Path of SubPath_type = 7.

  If it is determined in step S149 that there is no SubPath of SubPath_type = 7, the process proceeds to step S152 in FIG. However, the processing after step S152 will be described later.

  On the other hand, when it is determined in step S149 that there is a Sub Path of SubPath_type = 7, the process proceeds to step S150. In step S150, the controller 34 examines the Clip referred to by the selected SubPath (SubPath_type = 7 Sub Path).

  Then, the controller 34 instructs the storage drive 31 to read out the stream specified by the SubPath, that is, the video / audio stream, from the Clip. The storage drive 31 reads the target video / audio stream based on this instruction. Then, the controller 34 instructs the 2nd video decoder 412 / 2nd audio decoder 75-2 to decode the read video / audio stream, for example. Accordingly, the process proceeds to step S151.

  In step S151, the 2nd video decoder 412 / 2nd audio decoder 75-2 decodes the video / audio stream in the read Clip.

  Thus, when the process of step S151 is completed or when it is determined NO in the process of step S149 described above, the process proceeds to step S152 of FIG.

  In step S152, the controller 34 determines whether or not there is a Sub Path of SubPath_type = 5.

  If it is determined in step S152 that there is no SubPath_type = 5 Sub Path, the process proceeds to step S156. However, the processing after step S156 will be described later.

  In contrast, if it is determined in step S152 that there is a Sub Path of SubPath_type = 5, the process proceeds to step S153. In step S153, the controller 34 checks the Clip referred to by the selected SubPath (Sub Path of SubPath_type = 5).

  In step S154, the controller 34 determines whether or not there is a function for playing back an audio stream in the Clip based on the contents described in stream_attribute (). A specific example of the method for determining whether or not the audio stream has a playback function has been described in conjunction with the description of the processing in step S74 in FIG.

  If it is determined in step S154 that there is no function for reproducing the audio stream in the Clip, the process proceeds to step S156. However, the processing after step S156 will be described later.

  On the other hand, if it is determined in step S154 that the audio stream in the Clip is played, the controller 34 reads the stream specified by the SubPath, that is, the audio stream, from the Clip. 31 is instructed. Based on this instruction, the storage drive 31 reads the target audio stream. Then, the controller 34 instructs the 2nd audio decoder 75-2 to decode the read audio stream, for example. Accordingly, the process proceeds to step S155.

  In step S155, the 2nd audio decoder 75-2 decodes the audio stream in the read Clip.

  In this way, when the process of step S155 ends, or when it is determined NO in the process of step S152 or S154 described above, the process proceeds to step S156.

  In step S156, the controller 34 determines whether or not there is a Sub Path of SubPath_type = 6.

  If it is determined in step S156 that there is no Sub Path of SubPath_type = 6, the process proceeds to step S161 in FIG. However, the processing after step S161 will be described later.

  On the other hand, if it is determined in step S156 that there is a Sub Path of SubPath_type = 6, the process proceeds to step S157. In step S157, the controller 34 checks the Clip referred to by the selected SubPath (Sub Path of SubPath_type = 6).

  In step S158, the controller 34 determines whether or not there is a function for playing back an audio stream in the Clip, based on the contents described in stream_attribute (). A specific example of the method for determining whether or not the audio stream has a playback function has been described in conjunction with the description of the processing in step S74 in FIG.

  If it is determined in step S158 that there is no function to play back the audio stream in the Clip, the controller 34 instructs the storage drive 31 to read the stream specified by the SubPath, that is, the video stream from the Clip. . Based on this instruction, the storage drive 31 reads the target video stream. However, in this case, the controller 34 instructs prohibition of decoding of the audio stream included in the read Clip, and instructs the 2nd video decoder 412 to decode the read video stream, for example. Accordingly, the process proceeds to step S160. However, the processing after step S160 will be described later.

  On the other hand, if it is determined in step S158 that the audio stream in the Clip has a function to be played back, the controller 34 determines the stream specified by the SubPath from the Clip, that is, the video stream and the audio stream. The storage drive 31 is instructed to read. Based on this instruction, the storage drive 31 reads the target video stream and audio stream. Then, the controller 34 instructs the 2nd audio decoder 75-2 to decode the read audio stream, for example, and instructs the 2nd video decoder 412 to decode the read video stream, for example. To do. Accordingly, the process proceeds to step S159.

  In step S159, the 2nd audio decoder 75-2 decodes the audio stream in the read Clip.

  In this way, when the process of step S159 ends, or when it is determined NO in the process of step S158 described above, the process proceeds to step S160.

  In step S160, the 2nd video decoder 412 decodes the video stream in the read Clip.

  Thus, when the process of step S160 is completed or when it is determined NO in the process of step S156 described above, the process proceeds to step S161 of FIG.

  In step S161, the AV decoder unit 403 decodes the main Clip AV Stream. Here, the main Clip AV Stream is a video stream, an audio stream, or the like included in the Main TS referred to by the MainPath of the playlist to be played back, and is included in the Main TS. The main Clip AV Stream means, for example, a primary video stream or a primary audio stream in the picture-in-picture method. That is, in the AV decoder unit 403, the 1st video decoder 72, the 1st audio decoder 75-1, etc. execute the processing of step S161.

  In step S162, the AV decoder unit 403 combines the decoded main Clip AV Stream and the decoded SubPath stream, and outputs them. Here, the decoded SubPath stream is the IG decoded in step S144 in FIG. 61 and stored in the preload buffer 414 in step S145, and the text subtitle decoded in step S148 in FIG. 62, the audio stream decoded in step S151 in FIG. 62, the audio stream decoded in step S155 in FIG. 63, the audio stream decoded in step S159 in FIG. 62, and step S160 in FIG. This means zero or more of the video streams decoded by the above process.

  In step S163, the controller 34 determines whether or not there has been a SubPath switching instruction (user switching operation) while outputting the stream synthesized in step S162.

  If it is determined in step S163 that there has been no SubPath switching instruction while outputting the combined stream, the PlayList playback process with application_type = 1or2 ends.

  On the other hand, if it is determined in step S163 that there has been a SubPath switching instruction while outputting the combined stream, the process proceeds to step S164.

  In step S164, the controller 34 checks the SubPath_type of the SubPath selected by the switching instruction.

  In step S165, the controller 34 determines whether or not the investigation result in step S164 is SubPath_type = 3.

  If it is determined in step S165 that SubPath_type = 3, the process returns to step S143 in FIG. 61 and the process is repeated. That is, the IG combined with the main Clip AV Stream is switched.

  On the other hand, if it is determined in step S165 that SubPath_type = 3 is not satisfied, the process proceeds to step S166.

  In step S166, the controller 34 determines whether or not the investigation result in step S164 is SubPath_type = 4.

  If it is determined in step S166 that SubPath_type = 4, the process returns to step S147 of FIG. 61 and the process is repeated. That is, the text subtitle combined with the main Clip AV Stream is switched.

  On the other hand, if it is determined in step S166 that SubPath_type = 4 is not satisfied, the process proceeds to step S167.

  In step S167, the controller 34 determines whether or not the investigation result in step S164 is SubPath_type = 7.

  If it is determined in step S167 that SubPath_type = 7, the process returns to step S150 in FIG. 62 and the process is repeated. That is, the video / audio stream synthesized with the main Clip AV Stream is switched.

  On the other hand, when it is determined in step S167 that SubPath_type = 7 is not satisfied, the process proceeds to step S168.

  In step S168, the controller 34 determines whether or not the investigation result in step S164 is SubPath_type = 5.

  If it is determined in step S168 that SubPath_type = 5, the process returns to step S153 in FIG. 63 and the process is repeated. That is, the audio stream to be combined with the main Clip AV Stream is switched.

  On the other hand, if it is determined in step S168 that SubPath_type = 5 is not satisfied, the process proceeds to step S169.

  In step S169, the controller 34 determines whether or not the investigation result in step S164 is SubPath_type = 6.

  If it is determined in step S169 that SubPath_type = 6, the process returns to step S157 of FIG. 63 and the process is repeated. That is, the audio stream and video stream to be combined with the main Clip AV Stream are switched.

  On the other hand, if it is determined in step S169 that SubPath_type = 6 is not satisfied, the PlayList playback process with application_type = 1or2 is terminated.

  As described above, in order to realize an application to which the picture-in-picture method is applied, for example, the definition of the combination of streams that can be simultaneously reproduced in combination with the primary video stream, that is, the secondary video stream, the primary audio stream, the secondary It is necessary to define a combination of an audio stream and a subtitle stream.

  Furthermore, by defining a plurality of combinations, not only switching of the primary video stream and primary audio stream reproduced on the full screen 301 but also switching of the secondary video stream reproduced on the sub-screen 302 and switching of the subtitle stream are performed. Can also be easily performed.

  Next, with reference to FIGS. 65 and 66, a method of manufacturing the recording medium 21 on which data that can be reproduced by the reproducing apparatus 20 is recorded will be described by taking the case where the recording medium 21 is a disc-shaped recording medium as an example. .

That is, as shown in FIG. 65, a master disk made of glass or the like is prepared, and a recording material made of photoresist or the like is applied thereon. As a result, a recording master is produced.

  As shown in FIG. 66, in the software production processing unit, video data in a format that can be played back by the playback device 20 or the playback device 401 encoded by the encoding device (video encoder) is stored in a temporary buffer. Audio data encoded by the audio encoder is stored in the temporary buffer, and data other than the stream (for example, Indexes, Playlist, PlayItem, etc.) encoded by the data encoder is further stored in the temporary buffer. The Video data, audio data, and data other than the stream stored in each buffer are multiplexed together with a synchronization signal by a multiplexer (MPX), and an error correction code is added by an error correction code circuit (ECC). Is done. Then, predetermined modulation is applied by a modulation circuit (MOD), and software recorded once on a magnetic tape or the like according to a predetermined format and recorded on the recording medium 21 that can be reproduced by the reproducing device 20 or the reproducing device 401 is provided. Produced.

  This software is edited (premastered) as necessary, and a signal in a format to be recorded on the optical disc is generated. In response to this recording signal, the laser beam is modulated, and this laser beam is irradiated onto the photoresist on the master. Thereby, the photoresist on the master is exposed in accordance with the recording signal.

  Then, this master is developed and pits appear on the master. The master plate thus prepared is subjected to a process such as electroforming to produce a metal master plate in which pits on the glass master plate are transferred. A metal stamper is further produced from this metal master, and this is used as a molding die.

  A material such as PMMA (acrylic) or PC (polycarbonate) is injected into this molding die by, for example, injection and fixed. Alternatively, 2P (ultraviolet curable resin) or the like is applied on the metal stamper and then cured by irradiation with ultraviolet rays. Thereby, the pits on the metal stamper can be transferred onto the replica made of resin.

  A reflective film is formed on the replica thus generated by vapor deposition or sputtering. Alternatively, a reflective film is formed on the generated replica by spin coating.

  Thereafter, the inner and outer diameters of the disk are processed, and necessary measures such as bonding two disks are performed. Further, a label is attached or a hub is attached and inserted into the cartridge. In this way, the recording medium 21 on which data that can be reproduced by the reproducing apparatus 20 or the reproducing apparatus 401 is recorded is completed.

  The series of processes described above can be executed by hardware or can be executed by software. In this case, the processing described above is executed by a personal computer 500 as shown in FIG.

  67, a CPU (Central Processing Unit) 501 performs various processes according to a program stored in a ROM (Read Only Memory) 502 or a program loaded from a storage unit 508 to a RAM (Random Access Memory) 503. Run. The RAM 503 also appropriately stores data necessary for the CPU 501 to execute various processes.

  The CPU 501, ROM 502, and RAM 503 are connected to each other via an internal bus 504. An input / output interface 505 is also connected to the internal bus 504.

  The input / output interface 505 includes an input unit 506 including a keyboard and a mouse, a display including CRT and LCD, an output unit 507 including a speaker, a storage unit 508 including a hard disk, a modem, a terminal adapter, and the like. A communicator 509 is connected. A communication unit 509 performs communication processing via various networks including a telephone line and CATV.

  A drive 510 is connected to the input / output interface 505 as necessary, and a removable medium 521 made up of a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is appropriately mounted, and a computer program read therefrom is It is installed in the storage unit 508 as necessary.

  When a series of processing is executed by software, a program constituting the software is installed from a network or a recording medium.

  As shown in FIG. 67, this recording medium is not only composed of a package medium consisting of a removable medium 521 on which a program is recorded, which is distributed to provide a program to the user, separately from the computer. These are configured by a hard disk including a ROM 502 storing a program and a storage unit 508 provided to the user in a state of being pre-installed in the apparatus main body.

  In the present specification, the step of describing a computer program includes not only processing performed in time series according to the described order but also processing executed in parallel or individually even if not necessarily processed in time series. Is also included.

  Further, the drive 510 can read data recorded on the attached removable medium 521 and can record data on the attached removable medium 521. The personal computer 500 has the same function as the software production processing unit described with reference to FIG. 66 (for example, the CPU 501 is used to execute a program for realizing the same function as the software production processing unit). It goes without saying that it is possible.

  That is, the personal computer 500 generates data similar to the data generated by the software production processing unit described with reference to FIG. 66 by the processing of the CPU 501 or uses FIG. 66 generated by an external device. Data similar to the data generated by the software production processing unit described above can be acquired via the communication unit 509 or the removable medium 521 attached to the drive 510. Then, the personal computer 500 serves as a recording device that records data similar to the data generated or acquired by the software production processing unit described with reference to FIG. 66 on the removable medium 521 attached to the drive 510. Function can be realized.

  The embodiment of the present invention is not limited to the above-described embodiment, and various modifications can be made without departing from the gist of the present invention.

  20 playback devices, 31 storage drives, 32 switches, 33 AV decoder units, 34 controllers, 51 to 54 buffers, 55, 56 PID filters, 57 to 59 switches, 71 background decoders, 72 video decoders / first video decoders, 73 presentations Graphics decoder, 74 interactive graphics decoder, 75 audio decoder, 76 Text-ST composition, 77 switch, 91 background plane generator, 92 video plane generator / 1st video plane generator, 93 presentation graphics plane generator , 94 Interactive graphics plane generator, 95 Buffer 96 Video data processor, 97 Mixi Grayed processing unit, 101 a mixing processing unit 401 reproducing apparatus, 403 AV decoder unit, 411 PID filter, 412 2nd video decoder, 413 2nd video plane generating unit, 414 preload buffer

Claims (3)

Different from the main reproduction path indicating the position of the main stream group on the time axis, the first sub reproduction path indicating the position of the first substream group on the time axis, and the first substream group The second sub-stream group includes a second sub-reproduction path indicating the position on the time axis, and is classified into a predetermined type of a plurality of types predefined according to the type of the main stream group. A first sub-reproduction path and a second sub-reproduction path, each of which is a type of reproduction path permitted for the predetermined type. Obtaining means for obtaining the reproduction management information satisfying at least a condition;
Based on the reproduction management information acquired by the acquisition means, the selection of a stream to be reproduced is performed by a first combination of the main stream group and the first substream group, or the main stream group and the first stream. Receiving means for accepting the second combination with the two substream groups;
When the first combination is received by the receiving unit, the main stream group is read with reference to the main reproduction path, and the first substream group is referred to the first sub reproduction path. When the second combination is received by the receiving unit, the main stream group is read with reference to the main reproduction path, and the second substream group is read by the second sub stream. Read means for reading with reference to the reproduction path;
Reproducing means for reproducing the one of the first substream group and the second substream group read by the reading means together with the main stream group read by the reading means. ,
The types of reproduction management information include a synchronous type and an asynchronous type,
The first condition is that when the type of the reproduction management information is the asynchronous type, the type of the secondary reproduction path is a non-multiplex type path, and is an audio presentation path of a browsable slide show. Type or non-multiplexed path, which is either the second type which is an interactive graphics menu, and the type of the playback management information is the synchronous type, the type of the secondary playback path Is a condition that the type is other than the first type,
The main stream group, the first substream group, and the second substream group are included in any file,
The number of reading of the file that can be read by the reading means at a time is predetermined,
In addition to the first condition, the reproduction management information further determines one or more secondary reproduction paths so that the total number of the files read at one time is equal to or less than the read number. A playback device that meets the requirements. A recording medium on which data including reproduction management information for managing reproduction of a file including at least one stream is recorded,
The reproduction management information includes a main reproduction path indicating a position on the time axis of the main stream group, a first sub reproduction path indicating a position on the time axis of the first substream group, and the first A second sub-reproduction path indicating a position on the time axis of a second sub-stream group different from the sub-stream group, and a plurality of types defined in advance according to the type of the main stream group Classified into a given type,
Each of the first sub reproduction path and the second sub reproduction path satisfies at least a condition that the reproduction path is a type permitted for the predetermined type;
The types of reproduction management information include a synchronous type and an asynchronous type,
The condition is that when the type of the playback management information is the asynchronous type, the secondary playback path type is a non-multiplexed path and the audio presentation path of the browsable slide show is the first type, Alternatively, if the path is a non-multiplexed path and one of the second types that is an interactive graphics menu, and the type of the playback management information is the synchronous type, the secondary playback path type is It is a condition that it is a type other than the first type,
The main stream group, the first substream group, and the second substream group are included in any file, and the total number of the files to be read at a time is determined in advance in the playback device. A recording medium on which data for determining the first sub reproduction path and the second sub reproduction path is recorded so that the number of data can be read at one time or less. A method of manufacturing a storage medium on which data that can be played back by a playback device is recorded,
The reproduction management information includes a main reproduction path indicating a position on the time axis of the main stream group, a first sub reproduction path indicating a position on the time axis of the first substream group, and the first A second sub-reproduction path indicating a position on the time axis of a second sub-stream group different from the sub-stream group, and a plurality of types defined in advance according to the type of the main stream group Classified into a given type,
Each of the first sub reproduction path and the second sub reproduction path satisfies at least a condition that the reproduction path is a type permitted for the predetermined type;
The types of reproduction management information include a synchronous type and an asynchronous type,
The condition is that when the type of the playback management information is the asynchronous type, the secondary playback path type is a non-multiplexed path and the audio presentation path of the browsable slide show is the first type, Alternatively, if the path is a non-multiplexed path and one of the second types that is an interactive graphics menu, and the type of the playback management information is the synchronous type, the secondary playback path type is It is a condition that it is a type other than the first type,
The main stream group, the first substream group, and the second substream group are included in any file, and the total number of the files to be read at a time is determined in advance in the playback device. In addition, data having a data structure for determining the first sub reproduction path and the second sub reproduction path is generated so as to be equal to or less than the number of readouts that can be read at one time,
A method for manufacturing a recording medium, comprising the step of recording the generated data on a recording medium.

Images