JP2008193604A - Reproducing apparatus and method, and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To synthesize audio data as needed, and to prevent audio data from being deteriorated. <P>SOLUTION: A controller 34 appropriately controls a switch 45 and analyzes an encoded audio stream of a reproduction object acquired by a reproduction data acquisition section 32 to strictly determine presence/absence of the effect of re-encoding and performs control so as to avoid re-encoded audio stream from being output to the outside, as much as possible. Namely, if there is the effect of re-encoding, with respect to a non-compressed audio signal after audio synthesis processing from a mixing processing section 97, a re-encoded audio stream obtained as a result of re-encoding by an audio encoder 41 is output to the outside. In other cases, the non-compressed audio signal after audio synthesis processing from the mixing processing section 97 or an original encoded audio stream from a switch 101 is output to the outside. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to a playback apparatus, method, and program, and more particularly, to a playback apparatus, method, and program suitable for use when audio data to be played back can be synthesized.

  In order to view different contents simultaneously, after decoding video data of multiple contents, uncompressed image data is synthesized, then D / A conversion is performed and output to the video output terminal. There is a technique for displaying on a display device (for example, Patent Document 1).

Japanese Patent Laying-Open No. 2005-20242

  As described in Patent Document 1 described above, in order to synthesize video data, it is necessary to decode each compression-coded video data and synthesize uncompressed video data. This applies to audio data as well as video data. When audio data is synthesized, uncompressed audio data must be used.

  Depending on the function of the output destination device or the connection method with the output destination device, the output data may have to be encoded data.

  The processing when the other audio data is synthesized with the audio data mainly output and the output audio data is encoded data will be described with reference to FIG.

  The first audio data acquisition unit 11 acquires, for example, compression-encoded first audio data read and supplied from the optical disc, and supplies the first audio data to the decoding unit 12. It is assumed that the first audio data is audio data that is always reproduced and output in the content. The decoding unit 12 decodes the compression-coded first audio data supplied from the first audio data acquisition unit 11 and supplies the uncompressed first audio data to the synthesis unit 14.

  The second audio data acquisition unit 13 acquires uncompressed second audio data and supplies it to the synthesis unit 14. The synthesizer 14 receives the supply of uncompressed second audio data from the second audio data acquisition unit 13, and converts the second audio data into the uncompressed first audio data supplied from the decoder 12. The synthesized voice data is supplied to the encoding unit 15.

  The encoding unit 15 encodes the supplied audio data and supplies the encoded audio data to the digital interface 16. The digital interface sends the supplied encoded data to another device or the like via a predetermined network.

  This uncompressed second audio data may or may not be prepared for the content. That is, depending on the content, the second audio data acquisition unit 13 is configured not to acquire the uncompressed second audio data, or the second audio data is prepared only for a predetermined part of the content. In some cases, the second sound data acquisition unit 13 is configured to acquire uncompressed second sound data of only a predetermined portion. When the synthesis unit 14 does not receive the supply of uncompressed second audio data from the second audio data acquisition unit 13, the synthesis unit 14 sends the uncompressed first audio data supplied from the decoding unit 12 to the encoding unit 15. Supply.

  In the apparatus described with reference to FIG. 1, the first audio data mainly reproduced and output can be reproduced and output both the content synthesized with the second audio data and the content not synthesized. If it is possible to synthesize the second audio data only for a predetermined part of the same content, conventionally, the second audio data is synthesized in the reproduction output device. It was not possible to detect whether or not the second audio data exists.

  Therefore, the first audio data is decoded in the same manner as the case where the second audio data is synthesized even if the content is not synthesized with the second audio data or the portion where the second audio data is not synthesized. And re-encoded. Therefore, the sound quality of the first sound data is always deteriorated due to decoding and re-encoding.

  The present invention has been made in view of such a situation, and is intended to realize synthesis of audio data as necessary and to prevent deterioration of the audio data as much as possible.

  An image processing apparatus according to an aspect of the present invention includes a reproduction data acquisition unit that acquires reproduction data including an encoded audio stream, and a decoding that performs a decoding process on the encoded audio stream acquired by the reproduction data acquisition unit Means for synthesizing the uncompressed audio signal obtained as a result of the decoding process by the decoding means with synthesis data that is data to be synthesized with the uncompressed audio signal; Re-encoding means for performing re-encoding processing on the synthesized non-compressed audio signal, and the encoded audio stream to be reproduced exists in the reproduction data acquired by the reproduction data acquisition means, and is synthesized The first data in which the data for synthesis exists and the user permits or instructs the synthesis , Allowing the output of the re-encoded audio stream obtained as a result of the re-encoding process by the re-encoding means, and outputting the re-encoded audio stream to the outside in the second case other than that. Control means for prohibiting.

  In the second case, the control means also permits the output of the original encoded audio stream or the uncompressed audio signal synthesized with the synthesis data to the outside.

  The control means further determines the number of channels included in the encoded audio stream to be reproduced, and even in the first case described above, the number of channels determined by the external connection transmission path and the external device is determined. In the third case corresponding to transmission or reception of an uncompressed audio signal, output of the re-encoded audio stream is prohibited.

  The control means further permits the output of the uncompressed audio signal synthesized with the synthesis data to the outside in the first case and the third case.

  The control means, as at least a part of the determination process of whether the first case or the second case, the interactive graphics stream included in the reproduction data acquired by the reproduction data acquisition means Based on the data analysis result, it is determined whether or not the data for synthesis to be synthesized with the encoded audio stream to be reproduced exists.

  The control means further recognizes a synthesized part in which the synthesis data is synthesized with the uncompressed audio signal by the synthesizing means, and corresponds to the synthesized part of the encoded audio stream to be reproduced. The part to be output is output in the form of the re-encoded audio stream, and the other part is output in the form of the original encoded audio stream.

  Based on the control of the control means, in the first case, the output to the outside is switched to the re-encoded audio stream, and in the second case, the output to the outside is switched to the original encoded audio stream, Alternatively, a switching means for switching to the uncompressed audio signal synthesized with the synthesis data is further provided.

  An image processing method and program according to one aspect of the present invention are a method and program corresponding to the above-described image processing apparatus according to one aspect of the present invention.

  In the image processing apparatus and method, and the program according to one aspect of the present invention, reproduction data acquisition means for acquiring reproduction data including an encoded audio stream, and the encoded audio stream acquired by the reproduction data acquisition means Decoding means for performing decoding processing, combining means for combining data for combining with an uncompressed audio signal obtained as a result of the decoding processing by the decoding means, and data for combining The following processing is executed for a playback apparatus including re-encoding means for performing re-encoding processing on the uncompressed audio signal combined with data. In other words, the encoded audio stream to be reproduced exists in the reproduction data acquired by the reproduction data acquisition means, the combination data to be combined there exists, and the user permits or instructs the combination In the first case, output of the re-encoded audio stream obtained as a result of the re-encoding process by the re-encoding means is permitted, and in the second case other than that, the re-encoded audio is output. Output outside the stream is prohibited.

  As described above, according to the first aspect of the present invention, it is possible to reproduce audio (audio) data. In particular, it realizes synthesis of audio data as necessary, and re-encoded audio data (re-encoding). The audio data is prevented from being deteriorated as much as possible by controlling whether or not the (audio stream) can be output.

  Embodiments of the present invention will be described below. Correspondences between the configuration requirements of the present invention and the embodiments described in the detailed description of the present invention are exemplified as follows. This description is to confirm that the embodiments supporting the present invention are described in the detailed description of the invention. Accordingly, although there are embodiments that are described in the detailed description of the invention but are not described here as embodiments corresponding to the constituent elements of the present invention, It does not mean that the embodiment does not correspond to the configuration requirements. Conversely, even if an embodiment is described here as corresponding to a configuration requirement, that means that the embodiment does not correspond to a configuration requirement other than the configuration requirement. It's not something to do.

  Further, this description does not mean that all the inventions corresponding to the specific examples described in the embodiments of the invention are described in the claims. In other words, this description is an invention corresponding to the specific example described in the embodiment of the invention, and the existence of an invention not described in the claims of this application, that is, in the future, a divisional application will be made. Nor does it deny the existence of an invention added by amendment.

An information processing apparatus according to one aspect of the present invention (for example, the playback apparatus 20 in FIG. 37)
Reproduction data acquisition means (for example, reproduction data acquisition unit 31 in FIG. 37) for acquiring reproduction data including an encoded audio stream;
Decoding means (for example, 1st audio decoder 75-1 in FIG. 37) for performing decoding processing on the encoded audio stream acquired by the reproduction data acquisition means;
A synthesizing unit (for example, mixing processing units 102 and 97 in FIG. 37) for synthesizing synthesis data that is data to be synthesized with the uncompressed audio signal obtained as a result of the decoding process by the decoding unit;
Re-encoding means (for example, audio encoder 41 in FIG. 37) for performing re-encoding processing on the uncompressed audio signal synthesized with the synthesis data by the synthesis means;
The encoded audio stream to be reproduced exists in the reproduction data acquired by the reproduction data acquisition means, the combining data to be combined with the encoded audio stream exists, and the user permits or instructs the combining In the first case, output of the re-encoded audio stream obtained as a result of the re-encoding process by the re-encoding unit is permitted, and in the second case other than that, the re-encoded audio stream Control means for prohibiting output to the outside (for example, the controller 34 in FIG. 37 for executing the audio signal reproduction processing in FIG. 38).

  The control means further permits the output of the original encoded audio stream or the uncompressed audio signal synthesized with the synthesis data to the outside in the second case (for example, step of FIG. 38). (S3, S8, S15, etc. are executed).

  The control means further determines the number of channels included in the encoded audio stream to be reproduced, and even in the first case described above, the number of channels determined by the external connection transmission path and the external device is determined. In the third case corresponding to transmission or reception of an uncompressed audio signal (for example, when it is determined YES in step S18 of FIG. 38), the output of the re-encoded audio stream is prohibited. To do.

  The control means further permits the output of the uncompressed audio signal synthesized with the synthesis data to the outside in the first case and the third case (for example, in step S20 of FIG. 38). Process).

  The control means, as at least a part of the determination process of whether the first case or the second case, the interactive graphics stream included in the reproduction data acquired by the reproduction data acquisition means Based on the analysis result of the data (for example, the data generated by the interactive graphics plane generating unit 94 in FIG. 37), it is determined whether or not the synthesis data to be synthesized with the encoded audio stream to be reproduced exists. to decide.

  The control means further recognizes a synthesized part in which the synthesis data is synthesized with the uncompressed audio signal by the synthesizing means, and corresponds to the synthesized part of the encoded audio stream to be reproduced. The portion to be output is output in the form of the re-encoded audio stream obtained as a result of the re-encoding process by the re-encoding unit (for example, output as the section 221 in FIG. 40), and other portions are The encoded audio stream is output as it is (for example, output as sections 220 and 222 in FIG. 40).

  Based on the control of the control means, in the first case, the output to the outside is switched to the re-encoded audio stream, and in the second case, the output to the outside is switched to the original encoded audio stream, Alternatively, it further includes switching means (for example, switch 45 in FIG. 37) for switching to the uncompressed audio signal synthesized with the synthesis data.

A playback method according to one aspect of the present invention includes:
Reproduction data acquisition means (for example, reproduction data acquisition unit 31 in FIG. 37) for acquiring reproduction data including an encoded audio stream;
Decoding means (for example, 1st audio decoder 75-1 in FIG. 37) for performing decoding processing on the encoded audio stream acquired by the reproduction data acquisition means;
A synthesizing unit (for example, mixing processing units 102 and 97 in FIG. 37) for synthesizing synthesis data that is data to be synthesized with the uncompressed audio signal obtained as a result of the decoding process by the decoding unit;
Reproduction means (for example, reproduction of FIG. 37), comprising: re-encoding means (for example, audio encoder 41 of FIG. 37) for performing re-encoding processing on the uncompressed audio signal synthesized with the synthesis data by the synthesis means. In the playback method of the apparatus 20),
The encoded audio stream to be reproduced exists in the reproduction data acquired by the reproduction data acquisition means, the combining data to be combined with the encoded audio stream exists, and the user permits or instructs the combining In the first case, output of the re-encoded audio stream obtained as a result of the re-encoding process by the re-encoding unit is permitted, and in the second case other than that, the re-encoded audio stream This includes a step of prohibiting output to the outside (for example, each step of the audio signal reproduction process of FIG. 38 executed by the controller 34 of FIG. 37).

  A program according to one aspect of the present invention is a program corresponding to the information processing method according to one aspect of the present invention described above, and is executed by, for example, the computer 500 in FIG.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

  A reproducing apparatus to which the present invention is applied will be described with reference to FIG.

  For example, the playback device 20 may be connected to information recorded on a recording medium 21 such as an optical disc, information supplied via the network 22, or a recording medium (for example, an HDD) within itself. The recorded information can be reproduced. The reproduction device 20 can supply the reproduced data to a display / audio output device 23 connected by wire or wirelessly to display an image and output sound. The playback device 20 can also send the played back data to other devices via the network 22. For example, the playback device 20 can receive an operation input from the user by an input device such as a button provided in the main body in addition to the remote commander 24.

  The recording medium 21 records data including video and audio that can be played back by the playback device 20 in a predetermined format. The recording medium 21 may be, for example, a magnetic disk or a semiconductor memory in addition to the optical disk.

  When the display / audio output device 23 receives digital uncompressed data, the playback device 20 decodes the encoded data recorded on the recording medium 21 or the like, and outputs the uncompressed data. When the display / audio output device 23 has a decoding function and receives compressed data input, the playback device 20 uses the compressed data. Is supplied to the display / audio output device 23. Further, when the display / audio output device 23 is configured to receive input of uncompressed analog data, the playback device 20 decodes the encoded data recorded in the recording medium 21 or the like to perform non-compression. Data is D / A converted to generate an analog signal and supplied to the display / audio output device 23. The playback device 20 also plays back data recorded on the recording medium 21 and sends it to the network 22 in a compressed data state.

  3 shows information that can be played back by the playback device 20 of FIG. 2, that is, data recorded on the recording medium 21 attached to the playback device 20, data supplied via the network 22, or playback device 20 It is a figure which shows the example of the application format of the data currently recorded on the internal recording medium.

  The application format has two layers, PlayList and Clip, for managing AV (Audio Visual) streams. Here, a pair of one AV stream and Clip information as information accompanying the AV stream is considered as one object, and these are collectively referred to as a Clip. Hereinafter, the AV stream is also referred to as an AV stream file. Clip information is also referred to as a Clip information file.

  In general, a file used in a computer or the like is handled as a byte string, but the content of an AV stream file is expanded on the time axis, and an access point of a clip is mainly designated by a PlayList with a time stamp. That is, PlayList and Clip are layers for AV stream management.

  When the access point in the Clip is indicated by the PlayList with a time stamp, the Clip Information file is used to find address information to start decoding in the AV stream file from the time stamp.

  The PlayList is a collection of AV stream playback sections. One playback section in a certain AV stream is called PlayItem, which is represented by a pair of an IN point (playback start point) and an OUT point (playback end point) of the playback section on the time axis. Accordingly, the PlayList is composed of one or a plurality of PlayItems as shown in FIG.

  In FIG. 3, the first PlayList from the left is composed of two PlayItems, and the two PlayItems refer to the first half and the second half of the AV stream included in the left Clip, respectively. Also, the second PlayList from the left is composed of one PlayItem, whereby the entire AV stream included in the right Clip is referenced. Further, the third PlayList from the left is composed of two PlayItems, and the two PlayItems refer to a portion of the AV stream included in the left Clip and a portion of the AV stream included in the right Clip, respectively. ing.

  The navigation program has a function of controlling the playback order of the PlayList and interactive playback of the PlayList. The navigation program also has a function of displaying a menu screen for the user to instruct execution of various reproductions. This navigation program is described in a programming language such as Java (registered trademark).

  For example, when the left PlayItem included in the first PlayList from the left is specified as information representing the playback position at that time by the navigation program of FIG. 3, the AV included in the left Clip referred to by the PlayItem The first half of the stream is played back. Thus, the PlayList is used as playback management information for managing playback of AV stream files.

  The navigation program includes an index table and a navigation object that is read by the index table. The index table and the navigation object will be described with reference to FIG.

  The index table defines content titles and menus, and holds entry points for each title and menu. FirstPlayback includes information related to the NavigationObject that is automatically executed first when the index table is read, such as when the recording medium 21 on which this data is recorded is loaded in the playback device 20. In TopMenu, the NavigationObject of TopMenu that is called when the user displays a playback menu screen that includes content items such as playing all content, playing only a specific chapter, playing repeatedly, displaying the initial menu, etc. Contains information about. Each Title includes information on a NavigationObject that can be assigned and called for each title distinguished by a predetermined Title ID. In FIG. 4, there is one navigation command per title.

  The navigation object consists of a navigation command that is an executable command. The Navigation Command includes, for example, various commands such as PlayList playback and calling other Navigation Objects. For example, when Navigation Command # 3 includes a command statement for playing PlayList # 1, when Navigation Command # 3 is executed, PlayList # 1 is played.

  The Index file that is a data file including the index table will be described later with reference to FIGS.

  In this embodiment, a playback path created by a sequence of one or more PlayItems (by successive PlayItems) in the PlayList is called a main path (Main Path), and in the PlayList, parallel to the Main Path (in parallel) ) And a playback path created by a sequence of one or more sub paths (which may be discontinuous or may be continuous) is referred to as a sub path. In other words, the application format of data that can be played back by the playback device 20 has a sub-path (Sub Path) that is played back in association with (in association with) the main path in the PlayList.

  FIG. 5 is a diagram for explaining the structure of the main path and the sub path. The PlayList can have one main path and one or more sub paths. One main path is created by a sequence of one or more PlayItems, and one sub-path is created by a sequence of one or more SubPlayItems.

  In the case of the example in FIG. 5, the PlayList has one main path made up of a sequence of three PlayItems and three sub paths. Each PlayItem constituting the main path is assigned ID (Identification) in order from the top. Specifically, the main path is made up of PlayItems with PlayItem_id = 0, PlayItem_id = 1, and PlayItem_id = 2. Subpaths are also assigned IDs in order from the top, Subpath_id = 0, Subpath_id = 1, and Subpath_id = 2. The subpath with Subpath_id = 0 includes one SubPlayItem, the subpath with Subpath_id = 1 includes two SubPlayItems, and the subpath with Subpath_id = 2 includes one SubPlayItem.

  The stream referred to by the SubPlayItem included in the subpath of Subpath_id = 0 is assumed to be, for example, a Japanese dubbing sound of a movie, and may be played in place of the audio stream of the AV stream file referenced by Mainpath. Also, the stream referred to by SubPlayItem included in the subpath of Subpath_id = 1 is assumed to be a director's cut of a movie, for example, and a comment such as a movie director is included only in a predetermined part of the AV stream file referred to by Main Path. The case is considered.

  A Clip AV stream file referred to by one PlayItem includes at least video stream data (main image data). In addition, the Clip AV stream file may or may include one or more audio streams that are played back at the same timing (synchronously) as the video stream (main image data) included in the Clip AV stream file. It does not have to be. Furthermore, the Clip AV stream file may or may not include one or more bitmap subtitle streams that are played back at the same timing as the video stream included in the Clip AV stream file. The Clip AV stream file may or may not include one or more interactive graphics streams that are played back at the same timing as the video stream included in the Clip AV stream file. The video stream included in the Clip AV stream file and the audio stream, bitmap subtitle stream file, or interactive graphics stream reproduced at the same timing as the video stream are multiplexed. That is, a Clip AV stream file referred to by one PlayItem includes video stream data, zero or more audio streams to be played in accordance with the video stream, zero or more bitmap subtitle stream data, and zero or more. The interactive graphics stream data is multiplexed.

  That is, a Clip AV stream file referred to by one PlayItem includes a plurality of types of streams such as a video stream, an audio stream, a bitmap subtitle stream file, or an interactive graphics stream.

Also, one SubPlayItem refers to audio stream data and subtitle data of a stream (another stream) that is a Clip AV stream file referred to by the PlayItem.

  When playing a PlayList that has only the main path, the user can select audio and subtitles only from the audio stream and sub-picture stream multiplexed in the clip referenced by the main path. Can not do it. On the other hand, when playing a PlayList having a main path and a sub path, in addition to the audio stream and sub picture stream multiplexed in the Clip AV stream file referenced by the main path, the clip audio stream referenced by the SubPlayItem Or a sub-picture stream.

  In this way, since a plurality of SubPaths are included in one PlayList and each SubPath refers to each SubPlayItem, an AV stream with high expandability and high flexibility can be realized. In other words, in addition to the Clip AV stream referred to by MainPath, a SubPlayItem can be added later.

  FIG. 6 is a diagram illustrating an example of a main path and a sub path. In FIG. 6, an audio playback path that is played back at the same timing as the main path (synchronized with AV) is represented using a sub path.

  The PlayList in FIG. 6 includes one PlayItem with PlayItem_id = 0 as a main path and one SubPlayItem as a sub path. One PlayItem () with PlayItem_id = 0 in the main path refers to the main AV stream in FIG. SubPlayItem () includes the following data. First, SubPlayItem () includes Clip_Information_file_name for designating a Clip referred to by a Sub Path in the PlayList. In the example of FIG. 6, an auxiliary audio stream (audio stream) with SubClip_entry_id = 0 is referred to by SubPlayItem. Also, SubPlayItem () includes SubPlayItem_IN_time and SubPlayItem_OUT_time for specifying the playback section of the Sub Path in the stream (Auxiliary audio stream in this case) included in the specified Clip. Further, SubPlayItem () includes sync_PlayItem_id and sync_start_PTS_of_PlayItem for specifying the time at which the Sub Path starts to be reproduced on the time axis of the Main path. In the case of the example in FIG. 6, sync_PlayItem_id = 0 and sync_start_PTS_of_PlayItem = t1. As a result, it is possible to specify the time t1 at which the Sub Path starts playback on the time axis of PlayItem_id = 0 of the main path. That is, in the case of the example in FIG. 6, the reproduction start time t1 of the main path and the start time t1 of the sub path are the same time.

  Here, the Clip AV stream of the audio referred to by the Sub Path must not include STC discontinuities (system time base discontinuities). The clock of the audio sample of the stream included in the Clip used for the sub path is locked to the clock of the audio sample of the stream included in the Clip used for the main path.

  In other words, in SubPlayItem (), information specifying the Clip referred to by the Sub Path, information specifying the playback section of the Sub Path, and information specifying the time when the Sub Path starts playback on the time axis of the Main path It is included. Since the Clip AV stream used in Sub Path does not include STC, information included in SubPlayItem () (information specifying Clip referred to by Sub Path, information specifying Sub Path playback section, and time axis of Main path) Based on the above (information specifying the time when the Sub Path starts playback), the audio stream of the Clip AV stream different from the Clip AV stream (main AV stream) referred to by the main path can be referred to and reproduced. it can.

  In this way, PlayItem and SubPlayItem manage Clip AV stream files, respectively. Here, the Clip AV stream file (main AV stream) managed by PlayItem and the Clip AV stream file managed by SubPlayItem are different from each other. Become.

  Similar to the example of FIG. 6, a subtitle stream playback path that is played back at the same timing as the main path can be expressed using a sub path.

  FIG. 7 is a diagram illustrating another example of the main path and the sub path. In FIG. 7, an audio playback path that is played back at the same timing as the main path (synchronized with AV) is represented using a sub path. Here, the main AV stream file referred to by PlayItem in the main path is the same as in FIG.

  For example, a clip AV stream referred to by the main path is set as one movie content (AV content), an auxiliary audio stream referenced by the sub-path audio path is set as a director's comment for the movie, and a clip AV stream referred to by the main path is used. Such a configuration is used when the auxiliary audio stream referred to in the sub-path audio path is mixed (overlaid) with the audio stream. That is, when the user inputs a command to listen to the director's comment together with the movie while watching the movie, for example, the audio of the Clip AV stream referenced in the main path and the audio path of the sub path This is used when the Auxiliary audio stream referenced in the above is mixed and played back.

  In FIG. 7, three PlayItems are arranged in the main path as PlayItem_id = 0, 1, 2, and two SubPlayItems are arranged in the subpath (Subpath_id = 0). SubPlayItem (described later in FIG. 21) called by SubPath with Subpath_id = 0 (described later in FIG. 19) is an Auxiliary audio stream (English Auxiliary audio stream clip with SubClip_entry_id = 0 and SubClip_entry_id = 1 Japanese) SubPlayItem_IN_time and SubPlayItem_out_time for specifying the subpath playback section of (Auxiliary audio stream clip).

  7 is compared with FIG. 6, in FIG. 7, an auxiliary audio stream (English or Japanese audio stream) of SubClip_entry_id = 0, 1 can be referred to by SubPlayItem. That is, it has a structure that refers to a plurality of audio stream files using SubPlayItem, and when reproducing this SubPlayItem, one audio stream file is selected from the plurality of audio stream files and reproduced. In the case of the example in FIG. 7, one audio stream file is selected and reproduced from the English audio stream file and the Japanese audio stream file. Specifically, one of subClip_entry_id = 0 and 1 is selected (based on a user instruction), and an auxiliary audio stream referred to by the ID is reproduced. In addition to this, when playback by mixing with an audio stream referred to by the main path is selected (when two audio streams are selected as audio streams to be played back), for example, the playback is referred to by the main path. The audio stream file to be referred to and the audio stream file referred to by the sub-path audio path are mixed and reproduced.

  FIG. 8 is a diagram illustrating an example of a file system of a data file that can be played back by the playback device 20. FIG. 8 shows, as an example, a case where a data file that can be played back by the playback device 20 is supplied by a recording medium 21 such as an optical disk, and this file system has a directory structure.

  In this file system, a directory with the name “BDMV” is prepared under “root”, and a file with the name “Index.bdmv” and a file “NavigationObject.bdmv” Contains a file with a name. Hereinafter, these files will be referred to as an Index file and a NavigationObject file, respectively, as appropriate. As appropriate, each file is referred to by adding “file” to “file name”, or each directory is referred to by adding “directory” to “directory name”.

  The Index file is a file in which the above-described index table is described, and includes information related to a menu for reproducing a data file that can be reproduced by the reproducing apparatus 20. The playback device 20 includes, for example, a playback menu that includes items such as all content included in a data file that can be played back by the playback device 20, playback of only a specific chapter, repeated playback, and display of an initial menu. The screen is displayed on the display device based on the Index file. The navigation object to be executed when each item is selected can be set in the index table of the index file. When one item is selected from the playback menu screen by the user, the playback device 20 stores the navigation object in the index file index table. Execute the set NavigationObject command.

  The NavigationObject file is a file containing NavigationObject. The NavigationObject includes a command for controlling playback of a PlayList included in a data file that can be played back by the playback device 20. For example, the playback device 20 selects one of the NavigationObjects included in the file system. Then, the content can be reproduced by executing.

  The BDMV directory also includes a directory with the name “BACKUP” (BACKUP directory), a directory with the name “PLAYLIST” (PLAYLIST directory), and a directory with the name “CLIPINF” (CLIPINF directory). , A directory (STREAM directory) in which the name “STREAM” is set, and a directory (AUXDATA directory) in which the name “AUXDATA” is set.

  In the BACKUP directory, files and data for backing up files and data that can be played back by the playback device 20 are recorded.

  A PlayList file is stored in the PLAYLIST directory. As shown in the figure, each PlayList file is given a name obtained by adding an extension “.mpls” to a file name consisting of a 5-digit number.

  A Clip Information file is stored in the CLIPINF directory. Each Clip Information file is named by adding the extension “.clpi” to a file name consisting of five digits as shown in the figure.

  In the STREAM directory, a Clip AV stream file and a substream file are stored. Each stream file is given a name obtained by adding an extension “.m2ts” to a file name consisting of five digits as shown in the figure.

  The AUXDATA directory is not included in the Clip AV stream file or substream file, but is referred to from the Clip AV stream file or substream file, or data used independently of the Clip AV stream file or substream file. And other files are stored. In the example of FIG. 8, the AUXDATA directory stores sound data such as subtitle font files with the name “11111.otf” and sound effects with the name “sound.bdmv”. ing.

  Further, when a data file that can be played back by the playback device 20 is distributed by an optical disk, for example, the content author such as a production company or a movie distribution company, or the title author that is the supply source of the recording medium is displayed. Author_id, which is an identifier assigned to each title author for identification, and disc_id, which is an identifier assigned to identify the type of optical disc produced by the title author indicated by author_id, are rewritten by the user or the like. It cannot be recorded as secure electronic data or physically by pits.

  In addition, even when a data file that can be played back by the playback device 20 is recorded on a removable recording medium other than the optical disk or downloaded via a network, an id corresponding to author_id and author_id is assigned. It is preferable to have a directory structure similar to that shown in FIG. Further, even if the data file that can be played back by the playback device 20 is not assigned author_id and id corresponding to author_id, the name of “Index.bdmv” is the same as in the case described with reference to FIG. , Files with the name “NavigationObject.bdmv”, files with the name “BACKUP”, files with the name “PLAYLIST”, “CLIPINF Among the file group set with the name "STREAM", the file group set with the name "STREAM", and the file group set with the name "AUXDATA", necessary file groups are included as appropriate.

  By the way, the reproducing apparatus 20 outputs audio data as uncompressed data and outputs it as compressed and encoded data. Further, the playback device 20 can synthesize audio data as necessary. For example, the playback device 20 can have a function of synthesizing sound effects mainly on sound data to be reproduced and output. In addition, not only the sound effects but also the second sound (sub sound, etc.) is mainly used. It is possible to have a function of synthesizing the audio data reproduced and output. In other words, whether or not the playback device 20 synthesizes audio data depends on whether or not the stream to be played back includes audio data that can be synthesized by the function of the playback device 20.

  Hereinafter, the audio data mainly played back and output is referred to as audio stream # 1, and is described as sound.bdmv in the AUXDATA directory described above, for example, a click sound or a sound effect generated in response to a user operation input. The audio data is referred to as sound data. Sound data can be synthesized with the audio stream # 1. Also, audio data that is different from the audio stream # 1 and can be synthesized with the audio stream # 1, such as sub audio, is referred to as an audio stream # 2. In addition to the audio stream # 2, there may be a plurality of audio data that is different from the audio stream # 1 and can be combined with the audio stream # 1, and in this case, for example, the audio stream # 1 3, audio stream # 4, etc., can be distinguished from each other.

  Even when audio stream # 1 is output as compression-encoded data, when audio synthesis with audio stream # 2 or sound data is performed, audio stream # 1 is once decoded and synthesized. After that, it is re-encoded and output. On the other hand, when the audio stream # 1 is output as compression-encoded data and the voice synthesis with the audio stream # 2 or the sound data is not performed, the audio stream # 1 should be output as it is without being decoded. Sound quality is not deteriorated, which is preferable.

  Therefore, in the playback apparatus 20, in order to determine whether or not to perform the decoding process on the audio stream # 1, the output form and the type of audio data (audio stream # 1) included in the data file are determined. It is necessary to recognize whether or not other audio data to be synthesized is included. For this reason, the data file that can be played back by the playback device 20 includes a flag indicating whether or not sound data is included and a flag indicating whether or not the audio stream # 2 is included in a predetermined position. ing.

  Note that the position where the flag indicating whether or not other audio data to be synthesized is included in the audio stream # 1 is described as an index file, a playlist (PlayList) file, a play item (PlayItem), or the like. is there. When these flags are described in the Index file, when all data included in the data structure described with reference to FIG. 8 is reproduced, other audio data to be synthesized with the audio stream # 1 is included. Whether it is treated or not included is specified. Similarly, when these flags are described in the playlist, it includes that other audio data synthesized with the audio stream # 1 included in the data reproduced based on the corresponding playlist is included. If these flags are described in the play item, other clips that are combined with the audio stream # 1 are combined with the clip corresponding to the corresponding play item. It is specified whether audio data is included or not.

  FIG. 9 shows the syntax of the Index file (Index.bdmv).

  Type_indicator is a field in which the value of “INDX” encoded based on ISO646 is described.

  The version_number is a 4-character string indicating the version number of Index.bdmv, and a value “0089” encoded based on ISO646 is described.

  Indexes_start_address is a field in which the start address of the Indexes () block is described.

  In AppInfoBDMV (), an individual identifier indicating the supply source of the data file including this Index file is described.

  Indexes () includes, for example, a recording device 21 such as an optical disk on which a menu is read, a title is searched, a jump is made to a predetermined title, or a data file including the Index file is recorded. A link with an application (NavigationObject) that executes a process that automatically starts when it is attached to the device 20 is described. The syntax of Indexes () will be described later with reference to FIGS. 10 to 12.

  Padding_word is inserted based on the syntax of Index.bdmv, and is represented by 0 or a positive integer.

  A first example of the syntax of Indexes () is shown in FIG. FIG. 10 shows the syntax when the flag referred to by Indexes () does not contain a flag indicating whether other audio data to be combined with the audio stream # 1 is included in the data referred to by Indexes ().

  The length is a field indicating how many bytes of information are described in Indexes ().

  FirstPlayback is a data block that includes information related to a NavigationObject that is executed first when data of this file system is reproduced. Specifically, it is a data block that includes information related to a NavigationObject that is automatically executed first when a recording medium 21 such as an optical disk on which data of this file system is recorded is loaded in the playback device 20. .

  FirstPlayback_mobj_id_ref is a field for designating the value of mobj_id of the NavigationObject that is executed first. mobj_id is an ID to which NavigationObject is uniquely given. Also, if the NavigationObject that is automatically executed at the start of playback is not set, that is, no application is executed at the start of playback, and any application is executed in response to an instruction from the user. In this case, “0Xffff” is described in this field.

  TopMenu () is a data block that includes information related to NavigationObject of TopMenu that is called when the user displays a menu screen.

  TopMenu_mobj_id_ref is a field for designating the value of mobj_id of NavigationObject of TopMenu. When TopMenu is not set, “0xFFFF” is described in this field.

  number_of_Titles is a field representing the number of titles described in Indexes ().

  Title [title_id] () is a block in which information about each title uniquely distinguished by title_id is described. title_id is assigned from 0.

  Title_playback_type [title_id] is information indicating the playback type of the title specified by the title_id, for example, a movie title that is a general content with a video to be played and audio, or a video that is played by user operation input And Interactive Title, which is a content that can be interactively exchanged, such as a change in voice or sound. For example, when the title is Movie Title, playback processing based on PlayList is executed.

  Title_access_type [title_id] is information indicating whether or not the title specified by the title_id is permitted to be reproduced using Title_Search.

  reserved_for_future_use is a 29-bit field whose data description content is not set in preparation for future expansion.

  Title_mobj_id_ref [title_id] is a field for specifying the value of mobj_id of the NavigationObject in which the title specified by the title_id is entered.

  Next, FIG. 11 shows a second example of the syntax of Indexes (). In the example of FIG. 11, since the data referred to by Indexes () is synthesized with only sound data (audio stream # 2 is not included), Index (). Describes a flag indicating whether or not sound data to be synthesized with the audio stream # 1 is included in the data referred to by Indexes ().

  In the second example of the Indexes () syntax shown in FIG. 11, is_MixApp, which is a 1-bit flag indicating whether or not sound data is included in the data referenced by Indexes (), is newly described. , Except that reserved_for_future_use is changed from 29 bits to 28 bits, it has the same configuration as that of the first example of the syntax of Indexes () described with reference to FIG. Note that this is_MixApp is defined as a flag indicating whether or not the data referred to by Indexes () includes at least one of sound data or audio stream # 2 that can be synthesized with audio stream # 1. May be. In this case, it is possible to quickly recognize whether or not the audio data is synthesized, that is, the necessity of the decoding process, only by confirming this one flag.

  Next, FIG. 12 shows a third example of the syntax of Indexes (). In the example of FIG. 12, Index () is combined with audio stream # 1 and a flag indicating whether or not the data referred to by Indexes () includes sound data combined with audio stream # 1. Two types of flags, which indicate whether or not audio stream # 2 is included.

  In the third example of the Indexes () syntax shown in FIG. 12, is_MixApp_1, which is a 1-bit flag indicating whether or not the audio stream # 2 is included in the data referenced by Indexes (), and Indexes Two types of flags, is_MixApp_2, which is a 1-bit flag indicating whether or not sound data is included in the data referenced by (), are newly described, and reserved_for_future_use is changed from 29 bits to 27 bits. Other than the above, the configuration is the same as that of the first example of the Indexes () syntax described with reference to FIG.

  Also, it is not a flag indicating whether or not the data to be synthesized with the audio stream # 1 (at least one of the audio stream # 2 and the sound data) is included in the data referred to by Indexes (), but is synthesized. A flag indicating whether processing can be performed or not can be defined. In this case, the defined flag is a flag indicating whether or not the composite data can be applied to the PlayList when playback is performed based on the Index, PlayList, etc. for managing the playback order. It becomes.

  FIG. 13 shows the data structure of the PlayList file. The PlayList file has an extension “.mpls” stored in the PLAYLIST directory in a recording medium attached to the playback device 20 or a storage unit inside the playback device 20 (for example, a local storage including a hard disk). It is a data file.

  type_indicator describes information indicating the type of the file. That is, in this field, information indicating that this file type is PlayList (MoviePlayList) that is playback management information for managing video playback.

  version_number indicates four character characters indicating the version number of xxxx.mpls (MoviePlayList).

  PlayList_start_address indicates the start address of PlayList () with the relative number of bytes from the start byte of the PlayList file as a unit.

  PlayListMark_start_address indicates the head address of PlayListMark () with the relative number of bytes from the head byte of the PlayList file as a unit.

  ExtensionData_start_address indicates the start address of ExtensionData () in units of the relative number of bytes from the start byte of the PlayList file.

  AppInfoPlayList () stores parameters related to playback control of the PlayList, such as playback restrictions. Details of AppInfoPlayList () will be described later with reference to FIG. 14 or FIG.

  PlayList () stores parameters related to the main path and sub path of the PlayList. Details of PlayList () will be described later with reference to FIGS. 16 to 18.

  PlayListMark () stores PlayList mark information, that is, information about a mark that is a jump destination (jump point) in a user operation or command that commands chapter jump or the like.

  Private data can be inserted into ExtensionData ().

  Next, a first example of AppInfoPlayList () will be described with reference to FIG. In the AppInfoPlayList () of FIG. 14, when Indexes () is the one described with reference to FIG. 10, that is, the data referred to by Indexes () includes other audio data synthesized with the audio stream # 1. It can be applied when the flag indicating whether or not it is described in Index ().

  The length indicates a syntax length equal to or less than length, that is, the number of bytes of AppInfoPlayList () from immediately after this length field to the end of reserved_for_future_use. After this length, 8-bit reserved_for_future_use is prepared.

  Playlist_playback_type is a field that describes information indicating whether the playlist is to be played back continuously, randomly accessed, or shuffled.

  Playback_count is a field that describes information indicating the number of PlayItems used for playback in the case of a PlayList that performs random access and shuffle.

  UO_mask_table () is information that determines restrictions on user operations related to special playback such as pause, chapter search, jump, fast forward, fast reverse (backward fast playback), or display.

  PlayList_random_access_flag is flag information for controlling jump playback from another PlayList. Here, when PlayList_random_access_flag = 1 is set, jump playback from another PlayList is prohibited.

  For example, in the playback device 20, for example, a user operation that commands a jump of a playback position from a playback position referenced by the current PlayList to the start or chapter position of a predetermined PlayItem referenced by another PlayList, etc. It is possible to prepare an extended user operation unique to the device. PlayList_random_access_flag is, for example, when a jump from the playback position of a Clip AV Stream referenced by another PlayList to a Clip AV Stream referenced by this PlayList (movement of the playback position) is commanded by these user operations This is flag information for setting whether or not to restrict this user operation.

  Note that when a jump (movement of the playback position) is commanded by a navigation command rather than a jump (movement of the playback position) commanded by a user operation, the PlayList_random_access_flag is ignored (the command is executed and the command is generated) Accordingly, the playback position is moved).

  Is_MixApp is a flag indicating whether or not the stream reproduced by the Playlist file performs synthesis of sound and sound effects. That is, this is_MixApp is defined as a flag indicating whether or not the data referred to by this PlayList includes sound data that can be synthesized with the audio stream # 1 and the audio stream # 2. In this case, it is possible to quickly recognize whether or not the audio data is synthesized, that is, the necessity of the decoding process, only by confirming this one flag.

  Lossless_may_bypass_mixer_flag is a flag relating to the reproduction of lossless audio. After this, 13-bit reserved_for_future_use is prepared.

  As shown in FIG. 10, in the case where a flag indicating whether or not other audio data to be synthesized with the audio stream # 1 is included in the data referred to by Indexes () is not described in Indexes () As described with reference to FIG. 14, Is_MixApp may be described in AppInfoPlayList () to indicate whether or not this Playlist synthesizes sound and sound effects. For example, Is_MixApp is included in AppInfoPlayList (). In this case, Is_MixApp may be described in PlayList () or PlayItem.

  Next, a second example of AppInfoPlayList () will be described with reference to FIG.

  The second example of AppInfoPlayList () shown in FIG. 15 has been described with reference to FIG. 14 except that Is_MixApp, which is a flag indicating whether or not this playlist synthesizes sound and sound effects, is not described. It has the same configuration as the first example of AppInfoPlayList (). That is, in the second example of AppInfoPlayList () in FIG. 15, Indexes () is described with reference to FIG. 10, but flags such as Is_MixApp are described in PlayList () or PlayItem described later. That is, a flag indicating whether or not other audio data to be synthesized with the audio stream # 1 is included in the data referred to by Indexes () is not described in Index (), and Is_MixApp in PlayList () or PlayItem Is described, or when Indexes () is the one described with reference to FIG. 11 or FIG. 12, that is, other audio synthesized in the audio stream # 1 with the data referred to by Indexes () This is applied when a flag indicating whether or not data is included is described in Index ().

  Next, FIG. 16 is a diagram illustrating a first example of the syntax of PlayList (). FIG. 16 shows a syntax in the case where a flag indicating whether or not other audio data synthesized with the audio stream # 1 is included in data reproduced by referring to this playlist is not described in PlayList (). It is.

  That is, the first example of the syntax of PlayList () in FIG. 16 is that when Indexes () is the one described with reference to FIG. 10, that is, audio stream # 1 for data referenced by Indexes (). The flag indicating whether or not other audio data to be synthesized is included in Index () is not described in Index (), but if it is described in PlayItem described later, or Indexes () is shown in FIG. In other words, a flag indicating whether or not the data referred to by Indexes () includes other audio data to be synthesized with the audio stream # 1 is described in Index (). Applicable when

  length is a 32-bit unsigned integer indicating the number of bytes from immediately after this length field to the end of PlayList (). That is, this field indicates the number of bytes from reserved_for_future_use to the end of the Playlist. After this length, 16-bit reserved_for_future_use is prepared. In reserved_for_future_use, the description content of the data is not set in preparation for future expansion. number_of_PlayItems is a 16-bit field indicating the number of PlayItems in the PlayList. For example, in the example of FIG. 5, the number of PlayItems is three. The value of PlayItem_id is assigned from 0 in the order in which PlayItem () appears in the PlayList. For example, as shown in FIGS. 5 and 7, PlayItem_id = 0, 1, and 2 are allocated.

  number_of_SubPaths is a 16-bit field indicating the number of SubPaths (number of entries) in the PlayList. For example, in the example of FIG. 5, the number of Sub Paths is three. The value of SubPath_id is assigned from 0 in the order in which SubPath () appears in the PlayList. For example, as shown in FIG. 5, Subpath_id = 0, 1, and 2 are allocated. In the subsequent for statement, PlayItems are referenced by the number of PlayItems, and Sub Paths are referenced by the number of Sub Paths.

  Next, FIG. 17 is a diagram illustrating a second example of the syntax of PlayList (). In the example of FIG. 17, the audio stream reproduced by referring to this playlist is only the audio stream # 1 (the audio stream # 2 is not included), and PlayList () includes PlayList (). The flag indicating whether or not sound data to be synthesized with the audio stream # 1 is included in the data reproduced with reference to FIG.

  That is, the second example of the syntax of PlayList () in FIG. 17 is that when Indexes () is the one described with reference to FIG. 10, that is, audio stream # 1 for data referenced by Indexes (). This is applied when a flag indicating whether or not other audio data to be synthesized is included is not described in Index () and a flag is not described in PlayItem described later.

  In the second example of the syntax of PlayList () shown in FIG. 17, is_MixApp, which is a 1-bit flag indicating whether or not sound data is included in the data referred to by PlayList (), is newly described. , Except that reserved_for_future_use is changed from 16 bits to 15 bits, it has the same configuration as the first example of the syntax of PlayList () described with reference to FIG. Note that this is_MixApp may be defined as a flag indicating whether or not the data referred to by Playlist () includes sound data that can be combined with the audio stream # 1 and the audio stream # 2. In this case, it is possible to quickly recognize whether or not the audio data is synthesized, that is, the necessity of the decoding process, only by confirming this one flag.

  Next, a third example of the syntax of PlayList () is shown in FIG. In the example of FIG. 18, PlayList () is combined with audio stream # 1 and a flag indicating whether or not sound data to be combined with audio stream # 1 is included in the data referenced by PlayList (). And a flag indicating whether or not audio stream # 2 is included.

  That is, in the third example of the syntax of PlayList () in FIG. 18, when Indexes () is the same as that described with reference to FIG. 10, that is, the audio stream # 1 is the data referenced by Indexes (). This is applied when a flag indicating whether or not other audio data to be synthesized is included is not described in Index () and a flag is not described in PlayItem described later.

  In the third example of the PlayList () syntax shown in FIG. 18, is_MixApp_1 that is a 1-bit flag indicating whether or not the audio stream # 2 is included in the data referenced by the PlayList (), and the PlayList Two types of flags, is_MixApp_2, which is a 1-bit flag indicating whether or not sound data is included in the data referred to by (), are newly described, and reserved_for_future_use is changed from 16 bits to 14 bits. Except for the above, it has the same configuration as that of the first example of the syntax of PlayList () described with reference to FIG.

  FIG. 19 is a diagram illustrating an example of the syntax of SubPath ().

  length is a 32-bit unsigned integer indicating the number of bytes from immediately after this length field to the end of Sub Path (). That is, this field indicates the number of bytes from reserved_for_future_use to the end of the Playlist. After this length, 16-bit reserved_for_future_use is prepared. In reserved_for_future_use, the description content of the data is not set in preparation for future expansion. SubPath_type is an 8-bit field indicating the application type of SubPath. SubPath_type is used, for example, when the type indicates whether the Sub Path is audio, bitmap subtitle, text subtitle, or the like. This SubPath_type will be described later with reference to FIG. 15-bit reserved_for_future_use is prepared after SubPath_type. is_repeat_SubPath is a 1-bit field that specifies the SubPath playback method, and indicates whether the SubPath playback is repeated during playback of the main path or the SubPath playback is performed only once. For example, it is used when the playback timing of the clip specified by the main AV stream and the sub path is different (for example, when the main path is used as a still image slide show and the sub path audio path is used as the main path BGM). . 8-bit reserved_for_future_use is prepared after Is_repeat_SubPath. number_of_SubPlayItems is an 8-bit field indicating the number of SubPlayItems (number of entries) in one SubPath. For example, number_of_SubPlayItems has one SubPlayItem with SubPath_id = 0 in FIG. 5 and two SubPlayItems with SubPath_id = 1. In subsequent for statements, SubPlayItems are referenced by the number of SubPlayItems.

  FIG. 20 is a diagram illustrating an example of SubPath_type (subpath type). That is, the type of SubPath is defined as shown in FIG. 20, for example.

  In FIG. 20, SubPath_type = 0, 1 is reserved. SubPath_type = 2 is an Audio presentation path of the Browsable slideshow. For example, SubPath_type = 2 indicates that in the playlist, the audio presentation path referred to by the sub path and the main path referred to by the play item are asynchronous.

  SubPath_type = 3 is an interactive graphics presentation menu (interactive graphics presentation menu). For example, SubPath_type = 3 indicates that in the playlist, the interactive graphics menu referenced by the sub path and the main path referenced by the play item are asynchronous.

  SubPath_type = 4 is a Text subtitle presentation path (a text subtitle presentation path). For example, SubPath_type = 4 indicates that in the playlist, the text subtitle presentation path referenced by the sub path and the main path referenced by the play item are synchronized.

  SubPath_type = 5 is 2nd Audio Presentation path (second audio presentation path) (path for referring to the 2nd audio stream). Specifically, SubPath_type = 5 indicates that in the playlist, the second audio presentation path referred to by the sub path and the main path referred to by the play item are synchronized. For example, the (second) audio stream referenced in this subpath is a director's comment (voice) on the movie. In the case of Subpath_id as shown in FIG. 7, SubPath_type in FIG. 19 is SubPath_type = 5.

  SubPath_type = 6 is 2nd Video Presentation path (second video presentation path) (path for referring to the 2nd video stream). Specifically, SubPath_type = 6 indicates that in the playlist, the second video presentation path referred to by the sub path and the main path referred to by the play item are synchronized. For example, the (second) video stream referred to in this sub-path is a director's comment (moving image) for the movie.

  SubPath_type = 7 to 255 is reserved.

  FIG. 21 is a diagram illustrating the syntax of SubPlayItem (i).

  The length is a 16-bit unsigned integer indicating the number of bytes from immediately after this length field to the end of Sub playItem ().

  In FIG. 21, SubPlayItem is divided into a case of referring to one Clip and a case of referring to a plurality of Clips.

  First, a case where SubPlayItem refers to one Clip will be described.

  SubPlayItem includes Clip_Information_file_name [0] for designating a Clip. In addition, Clip_codec_identifier [0] that specifies the codec method of Clip, reserved_for_future_use, is_multi_Clip_entries that indicates the presence or absence of multi-clip registration, and ref_to_STC_id [0] that is information about STC discontinuity points (system time base discontinuity points) including. When the is_multi_Clip_entries flag is set, the syntax when SubPlayItem refers to a plurality of Clips is referenced. In addition, SubPlayItem_IN_time and SubPlayItem_OUT_time for specifying the playback section of the Sub Path in the Clip are included. Furthermore, sync_PlayItem_id and sync_start_PTS_of_PlayItem are included to specify the time when the Sub Path starts to be played on the time axis of the main path. As described above, the sync_PlayItem_id and sync_start_PTS_of_PlayItem are used in the case of FIGS. 6 and 7 (when the playback timing of the file indicated by the main AV stream and the sub path is the same), and the file indicated by the main AV stream and the sub path. Are not used (for example, when the still image referred to by the main path and the audio referred to by the sub path are not synchronized as in the BGM of a slide show composed of still images). Also, SubPlayItem_IN_time, SubPlayItem_OUT_time, sync_PlayItem_id, and sync_start_PTS_of_PlayItem are used in common in the Clip referred to by SubPlayItem.

  Next, a case where SubPlayItem refers to a plurality of Clips (if (is_multi_Clip_entries == 1b), that is, multi-clip registration is performed) will be described. Specifically, as shown in FIG. 7, a case where SubPlayItem refers to a plurality of Clips is shown.

  num_of_Clip_entries indicates the number of Clips, and Clip_Information_file_name [SubClip_entry_id] specifies Clips excluding Clip_Information_file_name [0]. That is, Clips such as Clip_Information_file_name [1] and Clip_Information_file_name [2] are specified excluding Clip_Information_file_name [0]. Further, SubPlayItem includes Clip_codec_identifier [SubClip_entry_id] that specifies the codec method of Clip, ref_to_STC_id [SubClip_entry_id] that is information related to STC discontinuity points (system time base discontinuity points), and reserved_for_future_use.

  Note that SubPlayItem_IN_time, SubPlayItem_OUT_time, sync_PlayItem_id, and sync_start_PTS_of_PlayItem are used in common among a plurality of clips. In the example of FIG. 7, SubPlayItem_IN_time, SubPlayItem_OUT_time, sync_PlayItem_id, and sync_start_PTS_of_PlayItem are used in common between SubClip_entry_id = 0 and SubClip_entry_id = 1, and the Text based subtitle for the selected SubClip_entry_id is timeSubPlayItem_id Playback is based on SubPlayItem_OUT_time, sync_PlayItem_id, and sync_start_PTS_of_PlayItem.

  Here, the value of SubClip_entry_id is assigned from 1 in the order in which Clip_Information_file_name [SubClip_entry_id] in SubPlayItem appears. Also, SubClip_entry_id of Clip_Information_file_name [0] is 0.

  FIG. 22 is a diagram illustrating a first example of the syntax of PlayItem (). FIG. 22 shows the syntax in the case where the flag indicating whether or not the other audio data synthesized with the audio stream # 1 is included in the data corresponding to this PlayItem () is not described.

  That is, the first example of the PlayItem () syntax in FIG. 22 is that when Indexes () is the one described with reference to FIG. 11 or FIG. When the flag indicating whether or not other audio data to be synthesized is included in the stream # 1 is described in Index (), or when AppInfoPlayList () has been described with reference to FIG. A flag indicating whether or not other audio data to be synthesized with the audio stream # 1 is included in the referenced data is described in AppInfoPlayList (), or PlayList () is used with reference to FIG. 17 or FIG. In other words, a flag indicating whether or not other audio data to be synthesized with the audio stream # 1 is included in the data referred to by the PlayList is described. It is applied to the case that.

  length is a 16-bit unsigned integer indicating the number of bytes from immediately after this length field to the end of PlayItem (). Clip_Information_file_name [0] is a field for designating a Clip referred to by PlayItem. In the example of FIG. 6, the main AV stream is referred to by Clip_Information_file_name [0]. Also, Clip_codec_identifier [0] that specifies the codec format of Clip, 11-bit reserved_for_future_use in which the data description content is not set in preparation for future expansion, and a flag indicating whether or not multi-angle playback is supported Is_multi_angle, and further includes connection_condition and ref_to_STC_id [0] which is information on STC discontinuity (system time base discontinuity). Furthermore, IN_time and OUT_time for specifying the playback section of PlayItem in Clip are included. In the example of FIG. 6, the playback range of the main ClipAV stream file is represented by IN_time and OUT_time. Also, UO_mask_table (), PlayItem_random_access_mode, and still_mode are included. The case where there are a plurality of is_multi_angles is not directly related to the present invention, and the description thereof will be omitted.

  STN_table () in PlayItem () refers to the PlayItem's operations such as audio switching and subtitle switching when the target PlayItem and one or more SubPaths to be played back are prepared. It provides a mechanism that allows the user to select from Clips and Clips referenced by one or more of these SubPaths. STN_table () provides a mechanism that allows selection of mixing reproduction of two audio streams.

  Next, FIG. 23 is a diagram illustrating a second example of the syntax of PlayItem (). In the example of FIG. 23, the audio stream # 2 is not included in the data corresponding to this PlayItem (), and the sound synthesized with the audio stream # 1 is combined with the data corresponding to PlayItem () in PlayItem (). A flag indicating whether or not data is included is described.

  That is, the second example of the syntax of PlayItem () in FIG. 23 is that when Indexes () is the one described with reference to FIG. 10, that is, the audio stream # 1 for the data referenced by Indexes (). Is not described in Index () and AppInfoPlayList () is the one described with reference to FIG. 15, that is, it is referred to by PlayList. The flag indicating whether or not other audio data to be synthesized with the audio stream # 1 is included in AppInfoPlayList () is not described in AppInfoPlayList (), and PlayList () is described with reference to FIG. That is, it is applied when a flag indicating whether or not other audio data to be synthesized with the audio stream # 1 is included in the data referred to by the PlayList.

  In the second example of the PlayItem () syntax shown in FIG. 23, is_MixApp, which is a 1-bit flag indicating whether or not sound data is included in the data referred to by PlayItem (), is newly described. , Except that reserved_for_future_use is changed from 11 bits to 10 bits, it has the same configuration as the first example of the syntax of PlayItem () described with reference to FIG. Note that this is_MixApp may be defined as a flag indicating whether or not the data referred to by PlayItem () includes sound data that can be synthesized with the audio stream # 1 and the audio stream # 2. In this case, it is possible to quickly recognize whether or not the audio data is synthesized, that is, the necessity of the decoding process, only by confirming this one flag.

  Next, a third example of the PlayItem () syntax is shown in FIG. In the example of FIG. 24, PlayItem () is combined with audio stream # 1 with a flag indicating whether or not the data corresponding to PlayItem () includes sound data combined with audio stream # 1. A flag indicating whether or not the audio stream # 2 is included is described.

  That is, in the third example of the PlayItem () syntax in FIG. 24, when Indexes () is the same as that described with reference to FIG. 10, that is, the audio stream # 1 is the data referenced by Indexes (). Is not described in Index () and AppInfoPlayList () is the one described with reference to FIG. 15, that is, it is referred to by PlayList. The flag indicating whether or not other audio data to be synthesized with the audio stream # 1 is included in AppInfoPlayList () is not described in AppInfoPlayList (), and PlayList () is described with reference to FIG. That is, it is applied when a flag indicating whether or not other audio data to be synthesized with the audio stream # 1 is included in the data referred to by the PlayList.

  In the third example of the PlayItem () syntax shown in FIG. 24, is_MixApp_1 which is a 1-bit flag indicating whether or not the audio stream # 2 is included in the data corresponding to PlayItem (), and PlayItem ( ), Two types of flags, is_MixApp_2, which is a 1-bit flag indicating whether or not sound data is included in the data, are described, and reserved_for_future_use is changed from 11 bits to 9 bits. FIG. 22 has the same configuration as that of the first example of the syntax of PlayItem () described with reference to FIG.

  FIG. 25 is a diagram illustrating the syntax of STN_table (). STN_table () is set as an attribute of PlayItem.

  length is a 16-bit unsigned integer indicating the number of bytes from immediately after this length field to the end of STN_table (). 16-bit reserved_for_future_use is prepared after the length. number_of_video_stream_entries indicates the number of streams to which video_stream_id entered (registered) in STN_table () is given. video_stream_id is information for identifying a video stream, and video_stream_number is a video stream number that is used for video switching and is visible to the user.

  number_of_audio_stream_entries indicates the number of streams of the first audio stream to which audio_stream_id entered in STN_table () is given. The audio_stream_id is information for identifying the audio stream, and the audio_stream_number is an audio stream number that can be seen by the user used for audio switching. number_of_audio_stream2_entries indicates the number of streams of the second audio stream to which audio_stream_id2 entered in STN_table () is given. The audio_stream_id2 is information for identifying an audio stream, and the audio_stream_number is an audio stream number that can be seen by the user used for audio switching. Specifically, the number_of_audio_stream_entries audio stream entered in STN_table () is an audio stream decoded by the 1st audio decoder 75-1 of the playback apparatus 20 shown in FIG. 28 described later, and number_of_audio_stream2_entries entered in STN_table (). The audio stream is an audio stream that is decoded by a 2nd audio decoder 75-2 of the playback device 20 in FIG. In this way, in STN_table () in FIG. 25, audio streams to be decoded by the two audio decoders can be entered.

  In the following, the audio stream of number_of_audio_stream_entries decoded by the 1st audio decoder 75-1 of the playback device 20 in FIG. 28 is referred to as audio stream # 1, and is decoded by the 2nd audio decoder 75-2 of the playback device 20 in FIG. The audio stream of number_of_audio_stream2_entries is referred to as audio stream # 2. Also, it is assumed that the audio stream # 1 is an audio stream that has priority over the audio stream # 2.

  number_of_PG_txtST_stream_entries indicates the number of streams to which PG_txtST_stream_id entered in STN_table () is given. In this, a stream (PG, Presentation Graphics stream) obtained by run-length encoding bitmap subtitles such as a DVD sub-picture and a text subtitle file (txtST) are entered. PG_txtST_stream_id is information for identifying a subtitle stream, and PG_txtST_stream_number is a subtitle stream number (text subtitle stream number) that can be seen by the user used for subtitle switching.

  number_of_IG_stream_entries indicates the number of streams provided with IG_stream_id entered in STN_table (). In this, an interactive graphics stream is entered. IG_stream_id is information for identifying an interactive graphics stream, and IG_stream_number is a graphics stream number that is visible to the user used for graphics switching.

  Here, the syntax of stream_entry () will be described with reference to FIG.

  length is an 8-bit unsigned integer indicating the number of bytes from immediately after this length field to the end of stream_entry (). type is an 8-bit field indicating the type of information necessary to uniquely specify the stream to which the above-described stream number is given.

  In type = 1, a 16-bit packet ID (PID) is used to identify one elementary stream from among a plurality of elementary streams multiplexed in a Clip (Main Clip) referenced by PlayItem. Is specified. ref_to_stream_PID_of_mainClip indicates this PID. That is, with type = 1, the stream is determined simply by specifying the PID in the main ClipAV stream file.

  In type = 2, when SubPath refers to multiple Clips at a time and each Clip multiplexes multiple elementary streams, multiple elementary streams of one Clip (SubClip) referenced by SubPath SubPath_id, Clip id, and packet ID (PID) of the SubPath are specified in order to identify one elementary stream from the list. ref_to_SubPath_id indicates this SubPath_id, ref_to_SubClip_entry_id indicates this Clip id, and ref_to_stream_PID_of_SubClip indicates this PID. This is used when a plurality of Clips are referred to in SubPlayItem and a plurality of elementary streams are referred to this Clip.

  In this way, by using type (two types of type = 1 and type = 2), when PlayItem and one or more SubPaths to be played back are prepared, this PlayItem refers to One elementary stream can be identified from Clips referenced by Clip and one or more SubPaths. Note that type = 1 indicates a clip (main clip) referred to by the main path, and type = 2 indicates a clip (sub clip) referred to by the sub path.

  Returning to the description of STN_table () in FIG. 25, video_stream_id is given from 0 to one video elementary stream that is identified for each stream_entry () in turn in the for loop of the video stream ID (video_stream_id). Note that a video stream number (video_stream_number) may be used instead of the video stream ID (video_stream_id). In this case, video_stream_number is given from 1 instead of 0. That is, video_stream_number is obtained by adding 1 to the value of video_stream_id. The video stream number is defined from 1 since it is a video stream number that is used for video switching and is visible to the user.

  Similarly, in the for loop of the audio stream ID (audio_stream_id), audio_stream_id is given from 0 to one audio elementary stream that is specified for each stream_entry () in order. As in the case of the video stream, an audio stream number (audio_stream_number) may be used instead of the audio stream ID (audio_stream_id). In this case, audio_stream_number is given from 1 instead of 0. That is, audio_stream_number is obtained by adding 1 to the value of audio_stream_id. The audio stream number is an audio stream number that can be seen by the user and is used for audio switching.

  Similarly, audio_stream_id2 is given from 0 to one audio elementary stream specified for each stream_entry () in order in the for loop of the audio stream ID2 (audio_stream_id2). As in the case of the video stream, audio stream number 2 (audio_stream_number2) may be used instead of audio stream ID2 (audio_stream_id2). In this case, audio_stream_number2 is given from 1 instead of 0. That is, audio_stream_number2 is obtained by adding 1 to the value of audio_stream_id2. The audio stream number 2 is an audio stream number 2 that can be seen by the user and used for audio switching.

  That is, in STN_table () of FIG. 25, an audio stream of number_of_audio_stream_entries (audio stream # 1) and an audio stream of number_of_audio_stream2_entries (audio stream # 2) are defined. In other words, since the audio stream # 1 and the audio stream # 2 can be entered using the STN_table (), the user can select two audio streams to be reproduced in synchronization.

  Similarly, PG_txtST_stream_id is assigned from 0 to one bitmap subtitle elementary stream or text subtitle identified in order for each stream_entry () in the for loop of the subtitle stream ID (PG_txtST_stream_id). As in the case of the video stream, a subtitle stream number (PG_txtST_stream_number) may be used instead of the subtitle stream ID (PG_txtST_stream_id). In this case, PG_txtST_stream_number is given from 1 instead of 0. That is, PG_txtST_stream_number is obtained by adding 1 to the value of PG_txtST_stream_id. The subtitle stream number is a subtitle stream number (text subtitle stream number) that can be seen by the user used for subtitle switching, and is defined from 1.

  Similarly, in the for loop of the graphics stream ID (IG_stream_id), IG_stream_id is assigned from 0 to one interactive graphics elementary stream that is identified for each stream_entry () in order. As in the case of the video stream, a graphics stream number (IG_stream_number) may be used instead of the graphics stream ID (IG_stream_id). In this case, IG_stream_number is given from 1 instead of 0. That is, IG_stream_number is obtained by adding 1 to the value of IG_stream_id. Since the graphics stream number is a graphics stream number that can be seen by the user used for switching graphics, it is defined from 1.

  Next, stream_attribute () of STN_table () in FIG. 25 will be described.

  In the for statement after reserved_for_future_use, the video stream is referred to as much as the video stream, and the provider of this data (for example, the manufacturer of the recording medium 21) sets the Main Path and Sub Path for the audio stream. The audio stream is referenced by the amount, the PG textST stream is referenced by the amount of the PG textST stream, and the IG stream is referenced by the amount of the IG stream.

  The stream_attribute () in the for loop of the video stream ID (video_stream_id) gives stream attribute information of one video elementary stream specified for each stream_entry (). That is, in this stream_attribute (), stream attribute information of one video elementary stream specified for each stream_entry () is described.

  Similarly, stream_attribute () in the for loop of the audio stream ID (audio_stream_id) gives stream attribute information of one audio elementary stream specified for each stream_entry (). That is, in this stream_attribute (), stream attribute information of one audio elementary stream specified for each stream_entry () is described. For example, since there is one audio elementary stream specified by type = 1 or type = 2 of stream_entry () in FIG. 26, stream_attribute () gives stream attribute information of that one audio elementary stream.

  Similarly, stream_attribute () in the for loop of audio stream ID2 (audio_stream_id2) gives stream attribute information of one audio elementary stream specified for each stream_entry (). That is, in this stream_attribute (), stream attribute information of one audio elementary stream specified for each stream_entry () is described. For example, since there is one audio elementary stream specified by type = 1 or type = 2 of stream_entry () in FIG. 26, stream_attribute () gives stream attribute information of that one audio elementary stream.

  Similarly, stream_attribute () in the for loop of the subtitle stream ID (PG_txtST_stream_id) gives stream attribute information of one bitmap subtitle elementary stream or text subtitle elementary stream specified for each stream_entry (). That is, in this stream_attribute (), stream attribute information of one bitmap subtitle elementary stream specified for each stream_entry () is described.

  Similarly, stream_attribute () in the for loop of the graphics stream ID (IG_stream_id) gives stream attribute information of one interactive graphics elementary stream specified for each stream_entry (). That is, stream attribute information of one interactive graphics elementary stream specified for each stream_entry () is described in the stream_attribute ().

  Here, the syntax of stream_attribute () will be described with reference to FIG.

  The length is a 16-bit unsigned integer indicating the number of bytes from immediately after this length field to the end of stream_attribute ().

  As shown in FIG. 28, stream_coding_type indicates the encoding type of the elementary stream. As encoding types of elementary streams, MPEG-2 video stream, HDMV LPCM audio, Dolby AC-3 audio, dts audio, Presentation graphics stream, Interactive graphics stream, and Text subtitle stream are described.

  video_format indicates the video format of the video elementary stream as shown in FIG. 480i, 576i, 480p, 1080i, 720p, and 1080p are described as the video format of the video elementary stream.

  As shown in FIG. 30, frame_rate indicates the frame rate of the video elementary stream. As frame rates of the video elementary stream, 24000/1001, 24, 25, 30000/1001, 50, and 60000/1001 are described.

  The aspect_ratio indicates the aspect ratio information of the video elementary stream as shown in FIG. As aspect ratio information of the video elementary stream, 4: 3 display aspect ratio and 16: 9 display aspect ratio are described.

  audio_presentation_type indicates presentation type information of an audio elementary stream as shown in FIG. As the presentation type information of the audio elementary stream, single mono channel, dual mono channel, stereo (2-channel), and multi-channel are described.

  Sampling_frequency indicates the sampling frequency of the audio elementary stream as shown in FIG. As sampling frequencies of audio elementary streams, 48 kHz and 96 kHz are described.

  audio_language_code indicates the language code of the audio elementary stream (Japanese, Korean, Chinese, etc.).

  PG_language_code indicates the language code (Japanese, Korean, Chinese, etc.) of the bitmap subtitle elementary stream.

  IG_language_code, which indicates the language code (Japanese, Korean, Chinese, etc.) of the interactive graphics elementary stream.

  textST_language_code indicates the language code (Japanese, Korean, Chinese, etc.) of the text subtitle elementary stream.

  Character_code indicates the character code of the text subtitle elementary stream as shown in FIG. The character code of the text subtitle elementary stream includes Unicode V1.1 (ISO 10646-1), Shift JIS (Japanese), KSC 5601-1987 including KSC 5653 for Roman character (Korean), GB 18030-2000 (Chinese), GB2312 (Chinese) and BIG5 (Chinese) are described.

  Hereinafter, the syntax of the stream_attribute () in FIG. 27 will be specifically described with reference to FIGS. 27 and 28 to 34.

  When the encoding type of the elementary stream (stream_coding_type in FIG. 27) is an MPEG-2 video stream (FIG. 28), the stream_attribute () includes the video format (FIG. 29) and the frame rate (FIG. 30) of the elementary stream. ), And aspect ratio information (FIG. 31).

The encoding type of the elementary stream (stream_coding_type in FIG. 27) is HDMV LPCM
In the case of audio, Dolby AC-3 audio, or dts audio (FIG. 28), stream_attribute () contains the presentation type information (FIG. 32), sampling frequency (FIG. 33), and language code of the audio elementary stream. included.

  When the encoding type of the elementary stream (stream_coding_type in FIG. 27) is a Presentation graphics stream (FIG. 28), stream_attribute () includes the language code of the bitmap subtitle elementary stream.

  When the encoding type of the elementary stream (stream_coding_type in FIG. 27) is Interactive graphics stream (FIG. 28), stream_attribute () includes the language code of the interactive graphics elementary stream.

  When the elementary stream encoding type (stream_coding_type in FIG. 27) is a Text subtitle stream (FIG. 28), stream_attribute () includes the character code (FIG. 34) and language code of the text subtitle elementary stream. .

  Note that the attribute information is not limited to this.

  As described above, when a PlayItem and one or more SubPaths to be played back are prepared, a stream_entry () is used to select a Clip referred to by the PlayItem and a Clip referred to by one or more SubPaths. The attribute information of one identified elementary stream can be known by stream_attribute ().

  By examining this attribute information (stream_attribute ()), the playback device can check whether or not it has the function of playing back the elementary stream. Further, the playback device can select the elementary stream corresponding to the initial information of the language setting of the playback device by examining the attribute information.

  For example, it is assumed that the playback apparatus has only a playback function for a bitmap subtitle elementary stream and does not have a playback function for a text subtitle elementary stream. When the user instructs the playback apparatus to switch the language, the playback apparatus sequentially selects and reproduces only the bitmap subtitle elementary stream from the for loop of the subtitle stream ID (PG_txtST_stream_id).

  Also, for example, assume that the initial information of the language setting of the playback device is Japanese. When the user instructs the playback apparatus to switch the audio, the playback apparatus sequentially selects only the audio elementary stream whose language code is Japanese from the for loop of the audio stream ID (Audio stream id). And play.

  Further, for example, when playing back an AV stream (movie) made up of a video stream and an audio stream that is referenced by the main path, the user instructs the playback device to switch the audio, and the audio stream # 1 (normal When the audio output in the movie) and the audio stream # 2 (comment by the director or performer) are designated (selected) as the audio to be played back, the playback device selects the audio stream # 1 and the audio stream # 2. Mix (superimpose) and play along with the video stream.

  Note that the audio stream # 1 and the audio stream # 2 may both be audio streams included in the Clip referred to by the main path, as can be seen from the STN_table () in FIGS. Also, one of the audio stream # 1 and the audio stream # 2 may be an audio stream included in a clip referred to by the main path, and the other may be an audio stream included in the clip referenced by the sub path. As described above, it is possible to select two audio streams superimposed on the main AV stream referred to by the main path, mix them, and reproduce them.

  In this way, STN_table () in PlayItem () can be used for operations such as audio switching and subtitle switching by the user when this PlayItem and one or more SubPaths to be played back are prepared. Since a mechanism is provided that allows the user to select from the clip referenced by this PlayItem and the clip referenced by one or more SubPaths, a stream different from the AV stream to be reproduced, Interactive operations can also be performed on data files.

  Also, since a plurality of SubPaths are used in one PlayList and each SubPath refers to a SubPlayItem, an AV stream with high expandability and high flexibility can be realized. That is, it can be configured such that SubPlayItem can be added later. For example, when there is a ClipAV stream file referred to by the Main Path and a PlayList associated with the ClipAV stream file, and this PlayList is rewritten to a PlayList added with a new Sub Path, the ClipAV referenced by the Main Path is based on the new PlayList. Along with the stream file, reproduction can be performed by referring to a ClipAV stream file different from the ClipAV stream file referred to by the Main Path. Thus, it can be set as the structure which has an expandability.

  Further, STN_table () in PlayItem () is an audio stream # 1 decoded by the 1st audio decoder 75-1 of the playback apparatus 20 shown in FIG. 37 to be described later, and an audio stream decoded by the 2nd audio decoder 75-2. Provide a mechanism that can be played by mixing (synthesizing) # 2. For example, when PlayItem () and one or more SubPaths to be played back are prepared, the clip audio stream referenced by PlayItem is defined as audio stream # 1, and the clip audio stream referenced by SubPath is defined as audio stream # 1. An audio stream # 2 is provided, and a mechanism capable of mixing and reproducing these is provided. Also, for example, two audio streams included in a clip (main clip) referred to by PlayItem are set as an audio stream # 1 and an audio stream # 2, respectively, and these can be mixed and played back. did. Accordingly, it is possible to perform superimposed reproduction of an audio stream (for example, a director's comment stream) that is different from the main audio stream to be reproduced, in which the main AV stream is recorded. Also, the two audio streams # 1 and # 2 superimposed on the main AV stream can be superimposed (mixed) and reproduced.

  A specific example will be described with reference to FIG. FIG. 35 is a diagram illustrating an example of a stream number table representing a relationship between an audio signal provided to a user and a caption signal.

In FIG. 35, the audio numbers are referred to as A_SN (Audio Stream Number) and A_SN2, and the subtitle numbers are referred to as S_SN (SubPicture Stream Number). In FIG. 35, A_SN is given to each of the audio streams # 1 (audio streams entered with audio_stream_id) entered in STN_table () of PlayItems constituting the Main Path of the PlayList, and PlayItems constituting the Main Path of the PlayList. A_SN2 is given to each of the audio streams # 2 (audio streams entered with audio_stream_id2) entered in STN_table ().

  Specifically, audio 2 is given to A_SN = 1, audio 1 is given to A_SN = 2, and audio 3 is given to A_SN = 3. Audio 4 is given to A_SN2 = 1, and audio 5 is given to A_SN2 = 2. The user selects the audio stream # 1 to be reproduced from the audio streams to which A_SN is given, and selects the audio stream # 2 to be mixed with the selected audio stream # 1 in the audio stream to which A_SN2 is given. Select from. For example, the user selects audio 1 with A_SN = 2 and audio 5 with A_SN2 = 2 as playback audio streams.

  As a specific example, when the audio 2 with A_SN = 1 is selected and the user commands the audio switching, the audio is switched to the audio 1 with A_SN = 2, and the user further selects the audio. When switching is instructed, the audio is switched to audio 3 with A_SN = 3. Further, when the user instructs to switch audio further, the audio is switched to audio 2 with A_SN = 1. Also, for example, when the audio 4 with A_SN2 = 1 is selected and the user commands to switch audio, the audio is switched to audio 5 with A_SN2 = 2, and the user further commands to switch audio. In this case, the audio is switched to the audio 4 with A_SN2 = 1. As described above, A_SN for selecting the audio stream # 1 and A_SN2 for selecting the audio stream # 2 are independent from each other. That is, the user selects one audio stream from A_SN = 1 to A_SN = 3, and selects one audio stream from A_SN2 = 1 and A_SN2 = 2.

  Here, the lower the numbers of A_SN and A_SN2, the higher the priority of the audio signal provided to the user. Further, the stream given by A_SN has higher priority than the stream given by A_SN2. That is, A_SN = 1 is an audio stream reproduced by default.

  Specifically, the audio reproduced based on the initial information of the language setting of the reproduction apparatus corresponds to audio 2 (FIG. 35) with A_SN = 1, and the audio reproduced after the audio is switched. Corresponds to audio 1 (FIG. 35) with A_SN = 2.

  In order to provide such a stream number table, in the STN table () (FIG. 25) in PlayItem () referenced by PlayList (), first, audio_stream_id = 0 for entry to audio stream # 1 Audio 2 is given by (A_SN = 1), audio 1 is given by audio_stream_id = 1 (A_SN = 2), and audio 3 is given by audio_stream_id = 2 (A_SN = 3). Next, in STN table () (FIG. 25), audio 4 is given by audio_stream_id2 = 0 (A_SN2 = 1) for entry into audio stream # 2, and audio 5 is given by audio_stream_id2 = 1 (A_SN2 = 2).

  That is, by defining two audio streams to be reproduced (audio stream # 1 and audio stream # 2) separately, the user can arbitrarily select two audio streams to be reproduced from the defined ones. Can do. In other words, the user can freely select two audio streams to be played (freely defined by the audio stream # 1 and the audio stream # 2). High selection can be made. For example, the user can select a combination of audio 2 + audio 4 (a combination of A_SN = 1 and A_SN2 = 1) or a combination of audio 2 + audio 5 (a combination of A_SN = 1 and A_SN2 = 2).

  In this way, since the stream_entry () (FIG. 26) of the STN table () (FIG. 25) in PlayItem () can be entered with two audio streams, the two audio streams are Can be mixed and played back. That is, two streams of the same type (in this example, an audio stream) can be mixed (superimposed or combined) from a plurality of types of streams and simultaneously played back. In addition, the user can instruct mixing reproduction of two desired types of streams.

  In the above example, the audio stream # 1 decoded by the 1st audio decoder 75-1 and the audio stream # 2 decoded by the 2nd audio decoder 75-2 of the playback apparatus 20 in FIG. However, when a combination of the audio stream # 1 and the audio stream # 2 is defined and mixed reproduction is performed, the user may select a combination from the combinations.

  Next, the syntax of sound.bdmv in the AUXDATA directory will be described with reference to FIG.

  sound.bdmv is a file containing one or more sound effects for a stream using interactive graphics.

  SoundData_start_address is a 32-bit field indicating the start address of SoundData (), and SoundIndex () indicates the sound effect attribute (for example, the number of channels, frequency, etc.) of SoundData () that is actual sound effect data. Information.

  SoundData (), which is actual sound effect data, is uncompressed audio data.For example, a sound effect is prepared separately from a sound stream and output at a predetermined position or a user. This is data prepared as click sound for content that is supposed to receive user's operation input, such as being able to change the data to be played based on the operation input, or interactive content is there. In other words, some specifications of content or content specified by a playlist or play item may not have SoundData (). SoundData () is, for example, synthesized with an audio stream and output when a sound effect reproduction is instructed by a command or a user operation input is received.

  Next, a configuration example and processing of the playback device 20 that plays back and outputs data having the above-described configuration will be described.

  FIG. 37 is a block diagram illustrating a configuration example of the playback device 20 to which the present invention has been applied.

  The playback device 20 can play back the PlayList having the main path and the sub path described above. The playback device 20 can synthesize and play back the audio stream # 2 given by audio_stream_id2 and the sound data to the audio stream # 1 given by audio_stream_id.

  The reproduction apparatus 20 includes a reproduction data acquisition unit 31, a switch 32, an AV decoder unit 33, a controller 34, an audio encoder 41, a video encoder 42, a D / A conversion unit 43, a D / A conversion unit 44, a switch 45, analog audio. A signal interface 111, a digital audio signal interface 112, an uncompressed video signal interface 113, a compressed video signal interface 114, and an analog video signal interface 115 are provided.

  In the case of the example of FIG. 37, first, the controller 34, for example, via a storage drive that reads data from the attached recording medium 21, a data drive that reads data from its own internal recording medium, or the network 22 The index file is read out via the reproduction data acquisition unit 31 such as a network interface for acquiring data, the PlayList file is read out based on the generated command, and the PlayItem is read out based on the information of the PlayList file. Clip is detected, and the corresponding AV stream, AV data, etc. are read based on ClipInfo. The user can instruct the controller 34 to switch between audio and subtitles using the user interface such as the remote commander 24 described above. Furthermore, the user can instruct the controller 34 for various settings such as settings relating to speech synthesis processing, which will be described later, using a user interface such as the remote commander 24 described above. Further, the controller 34 is supplied with initial information of the language setting of the playback device 20 from a storage unit (not shown).

  Further, the controller 34 is a flag indicating the presence / absence of sound data that can be synthesized with the audio stream # 1 described in any one of the Index file and the PlayList file (AppInfoPlayList (), PlayList (), or PlayItem). The value of a certain is_MixApp or is_MixApp_2 and is_MixApp_1 that is a flag indicating whether or not the audio stream # 2 can be combined with the audio stream # 1 are appropriately used for the control of the switch 45.

  The PlayList file includes STN_table () in addition to Main Path and Sub Path information. The controller 34 reproduces the main Clip AV stream file included in the main Clip referred to by the PlayItem included in the PlayList file, the sub Clip AV stream file referred to by the SubPlayItem included in the sub Clip, and the text subtitle data referred to by the SubPlayItem. Reading from the recording medium 21 or the like via the data acquisition unit 31. Here, the main clip referred to by PlayItem and the sub Clip referred to by SubPlayItem may be recorded on different recording media. For example, the main clip is recorded on the recording medium 21, and the corresponding sub clip is supplied via the network 22 and stored in an HDD (not shown) built in the playback device 20. Good. In addition, the controller 34 selects and plays an elementary stream corresponding to the playback function of itself (playback device 20), or only the elementary stream corresponding to the initial information of the language setting of the playback device 20. Select and control to play.

  The AV decoder unit 33 includes buffers 51 to 54, a PID filter 55, a PID filter 56, switches 57 and 58, a background decoder 71, a video decoder 72, a presentation graphics decoder 73, an interactive graphics decoder 74, and a 1st audio decoder 75. -1, 2nd audio decoder 75-2, Text-ST composition 76, switch 77, background plane generation unit 91, video plane generation unit 92, presentation graphics plane generation unit 93, interactive graphics plane generation unit 94, buffer 95, a video data processing unit 96, a mixing processing unit 97, a switch 101, and a mixing processing unit 102 are provided.

  The file data read by the controller 34 is demodulated by a demodulation and ECC decoding unit (not shown), and error correction is performed on the demodulated multiplexed stream. The switch 32 selects the demodulated and error-corrected data for each stream type based on the control from the controller 34 and supplies the selected data to the corresponding buffers 51 to 54. Specifically, the switch 32 supplies the background image data to the buffer 51, supplies the main Clip AV Stream data to the buffer 52, and buffers the sub Clip AV Stream data based on the control from the controller 34. The switch 32 is switched so that the Text-ST data is supplied to the buffer 54. The buffer 51 buffers background image data, the buffer 52 buffers main Clip AV Stream data, the buffer 53 buffers sub Clip AV Stream data, and the buffer 54 Text-ST. Buffer data.

  The main Clip AV Stream is a stream (for example, a transport stream) obtained by multiplexing one or more streams in addition to video among video, audio, bitmap subtitle (Presentation Graphics stream), and interactive graphics. The sub clip is a stream obtained by multiplexing one or more streams of audio, bitmap subtitle (Presentation Graphics stream), interactive graphics, and audio. Note that the data of the text subtitle data file (Text-ST) may or may not be in the form of a multiplexed stream such as a transport stream.

  Further, the reproduction data acquisition unit 31 may alternately read each file of the main Clip AV Stream, the sub Clip AV Stream, and the text subtitle data in a time division manner, or before reading the main Clip AV Stream, Clip AV Stream and text subtitle data may be read in advance and preloaded into the buffer (buffer 53 or buffer 54).

  Then, the stream data read from the buffer 52, which is the main Clip AV Stream read buffer, is output to the PID (packet ID) filter 55 at the subsequent stage at a predetermined timing. The PID filter 55 distributes the input main Clip AV Stream to the decoders of the elementary streams in the subsequent stage according to the PID (packet ID) and outputs the result. That is, the PID filter 55 supplies the video stream to the video decoder 72, supplies the presentation graphics stream to the switch 57 serving as the supply source to the presentation graphics decoder 73, and supplies the interactive graphics stream to the interactive graphics decoder 74. The audio stream is supplied to a switch 101 which is a supply source to the switch 45, the 1st audio decoder 75-1, and the 2nd audio decoder 75-2.

  The presentation graphics stream is, for example, bitmap subtitle data, and the text subtitle data is, for example, text subtitle data.

  The stream data read from the buffer 53, which is a sub-Clip AV Stream read buffer, is output to a PID (packet ID) filter 56 at the subsequent stage at a predetermined timing. The PID filter 56 distributes the input sub Clip AV Stream to the decoders of the elementary streams in the subsequent stage according to the PID (packet ID) and outputs the result. That is, the PID filter 56 supplies the presentation graphics stream to the switch 57 serving as the supply source to the presentation graphics decoder 73 and supplies the interactive graphics stream to the switch 58 serving as the supply source to the interactive graphics decoder 74. The audio stream is supplied to the switch 101 which is a supply source to the switch 45, the 1st audio decoder 75-1, and the 2nd audio decoder 75-2.

  Data read from the buffer 51 that buffers background image data is supplied to the background decoder 71 at a predetermined timing. The background decoder 71 decodes the background image data and supplies the decoded background image data to the background plane generation unit 91.

  The video stream distributed by the PID filter 55 is supplied to the video decoder 72 at the subsequent stage. The video decoder 72 decodes the video stream and outputs the decoded video data to the video plane generation unit 92.

  The switch 57 selects one of the presentation graphics stream included in the main clip supplied from the PID filter 55 and the presentation graphics stream included in the sub clip, and selects the selected presentation graphics stream. This is supplied to the presentation graphics decoder 73 in the subsequent stage. The presentation graphics decoder 73 decodes the presentation graphics stream, and supplies the decoded presentation graphics stream data to the switch 77 serving as a supply source to the presentation graphics plane generation unit 93.

  Further, the switch 58 selects one of the interactive graphics stream included in the main clip supplied from the PID filter 55 and the interactive graphics stream included in the sub clip, and selects the selected interactive graphics stream. Is supplied to the interactive graphics stream decoder 74 in the subsequent stage. That is, the interactive graphics stream input to the interactive graphics decoder 74 at the same time is a stream separated from either the main Clip AV Stream or the sub Clip AV Stream. The interactive graphics decoder 74 decodes the interactive graphics stream and supplies the decoded interactive graphics stream data to the interactive graphics plane generating unit 94.

  Further, the switch 101 selects one predetermined audio stream included in the main clip or the sub clip supplied from the PID filter 55 or 56, and selects the selected audio stream as the switch 45 and the 1st audio decoder 75-1 at the subsequent stage. Or supplied to the 2nd audio decoder 75-2.

  In the present embodiment, for example, the 1st audio decoder 75-1 is for decoding the audio stream # 1, and the 2nd audio decoder 75-2 is for decoding the audio stream # 2. Specifically, in STN_table () of FIG. 25, the 1st audio decoder 75-1 is for decoding the audio stream given by audio_stream_id, and the 2nd audio is for decoding the audio stream given by audio_stream_id2. This is a decoder 75-2.

  The audio stream # 1 output from the digital audio signal interface 112 via the switch 45 and input to the 1st audio decoder 75-1 is a stream separated from either the main clip or the sub clip. Similarly, the audio stream # 2 input to the 2nd audio decoder 75-2 is also a stream separated from either the main clip or the sub clip.

  For example, when the audio stream # 1 and the audio stream # 2 are included in the main clip, the PID filter 55 filters the audio stream # 1 and the audio stream # 2 based on the PID of the audio stream, and switches 101. In this case, the switch 101 selects the switch so that the audio stream # 1 supplied from the PID filter 55 is supplied to the 1st audio decoder 75-1, and the audio stream # 2 supplied from the PID filter 55 is selected as the 2nd audio. A switch is selected to be supplied to the decoder 75-2.

  The 1st audio decoder 75-1 decodes the encoded audio stream as the audio destream # 1, and supplies an uncompressed audio signal obtained as a result to the mixing processing unit 102. The 2nd audio decoder 75-2 decodes the encoded audio stream as the audio destream # 2, and supplies the uncompressed audio signal obtained as a result to the mixing processing unit 102.

  Here, when audio stream # 1 and audio stream # 2 are superimposed and reproduced (when two audio streams are selected as audio streams to be reproduced by the user), decoding is performed by the 1st audio decoder 75-1. The audio stream # 1 (uncompressed audio signal) and the audio stream # 2 (uncompressed audio signal) decoded by the 2nd audio decoder 75-2 are supplied to the mixing processing unit 102.

  The mixing processing unit 102 mixes (superimposes) the audio stream # 1 from the 1st audio decoder 75-1 and the audio stream # 2 from the 2nd audio decoder 75-2, and outputs the result to the mixing processing unit 97 at the subsequent stage. In the present embodiment, mixing (superimposing) the audio stream # 1 output from the 1st audio decoder 75-1 and the audio stream # 2 output from the 2nd audio decoder 75-2 is combined. Also called. In other words, synthesis means to mix two audio data.

  The sound data selected by the switch 32 is supplied to the buffer 95 and buffered. The buffer 95 supplies the sound data to the mixing processing unit 97 at a predetermined timing. In this case, the sound data is sound data existing independently of the stream, such as sound effect data by menu selection or the like. The mixing processing unit 97 mixes the audio data (the audio stream # 1 output from the 1st audio decoder 75-1 and the audio stream # 2 output from the 2nd audio decoder 75-2) mixed by the mixing processing unit 102. (Uncompressed audio signal) and the sound data supplied from the buffer 95 are mixed (superimposed or synthesized) and output as an uncompressed audio signal.

  The uncompressed audio signal output from the mixing processing unit 97 of the AV decoder unit 33 is output to the audio encoder 41, the switch 45, and the D / A conversion unit 43, respectively.

  The D / A converter 43 converts the uncompressed audio signal supplied from the mixing processor 97 from a digital signal to an analog signal and supplies the analog signal to the analog audio signal interface 111. That is, the analog audio signal interface 111 outputs the analog uncompressed audio signal to the outside.

  The audio encoder 41 encodes the uncompressed audio signal supplied from the mixing processing unit 97 and supplies the encoded audio stream obtained as a result to the switch 45. That is, the audio encoder 41 performs a re-encoding process. Therefore, the output data from the audio encoder 41 is appropriately referred to as a re-encoded audio stream.

  In this way, the encoded audio stream from the switch 101, the re-encoded audio stream from the audio encoder 41, or the uncompressed audio signal from the mixing processing unit 97 is appropriately supplied to the switch 45 as an input signal. The

  Therefore, the switch 45 selects any input signal under the control of the controller 34 and outputs it as an output signal. That is, any output signal from the switch 45 is a digital audio signal. These digital audio signals are supplied to the digital audio signal interface 112. That is, the digital audio signal interface 112 outputs this digital audio signal to the outside.

  The control of the controller 34 for switching the switch 45 will be described as one process of the audio reproduction process of FIG. 38 described later.

  By the way, in the AV decoder 33, the data read from the buffer 54, which is a text subtitle read buffer, is output to the subsequent text subtitle composition (decoder) 76 at a predetermined timing. The text subtitle composition 76 decodes the Text-ST data and supplies it to the switch 77.

  The switch 77 selects either the presentation graphics stream decoded by the presentation graphics decoder 73 or Text-ST (text subtitle data), and supplies the selected data to the presentation graphics plane generating unit 93. . That is, the subtitle image simultaneously supplied to the presentation graphics plane generating unit 93 is an output of either the presentation graphics decoder 73 or the text subtitle (Text-ST) composition 76. The presentation graphics stream that is simultaneously input to the presentation graphics decoder 73 is a stream separated from either the main Clip AV Stream or the sub Clip AV Stream (selected by the switch 57). Therefore, the subtitle image simultaneously output to the presentation graphics plane generating unit 93 is a decoded output of the presentation graphics stream from the main Clip AV Stream, the presentation graphics stream from the sub Clip AV Stream, or the text subtitle data. .

  Based on the background image data supplied from the background decoder 71, for example, the background plane generation unit 91 generates a background plane that becomes a wallpaper image when the video image is displayed in a reduced size, and this is converted into the video data. This is supplied to the processing unit 96. The video plane generation unit 92 generates a video plane based on the video data supplied from the video decoder 72 and supplies this to the video data processing unit 96. The presentation graphics plane generation unit 93 generates, for example, a presentation graphics plane which is a rendering image based on the data (presentation graphics stream or text subtitle data) selected by the switch 77 and supplied, and this is converted into a video. The data is supplied to the data processing unit 96. The interactive graphics plane generating unit 94 generates an interactive graphics plane based on the data of the interactive graphics stream supplied from the interactive graphics decoder 74, and supplies this to the video data processing unit 96.

  The video data processing unit 96 includes a background plane from the background plane generation unit 91, a video plane from the video plane generation unit 92, a presentation graphics plane from the presentation graphics plane generation unit 93, and an interactive graphics plane generation unit. The interactive graphics plane from 94 is synthesized and output as a video signal.

  These switches 57, 58, 101, and 77 are switched based on selection from the user via the user interface or the file side that includes the target data.

  The video encoder 42 encodes the uncompressed video signal supplied from the video data processing unit 96 and supplies it to the compressed video signal interface 114. The D / A conversion unit 44 converts the uncompressed video signal supplied from the video data processing unit 96 from a digital signal to an analog signal and supplies the analog signal to the analog video signal interface 115.

  The uncompressed video signal interface 113 outputs the uncompressed video signal supplied from the video data processing unit 96 to the outside. The compressed video signal interface 114 outputs the encoded video stream supplied from the video encoder 42 to the outside. The analog video signal interface 115 outputs the analog uncompressed video signal supplied from the D / A converter 44 to the outside.

  Next, an example of audio signal reproduction processing executed when reproducing an audio signal will be described with reference to the flowchart of FIG.

  However, in FIG. 38, in order to clarify the features of the present invention, the switching control method of the switch 45 is mainly described, and the process inside the AV decoder unit 33 is outlined. That is, since the processing within the AV decoding unit 33 can be easily understood by referring to the description of the configuration of each block described above, only the outline of the description of the audio signal reproduction processing will be described.

  Hereinafter, the reproduction process will be described on the assumption that the controller 34 performs switching control of the switch 45.

  Furthermore, in practice, the audio signal reproduction process is rarely performed alone and is often performed in parallel with the video signal reproduction process. However, in the following description, for simplicity of explanation, There is no mention of video signal playback processing.

  In step S1, the controller 34 determines whether or not the output of the encoded audio stream is instructed by the user setting.

  That is, in this embodiment, the user selects, for example, whether to output an uncompressed audio signal for each codec type or an encoded audio stream using a user interface or the like. The selection is accepted by the controller 34 as a part of the user setting.

  In the case of this example, if the user has selected to output an uncompressed audio signal for each codec type, it is determined NO in the process of step S1, and the process proceeds to step S2.

  In step S2, the controller 34 reads the PlayList file.

  In step S3, the controller 34 controls the AV decoder unit 33 to appropriately decode the audio stream # 1 and the audio stream # 2 selected according to the PlayList file based on user settings relating to speech synthesis described later, together with sound data. Perform synthesis processing as appropriate.

  That is, the controller 34 controls the PID filters 55 and 56 and the switch 101 in the AV decoder unit 33 as appropriate, and the audio stream # 1 buffered in the buffer 52 or 53 is sent to the 1st audio decoder 75-via the switch 101. The audio stream # 2 supplied to 1 and decoded and buffered in the buffer 52 or 53 is supplied to the 2nd audio decoder 75-2 via the switch 101 to be decoded, and the mixing processing unit 102 is controlled. The audio stream synthesized by the 1st audio decoder 75-1 and the 2nd audio decoder 75-2 is appropriately executed, the mixing processor 97 is controlled, and the audio stream and the buffer 95 synthesized by the mixing processor 102 are stored in the buffer 95. Retained The process of synthesizing with the sound data is appropriately executed.

  The processed data in step S3 is output from the AV decoder unit 33 as a digital uncompressed audio signal.

  Therefore, in step S4, the controller 34 controls the switch 45 so that the processed data is output to the outside as an uncompressed audio signal.

  More specifically, the processed data in step S3, that is, the digital uncompressed audio signal is supplied directly to the switch 45 (supplied to the lower end of the three input ends in FIG. 37), as well as an audio encoder. 41. Then, the uncompressed audio signal supplied to the audio encoder 41 is subjected to an encoding process (re-encoding process) there, and a re-encoded audio stream obtained as a result is supplied to the switch 45 (3 in FIG. 37). Supplied to the middle of the two inputs).

  Therefore, in the process of step S4, the input of the switch 45 is switched to the lower end of the three input ends in FIG. 37, and the digital uncompressed audio signal supplied to the lower end is supplied from the output end of the switch 45. It is output to the outside via the digital audio signal interface 112. In other words, external output of the re-encoded audio stream is prohibited.

  The processed data in step S3, that is, the digital uncompressed audio signal is also supplied to the D / A converter 43, where it is converted into an analog uncompressed audio signal, and then the analog audio signal interface 111. Is output to the outside.

  In step S5, the controller 34 determines whether or not the reproduction is finished.

  If it is determined in step S5 that the playback has ended, the audio signal playback process ends.

  On the other hand, if it is determined in step S5 that the reproduction has not ended yet, the process returns to step S2, and the subsequent processes are repeated.

  In this way, when the user setting is selected to output an uncompressed audio signal for each codec type in the above-described example, it is determined NO in the process of step S1, and step S2 is performed. The process from S4 to S4 is executed.

  On the other hand, when the user setting is selected to output the encoded audio stream in the above-described example, it is determined as YES in the process of step S1, and the process proceeds to step S6. .

  In step S6, the controller 34 determines whether or not prohibition of re-encoding is instructed by the user setting.

  That is, in this embodiment, the user outputs, for example, a re-encoded audio stream using a user interface or the like, or the encoded audio stream (audio stream # acquired by the reproduction data acquisition unit 31). It is possible to select whether to output 1) as it is, and such selection is accepted by the controller 34 as a part of the user setting.

  In the case of this example, if the user has selected to output the encoded audio stream (audio stream # 1) acquired by the reproduction data acquisition unit 31 as it is, it is determined as YES in the process of step S6. Then, the process proceeds to step S7.

  In step S7, the controller 34 reads the PlayList file.

  In step S8, the controller 34 controls the AV decoder 33 and the switch 45 to output the encoded audio stream of the audio stream # 1 selected according to the PlayList file to the outside as it is.

  That is, the controller 34 appropriately controls the PID filters 55 and 56 and the switch 101 in the AV decoder unit 33, and the audio stream # 1 buffered in the buffer 52 or 53 is switched via the switch 101 (FIG. 37). The upper three of the three input terminals). At this time, the controller 34 switches the input of the switch 45 to the upper end of the three input ends in FIG. Then, the digital encoded audio stream (audio stream # 1) supplied to the upper end thereof is output from the output end of the switch 45 and output to the outside via the digital audio signal interface 112. In other words, external output of the re-encoded audio stream is prohibited.

  In step S9, the controller 34 determines whether or not the reproduction is finished.

  If it is determined in step S9 that the playback has ended, the audio signal playback process ends.

  On the other hand, if it is determined in step S9 that the reproduction has not been completed yet, the process returns to step S6, and the subsequent processes are repeated.

  In this way, as the user setting, the encoded audio stream (audio stream # 1) that is selected to output the encoded audio stream in the above-described example and is acquired by the reproduction data acquisition unit 31 is output as it is. If it is selected, it is determined as YES in the process of step S1, and the processes of steps S7 and S8 are executed.

  On the other hand, if the user setting is selected to output the encoded audio stream and the output of the re-encoded audio stream is selected in the above-described example, YES is determined in the process of step S1. Is determined as NO in the process of step S6, and the process proceeds to step S10.

  In step S10, the controller 34 reads the PlayList file.

  In step S11, the controller 34 determines whether or not the flag indicating the presence of data to be combined with the audio stream # 1 is 1 in the PlayList file.

  That is, the controller 34 has one of the flags indicating that there is data to be combined with the audio stream # 1 that is the main audio stream described in indexes () of the index file of the PlayList file. Specifically, whether one of the flag is_MixApp or is_MixApp_2 indicating that sound data exists or the flag is_MixApp_1 indicating that the audio stream # 2 exists is 1 Judge whether or not.

  If it is determined that none of the flags is 1, the data related to the index file includes sound data and audio stream # 2 that can be combined with the audio stream # 1 in the playback device 20. In such a case, it is determined as NO in step S11, and the processing from step S8 onward is executed. That is, the encoded audio stream of the audio stream # 1 selected according to the PlayList file is output to the outside as it is. In other words, external output of the re-encoded audio stream is prohibited.

  On the other hand, when it is determined that one of the flags is 1, it is determined as YES in step S11, and the process proceeds to step S12.

  In step S12, the controller 34 determines whether or not the audio stream # 2 exists in the PlayItem to be reproduced among the PlayItems included in the PlayList file.

  If there is no audio stream # 2 in the PlayItem to be played back, NO is determined in step S12, and the process proceeds to step S14. However, the processing after step S14 will be described later.

  On the other hand, when the audio stream # 2 exists in the PlayItem to be reproduced, it is determined as YES in Step S12, and the process proceeds to Step S13. In step S13, the controller 34 determines whether or not the output of the audio stream # 2 is instructed by the user setting.

  That is, in this embodiment, the user uses a user interface or the like to select various settings related to speech synthesis, for example, whether or not to perform synthesis when there is an interactive sound, and secondary audio (audio stream # It is possible to select whether to reproduce and synthesize 2), and such selection is accepted by the controller 34 as part of the user settings.

  In the case of this example, if the user has selected to reproduce and synthesize secondary audio (audio stream # 2), it is determined YES in the process of step S13, and the process proceeds to step S17. However, the processing after step S17 will be described later.

  On the other hand, if the user has selected not to reproduce and synthesize secondary audio (audio stream # 2), it is determined NO in the process of step S13, and the process proceeds to step S14.

  In step S <b> 14, the controller 34 determines whether there is data that can be combined with the PlayItem to be reproduced other than the audio stream # 2 among the PlayItems included in the PlayList file.

  Here, the data other than the audio stream # 2 that can be synthesized is sound data such as interactive sound, for example. If such sound data does not exist in the PlayItem to be reproduced, it is determined as NO in step S14. Then, the process proceeds to step S15. The process of step S15 is basically the same as the process of step 8 described above. That is, the encoded audio stream of the audio stream # 1 selected according to the PlayList file is output to the outside as it is. In other words, external output of the re-encoded audio stream is prohibited. Thereafter, the process proceeds to step S21. However, the processing after step S21 will be described later.

  On the other hand, for example, when sound data such as interactive sound is present in the PlayItem to be reproduced, it is determined as YES in Step S14, and the process proceeds to Step S16.

  In step S16, the controller 34 determines whether or not the synthesis process other than the audio stream # 2 is permitted by the user setting.

  That is, in the present embodiment, the user uses the user interface or the like to set various settings related to speech synthesis, for example, whether to perform synthesis when there is sound data such as interactive sound, as described above. Selections can be made and such selections are accepted by the controller 34 as part of the user settings.

  In the case of this example, if the user has selected not to perform synthesis even when there is sound data such as interactive sound, it is determined NO in the process of step S16, and the process proceeds to step S15. Proceed to That is, the encoded audio stream of the audio stream # 1 selected according to the PlayList file is output to the outside as it is. In other words, external output of the re-encoded audio stream is prohibited. Thereafter, the process proceeds to step S21. However, the processing after step S21 will be described later.

  On the other hand, if the user has selected to perform synthesis when there is sound data such as interactive sound, it is determined YES in the process of step S16, and the process proceeds to step S17.

  In step S17, the controller 34 controls the AV decoder unit 33 to appropriately decode the audio stream # 1 and the audio stream # 2 selected according to the PlayList file, and appropriately combine them with the sound data.

  That is, the controller 34 controls the PID filters 55 and 56 and the switch 101 in the AV decoder unit 33 as appropriate, and the audio stream # 1 buffered in the buffer 52 or 53 is sent to the 1st audio decoder 75-via the switch 101. The audio stream # 2 supplied to 1 and decoded and buffered in the buffer 52 or 53 is supplied to the 2nd audio decoder 75-2 via the switch 101 to be decoded, and the mixing processing unit 102 is controlled. The audio stream synthesized by the 1st audio decoder 75-1 and the 2nd audio decoder 75-2 is appropriately executed, the mixing processor 97 is controlled, and the audio stream and the buffer 95 synthesized by the mixing processor 102 are stored in the buffer 95. Retained The process of synthesizing with the sound data is appropriately executed.

  The processed data in step S17 is output from the AV decoder unit 33 as a digital uncompressed audio signal.

  In step S18, the controller 34 determines whether or not the external device and the connection transmission path correspond to an uncompressed audio signal having the same Ch number as the Ch number (channel number) of the audio stream # 1.

  The case of not corresponding to the uncompressed audio signal means that re-encoding is necessary. In such a case, the controller 34 determines NO in the process of step S18. In step S19, the switch 45 is controlled so that the data processed in step S17 is re-encoded and output to the outside.

  Specifically, the processed data in step S17, that is, the digital uncompressed audio signal is directly supplied to the switch 45 (supplied to the lower end of the three input ends in FIG. 37), and the audio encoder. 41. Then, the uncompressed audio signal supplied to the audio encoder 41 is subjected to an encoding process (re-encoding process) there, and a re-encoded audio stream obtained as a result is supplied to the switch 45 (3 in FIG. 37). Supplied to the middle of the two inputs).

  Therefore, in the process of step S19, the input of the switch 45 is switched to the center end of the three input ends in FIG. 37, and the digital re-encoded audio stream supplied to the center end is It is output from the output end and output to the outside via the digital audio signal interface 112.

  The processed data in step S17, that is, the digital uncompressed audio signal is also supplied to the D / A converter 43, where it is converted into an analog uncompressed audio signal, and then the analog audio signal interface 111. Is output to the outside.

  On the other hand, when the external device and the connection transmission path correspond to the uncompressed audio signal having the same Ch number as that of the audio stream # 1, it can be output to the outside without being re-encoded. . Therefore, in such a case, the controller 34 determines YES in the process of step S18, and in step S20, the switch 34 outputs the processed data of step S17 to the outside as an uncompressed audio signal. 45 is controlled.

  Specifically, the processed data in step S17, that is, the digital uncompressed audio signal is directly supplied to the switch 45 (supplied to the lower end of the three input ends in FIG. 37), and the audio encoder. 41. Then, the uncompressed audio signal supplied to the audio encoder 41 is subjected to an encoding process (re-encoding process) there, and a re-encoded audio stream obtained as a result is supplied to the switch 45 (3 in FIG. 37). Supplied to the middle of the two inputs).

  Therefore, in the process of step S20, the input of the switch 45 is switched to the lower end of the three input ends in FIG. 37, and the digital uncompressed audio signal supplied to the lower end is supplied from the output end of the switch 45. It is output to the outside via the digital audio signal interface 112. In other words, external output of the re-encoded audio stream is prohibited.

  The processed data in step S17, that is, the digital uncompressed audio signal is also supplied to the D / A converter 43, where it is converted into an analog uncompressed audio signal, and then the analog audio signal interface 111. Is output to the outside.

  In this way, when any one of steps S15, S19, and S20 is executed, that is, according to the PlayItem to be reproduced in the PlayList, the audio signal is externally output in any of the various forms described above. If output, the process proceeds to step S21.

  In step S21, the controller 34 determines whether there is a next PlayItem.

  If there is a next PlayItem, it is determined YES in the process of step S21, the process returns to step S12, and the subsequent processes are repeated.

  On the other hand, if there is no next PlayItem, it is determined as NO in the process of step S21, and the process proceeds to step S22.

  In step S22, the controller 34 determines whether or not the reproduction is finished.

  If it is determined in step S22 that the playback has ended, the audio signal playback process ends.

  On the other hand, if it is determined in step S22 that the reproduction is not yet finished, the process returns to step S10, and the subsequent processes are repeated.

  The audio signal reproduction process has been described above independently of the video signal reproduction. However, as described above, the audio signal reproduction process is often executed in parallel with the video signal reproduction process. Therefore, in such a case, the result of the video signal reproduction process can also be incorporated into the determination process of the audio signal reproduction process.

  For example, as the determination processing in step S14 in FIG. 38 described above, that is, as determination processing for determining whether or not there is audio data for synthesis to be synthesized as sound data, interactive graphics loaded in the memory in advance such as for a pop-up menu A determination process of analyzing data such as a stream and performing a determination based on the analysis result can be employed.

  In other words, the interactive graphics stream is combined when the number of parts such as buttons displayed on the screen or a user's operation for selecting the button is determined by receiving a command from the outside of the device. Audio data identification information and the like. Therefore, the controller 34 controls the AV decoder unit 33 and the like to analyze the interactive graphic stream, thereby confirming the presence or absence of graphics data that may be accompanied by a voice synthesis process, such as a button. If there is graphics data, it can be further confirmed from the identification information of the audio data whether the graphics data is linked with the audio data, and the audio data is reproduced in the corresponding playback section of the interactive graphics stream. It is possible to determine whether or not there is a possibility that the synthesis process is performed.

  As described above, the controller 34 appropriately controls the switch 45, and analyzes the encoded audio stream to be reproduced (audio stream # 1 in the above-described example), thereby determining whether or not there is a re-encoding effect (described above). In this example, whether or not a re-encoded audio stream is output from the audio encoder 41) is strictly determined, and output of the re-encoded audio stream is avoided as much as possible. As a result, an unnecessary re-encoded audio stream is not output, and the effect that the sound quality of the content can be maintained to the maximum can be achieved.

  Hereinafter, this effect will be described in more detail.

  That is, in the playback device disclosed in Japanese Patent Application No. 2006-147981 already filed by the present applicant, the switch 45 does not exist, and whether or not the speech synthesis process is actually performed, When the flag indicating the presence of data to be synthesized is set in the encoded audio stream to be reproduced (corresponding to audio stream # 1 in the above example) (YES in the process of step S11 in the example of FIG. 38 described above). In other words, the re-encoding process (corresponding to the process of step S19 in the example of FIG. 38 described above) is always performed. For this reason, the re-encoded audio stream is always output even though the voice synthesis process is not actually performed, and as a result, the sound quality of the reproduced voice is unnecessarily degraded. .

  When the above flag is set, re-encoding is performed regardless of the number of channels (number of channels) of the main audio data, that is, the encoded audio stream to be reproduced (corresponding to audio stream # 1 in the above example). It was done. For this reason, for example, an encoded audio stream of 2-channel stereo sound is decoded, and re-encoding is performed even when synthesis processing is performed on the resulting uncompressed audio signal. Even if the connection interface (eg S / PDIF) supports transmission of 2-channel uncompressed audio data, unnecessary re-encoding is performed, resulting in degraded audio quality and re-encoding. If the Ch number of the audio stream does not match the source, the correct sound field was not generated in the external decoder.

  Therefore, the playback device 20 of the present embodiment plays back the encoded audio stream to be played back (for example, the audio stream # 1 in the above example) (in parallel with the playback of the video signal as well as the generation of the audio alone). Whether or not speech synthesis processing is actually performed based on the setting contents and information obtained from the encoded audio stream to be played back. Is strictly determined (see, for example, steps S12 to S16 in FIG. 38). The playback apparatus to which the present invention is applied permits the external output of the re-encoded audio stream only when the speech synthesis process is actually performed (see, for example, the process of step S19 in FIG. 38). In other cases, external output of the re-encoded audio stream is prohibited, and the original encoded audio stream (audio stream # 1) and the like are output to the outside (see, for example, step S15 in FIG. 38, for example). )

  A more generalized representation of the configuration of the playback apparatus 20 to which the present invention is applied is as follows, for example.

  That is, for example, a playback apparatus to which the present invention is applied includes a playback data acquisition unit that acquires playback data including an encoded audio stream, a decoding unit that performs a decoding process on the encoded audio stream, and a decoding unit. The uncompressed audio signal obtained as a result of the decoding process is combined with synthesis means for synthesizing data, which is data to be synthesized with the uncompressed audio signal, and the uncompressed audio signal synthesized with the synthesis data by the synthesis means Re-encoding means for performing re-encoding processing, and further, there is an encoded audio stream to be reproduced and data for synthesis with this, and the user permits or instructs the synthesis In the first case, the output of the re-encoded audio stream obtained as a result of the re-encoding process by the re-encoding means is permitted, and otherwise In the second case, it may be configured to include a control means for prohibiting the output of the re-encoded audio stream.

  A playback apparatus to which the present invention having such a characteristic configuration is applied detects that speech synthesis processing is not performed with higher accuracy than the playback apparatus disclosed in Japanese Patent Application No. 2006-147981. When such detection is performed, for example, the encoded audio stream recorded on the medium (audio stream # 1 in the above example) is output as it is without outputting the re-encoded audio stream. Therefore, unnecessary sound quality deterioration can be prevented.

  Furthermore, the playback device 20 according to the present embodiment has the same number of channels as the number of channels that are connected to the external device and the external device are the same as the encoded audio stream to be played back, even when speech synthesis processing is performed. When it is determined that the transmission and reception of the uncompressed audio signal is supported, the uncompressed audio signal after the speech synthesis process can be output as it is without being re-encoded (for example, step of FIG. 38). As a result, unnecessary deterioration of sound quality can be prevented.

  That is, regarding the playback apparatus to which the present invention expressed in more general terms is applied, the control means described above further determines the number of channels (number of channels) included in the encoded audio stream to be played back, and Even in the third case, in the third case where the externally connected transmission path and the external device support transmission or reception of the uncompressed audio signal of the number of channels, the output of the re-encoded audio stream is Can be banned. That is, in this case, the uncompressed audio signal obtained by synthesizing the synthesis data by the synthesizing means is output to the outside as it is, and as a result, unnecessary deterioration in sound quality can be prevented.

  Furthermore, the controller 34 can execute the following processing by controlling the switch 45.

  In other words, even when the external output of the re-encoded audio stream from the audio encoder 41 is not prohibited, the controller 34 performs the speech synthesis process of the uncompressed audio signal output from the AV decoder unit 33. Only the part actually performed (for example, Access Unit unit) can be replaced with a re-encoded audio stream and output to the outside, and the other part can be output to the outside as an uncompressed audio signal.

  Specifically, for example, it is assumed that the encoded audio stream 201 as shown in FIG. 39 is acquired as the audio stream # 1 to be reproduced by the reproduction data acquisition unit 31. Here, in FIG. 39, AUk (k is an integer value of 0 to 10) indicates an Access Unit. Therefore, in the description of FIG. 39 or FIG. 40 described later, the Access Unit is appropriately referred to as AUk so as to be distinguished from other units.

  The encoded audio stream 201 is subjected to speech synthesis processing in the mixing processing unit 102 or 97. However, the speech synthesis processing is not performed on all the portions, and here, Suppose that speech synthesis processing is performed only for the section 211 of AU3 to AU7.

  In this case, the controller 34 monitors the processing of the mixing processing units 102 and 97 for each Access Unit. In other words, after the encoded audio stream 201 is converted into an uncompressed audio signal by the 1st audio decoder 75-1, the speech synthesis process is actually performed in any section (for example, Access Unit) of the uncompressed audio signal. Information indicating whether it has been applied is transmitted from the mixing processing units 102 and 97 to the controller 34.

  Accordingly, in the example of FIG. 39, the controller 34 can recognize that the section where the speech synthesis process is actually performed is the section 211 of AU3 to AU7 based on this information.

  Therefore, the controller 34 can output the encoded audio stream 202 as shown in FIG. 40 by appropriately controlling the switching of the switch 45 based on the recognition result. Specifically, the controller 34 appropriately switches the switch 45 so that an encoded audio stream (audio stream) provided from the switch 101 is provided for a section from AU0 to AU2 where speech synthesis processing is not actually performed. The section 220 in # 1) is output to the outside, and the section 221 in the re-encoded audio stream from the audio encoder 41 is output to the outside for the sections from AU3 to AU7 in which speech synthesis processing is actually performed. As for the section from AU8 to AU10 in which speech synthesis processing is not actually performed, the section 222 in the encoded audio stream (audio stream # 1) provided from the switch 101 can be output to the outside.

  Next, with reference to FIG. 41 and FIG. 42, a method for manufacturing the recording medium 21 on which data that can be reproduced by the reproducing apparatus 20 is recorded will be described by taking the case where the recording medium 21 is a disc-shaped recording medium as an example. .

That is, as shown in FIG. 41, a master disc made of glass or the like is prepared, and a recording material made of photoresist or the like is applied thereon. As a result, a recording master is produced.

  Then, as shown in FIG. 42, video data in a format that can be played back by the playback device 20 and encoded by the encoding device (video encoder) is stored in a temporary buffer in the software production processing unit, and is recorded by the audio encoder. The encoded audio data is stored in the temporary buffer, and data other than the stream (for example, Indexes, Playlist, PlayItem, etc.) encoded by the data encoder is further stored in the temporary buffer. Video data, audio data, and data other than the stream stored in each buffer are multiplexed together with a synchronization signal by a multiplexer (MPX), and an error correction code is added by an error correction code circuit (ECC). Is done. Then, predetermined modulation is applied by a modulation circuit (MOD), and software that is once recorded on a magnetic tape or the like according to a predetermined format and recorded on a recording medium 21 that can be played back by the playback device 20 is manufactured.

  This software is edited (premastered) as necessary, and a signal in a format to be recorded on the optical disc is generated. As shown in FIG. 41, the laser beam is modulated in response to the recording signal, and this laser beam is irradiated onto the photoresist on the master. Thereby, the photoresist on the master is exposed in accordance with the recording signal.

  Then, this master is developed and pits appear on the master. The master plate thus prepared is subjected to a process such as electroforming to produce a metal master plate in which pits on the glass master plate are transferred. A metal stamper is further produced from this metal master, and this is used as a molding die.

  A material such as PMMA (acrylic) or PC (polycarbonate) is injected into this molding die by, for example, injection and fixed. Alternatively, 2P (ultraviolet curable resin) or the like is applied on the metal stamper and then cured by irradiation with ultraviolet rays. Thereby, the pits on the metal stamper can be transferred onto the replica made of resin.

  A reflective film is formed on the replica thus generated by vapor deposition or sputtering. Alternatively, a reflective film is formed on the generated replica by spin coating.

  Thereafter, the inner and outer diameters of the disk are processed, and necessary measures such as bonding two disks are performed. Further, a label is attached or a hub is attached and inserted into the cartridge. In this way, the recording medium 21 on which data reproducible by the reproducing apparatus 20 is recorded is completed.

  The series of processes described above can be executed by hardware or can be executed by software. In this case, the processing described above is executed by a personal computer 500 as shown in FIG.

  43, a CPU (Central Processing Unit) 501 performs various processes according to a program stored in a ROM (Read Only Memory) 502 or a program loaded from a storage unit 508 to a RAM (Random Access Memory) 503. Execute. The RAM 503 also appropriately stores data necessary for the CPU 501 to execute various processes.

  The CPU 501, ROM 502, and RAM 503 are connected to each other via an internal bus 504. An input / output interface 505 is also connected to the internal bus 504.

  The input / output interface 505 includes an input unit 506 including a keyboard and a mouse, a display including CRT and LCD, an output unit 507 including a speaker, a storage unit 508 including a hard disk, a modem, a terminal adapter, and the like. A communicator 509 is connected. A communication unit 509 performs communication processing via various networks including a telephone line and CATV.

  A drive 510 is connected to the input / output interface 505 as necessary, and a removable medium 521 made up of a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is appropriately mounted, and a computer program read therefrom is It is installed in the storage unit 508 as necessary.

  When a series of processing is executed by software, a program constituting the software is installed from a network or a program storage medium.

  As shown in FIG. 43, this program storage medium is configured only by a package medium including a removable medium 521 on which a program is recorded, which is distributed to provide a program to the user, separately from the computer. Rather, it is provided with a ROM 502 in which a program is recorded and a hard disk including a storage unit 508 provided to the user in a state of being preinstalled in the apparatus main body.

  In the present specification, the step of describing a computer program includes not only processing performed in time series according to the described order but also processing executed in parallel or individually even if not necessarily processed in time series. Is also included.

  Further, the drive 510 can read data recorded on the attached removable medium 521 and can record data on the attached removable medium 521. The personal computer 500 has the same function as the software production processing unit described with reference to FIG. 42 (for example, the CPU 501 is used to execute a program for realizing the same function as the software production processing unit). It goes without saying that it is possible.

  That is, the personal computer 500 generates data similar to the data generated by the software production processing unit described with reference to FIG. 42 by the processing of the CPU 501 or uses FIG. 42 generated by an external device. Data similar to the data generated by the software production processing unit described above can be acquired via the communication unit 509 or the removable medium 521 attached to the drive 510. Then, the personal computer 500 serves as a recording device that records data similar to the data generated or acquired by the software production processing unit described with reference to FIG. 42 on the removable medium 521 attached to the drive 510. Function can be realized.

  The embodiment of the present invention is not limited to the above-described embodiment, and various modifications can be made without departing from the gist of the present invention.

It is a figure explaining the conventional speech synthesis. It is a figure for demonstrating the reproducing | regenerating apparatus to which this invention is applied. It is a figure which shows the example of the application format on the recording medium with which the reproducing | regenerating apparatus to which this invention is applied is mounted | worn. It is a figure explaining an index table and a navigation object. It is a figure explaining the structure of a main path and a sub path. It is a figure explaining the example of a main path and a sub path. It is a figure explaining another example of a main path and a sub path. It is a figure explaining the data structural example of the data which can be reproduced | regenerated with a reproducing | regenerating apparatus. It is a figure which shows the syntax of index.bdmv. It is a figure which shows indexes syntax. It is a figure which shows indexes syntax. It is a figure which shows indexes syntax. It is a figure for demonstrating the data structure of a PlayList file. It is a figure for demonstrating the syntax of AppInfoPlayList (). It is a figure for demonstrating the syntax of AppInfoPlayList (). It is a figure which shows the syntax of PlayList (). It is a figure which shows the syntax of PlayList (). It is a figure which shows the syntax of PlayList (). It is a figure which shows the syntax of SubPath (). It is a figure explaining SubPath_type. It is a figure which shows the syntax of SubPlayItem (i). It is a figure which shows the syntax of PlayItem (). It is a figure which shows the syntax of PlayItem (). It is a figure which shows the syntax of PlayItem (). It is a figure which shows the syntax of STN_table (). It is a figure which shows the syntax of stream_entry (). It is a figure which shows the syntax of stream_attribute (). It is a figure explaining stream_cording_type. It is a figure explaining video_format. It is a figure explaining frame_rate. It is a figure explaining aspect_ratio. It is a figure explaining audio_presentation_type. It is a figure explaining sampling_frequency. It is a figure explaining Character code. It is a figure explaining the example of the stream number table showing the relationship between the audio | voice signal provided to a user, and a caption signal. It is a figure which shows the syntax of sound.bdmv. It is a block diagram which shows the structural example of the reproducing | regenerating apparatus to which this invention is applied. It is a flowchart for demonstrating an audio signal reproduction | regeneration process. It is a figure explaining another example of control of the controller of FIG. It is a figure explaining another example of control of the controller of FIG. It is a figure for demonstrating manufacture of the recording medium which recorded the data which can be reproduce | regenerated with a reproducing | regenerating apparatus. It is a figure for demonstrating manufacture of the recording medium which recorded the data which can be reproduce | regenerated with a reproducing | regenerating apparatus. It is a figure which shows the structure of a personal computer.

Explanation of symbols

  20 playback device, 31 playback data acquisition unit, 32 switch, 33 AV decoder unit, 34 controller, 41 audio encoder, 45 switch, 51 to 54 buffer, 55, 56 PID filter, 57, 58 switch, 71 background decoder, 72 Video decoder, 73 presentation graphics decoder, 74 interactive graphics decoder, 75 audio decoder, 76 Text-ST composition, 77 switch, 91 background plane generation unit, 92 video plane generation unit, 93 presentation graphics plane generation unit, 94 interactive graphics plane generator, 95 buffer 96 video data processor, 97 Miki Ing unit, 101 switch, 102 mixing processor

Claims (9)

Reproduction data acquisition means for acquiring reproduction data including an encoded audio stream;
Decoding means for performing a decoding process on the encoded audio stream acquired by the reproduction data acquisition means;
A synthesizing unit that synthesizes synthesis data, which is data to be synthesized with the uncompressed audio signal obtained as a result of the decoding process by the decoding unit;
Re-encoding means for performing re-encoding processing on the uncompressed audio signal synthesized with the synthesis data by the synthesis means;
The encoded audio stream to be reproduced exists in the reproduction data acquired by the reproduction data acquisition means, the combining data to be combined with the encoded audio stream exists, and the user permits or instructs the combining In the first case, output of the re-encoded audio stream obtained as a result of the re-encoding process by the re-encoding unit is permitted, and in the second case other than that, the re-encoded audio stream And a control device for prohibiting output to the outside. The said control means further permits the output to the outside of the original encoded audio stream or the uncompressed audio signal synthesized with the synthesis data in the second case. Playback device. The control means further determines the number of channels included in the encoded audio stream to be reproduced, and even in the first case described above, the number of channels determined by the external connection transmission path and the external device is determined. The playback device according to claim 1, wherein in a third case corresponding to transmission or reception of an uncompressed audio signal, output of the re-encoded audio stream is prohibited. 4. The playback device according to claim 3, wherein the control unit further permits the output of the uncompressed audio signal synthesized with the synthesis data to the outside in the first case and the third case. 5. . The control means, as at least a part of the determination process of whether the first case or the second case, the interactive graphics stream included in the reproduction data acquired by the reproduction data acquisition means The playback device according to claim 1, wherein it is determined whether or not the synthesis data to be synthesized with the encoded audio stream to be reproduced exists based on a data analysis result. The control means further recognizes a synthesized part in which the synthesis data is synthesized with the uncompressed audio signal by the synthesizing means, and corresponds to the synthesized part of the encoded audio stream to be reproduced. The playback apparatus according to claim 1, wherein a portion to be output is output in the form of the re-encoded audio stream, and the other portion is output in the form of the original encoded audio stream. Based on the control of the control means, in the first case, the output to the outside is switched to the re-encoded audio stream, and in the second case, the output to the outside is switched to the original encoded audio stream, The playback apparatus according to claim 1, further comprising a switching unit that switches to the uncompressed audio signal synthesized with the synthesis data. Reproduction data acquisition means for acquiring reproduction data including an encoded audio stream;
Decoding means for performing a decoding process on the encoded audio stream acquired by the reproduction data acquisition means;
A synthesizing unit that synthesizes synthesis data, which is data to be synthesized with the uncompressed audio signal obtained as a result of the decoding process by the decoding unit;
In a reproducing method of a reproducing apparatus, comprising: re-encoding means for performing re-encoding processing on the uncompressed audio signal synthesized with the synthesis data by the synthesizing means,
The encoded audio stream to be reproduced exists in the reproduction data acquired by the reproduction data acquisition means, the combining data to be combined with the encoded audio stream exists, and the user permits or instructs the combining In the first case, output of the re-encoded audio stream obtained as a result of the re-encoding process by the re-encoding unit is permitted, and in the second case other than that, the re-encoded audio stream A playback method that includes a step that prohibits output to the outside. Reproduction data acquisition means for acquiring reproduction data including an encoded audio stream;
Decoding means for performing a decoding process on the encoded audio stream acquired by the reproduction data acquisition means;
A synthesizing unit that synthesizes synthesis data, which is data to be synthesized with the uncompressed audio signal obtained as a result of the decoding process by the decoding unit;
A program that causes a computer that controls a playback device to include a re-encoding unit that performs a re-encoding process on the uncompressed audio signal combined with the synthesis data by the combining unit,
The encoded audio stream to be reproduced exists in the reproduction data acquired by the reproduction data acquisition means, the combining data to be combined with the encoded audio stream exists, and the user permits or instructs the combining In the first case, output of the re-encoded audio stream obtained as a result of the re-encoding process by the re-encoding unit is permitted, and in the second case other than that, the re-encoded audio stream A program that includes steps that prohibit external output.

Images