JP2008059666A - Reproduction apparatus, reproduction method and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To absorb time lag between time for separate processing and time for application processing based on time of moving image data. <P>SOLUTION: A reproduction apparatus 100 is equipped with; a separate processing part 10 which separates data which is obtained by multiplexing moving image data and application data; a reproduction processing part 12 which decodes the moving image data; an application processing part 18 which processes application data; an output synthesizing part 20 which synthesizes decoded moving image data and processed application data and generates an output image. The separation processing part 10 inquires the application processing part 18 whether the application data is required. If a notice that application data is required is received, the application data is stored in the application processing part 18. If a notice that whether the application data is needed or not cannot be determined is received, the application data is temporarily stored in an intermediate buffer 14. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention generally relates to a playback device, a playback method, and a program, and more particularly, to a playback device, a playback method, and a program for processing multiplexed data.

In recent years, in order to provide an interactive function to a user, a reproducing apparatus having a function of processing application data (a program or the like) in addition to moving image data is being developed. In such a playback apparatus, the processing of moving image data and the processing of application data are synchronized. For example, a technique for efficiently executing a program executed in synchronization with reproduction of AV data or the like has been proposed (see, for example, Patent Document 1).
JP 2006-109317 A

  As described above, when the moving image data is supplied to the playback device separately from the application data, the processing of the moving image data and the processing of the application data are synchronized. However, when handling data in which moving image data and application data are multiplexed, processing for separating the data is required. In such a case, there may be a difference between the processing time of the separation process and the processing time of the application process that operates based on the processing time of the moving image data. Patent Document 1 describes a method of synchronizing AV data and a program, but does not handle data in which moving image data and application data are multiplexed as described above.

  Therefore, the present invention absorbs a difference between the processing time of the separation processing and the processing time of the application processing based on the time of the moving image data when handling the data in which the moving image data and the application data are multiplexed. An object of the present invention is to provide a playback device, a playback method, and a program that can perform the above-described processing.

  In order to solve the above problem, according to one aspect of the present invention, data obtained by multiplexing compressed and encoded moving image data and application data to be processed in synchronization with the reproduction of the moving image data is obtained. A separation processing unit that separates the moving image data from the application data, an application data storage unit that stores the application data separated by the separation processing unit, and the moving image data separated by the separation processing unit. A playback processing unit for decoding, an application processing unit for processing the application data stored in the application data storage unit, the video data decoded by the playback processing unit, and the application processed by the application processing unit Combine with data and output An output composition unit that generates an image, and the separation processing unit inquires of the application processing unit whether the application data is necessary, and notifies the application processing unit that the application data is necessary. The application data is stored in the application data storage unit, and when the application processing unit receives notification that the application data cannot be determined, the application data is temporarily stored in an intermediate buffer. When the application processing unit receives a notification that the application data is necessary after receiving notification from the application processing unit that the application data is not necessary, it is temporarily stored in the intermediate buffer. Reproducing apparatus is provided for storing the serial application data to the application data storage unit.

  Further, according to another aspect of the present invention, data obtained by multiplexing compressed and encoded moving image data and application data to be processed in synchronization with the reproduction of the moving image data, the moving image data, When it is determined that the application data is necessary, the step of decoding the separated video data, the step of determining whether or not the application data is necessary, A step of processing the application data; and a step of temporarily storing the application data in an intermediate buffer if it is not possible to determine whether the application data is necessary; and an application data temporarily stored in the intermediate buffer. Determining whether or not A step of processing the temporarily stored application data, the decoded video data, and the processed application data when it is determined that the application data temporarily stored in the inter-buffer is needed And a step of generating an output image.

  Furthermore, according to still another aspect of the present invention, the computer is configured to store data obtained by multiplexing compressed and encoded moving image data and application data to be processed in synchronization with the reproduction of the moving image data. It is determined that the application data is necessary, the function of separating the video data from the application data, the function of decoding the separated video data, the function of determining whether or not the application data is necessary, and the like. A function for processing the application data, and a function for temporarily storing the application data in an intermediate buffer and a temporary storage in the intermediate buffer when it is not possible to determine whether or not the separated application data is necessary. A function for determining whether or not the required application data is required, A function for processing the temporarily stored application data, the decoded video data, and the processed application when it is determined that the application data temporarily stored in the intermediate buffer is necessary A program for realizing a function of combining data and generating an output image is provided.

  According to the present invention, when handling data in which moving image data and application data are multiplexed, the difference between the processing time of the separation processing and the processing time of the application processing based on the time of the moving image data is absorbed. A playback device, a playback method, and a program can be provided.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

  First, a playback apparatus 100 according to an embodiment of the present invention will be described with reference to FIG. FIG. 1 is a block diagram showing the configuration of a playback apparatus 100 according to an embodiment of the present invention. Note that the playback device 100 is a device that can process data in which moving image data and application data are multiplexed, such as an HD DVD player or a personal computer.

  The playback apparatus 100 includes a separation processing unit 10, an AV playback unit (playback processing unit) 12, an intermediate buffer 14, an application data storage unit 16, an application processing unit 18, and an output composition unit 20.

  Data 22 is input to the separation processing unit 10. The data 22 is data in which moving image data (video data), audio data (audio data), and data (application data) corresponding to each of a plurality of types of applications are multiplexed. The data 22 is stored in a storage medium 24 such as an optical disc (DVD, HD DVD, etc.). The data 22 can be supplied from the storage medium 24 to the separation processing unit 10 together with the timing description 28 via an optical disk drive (not shown). The data 22 may be supplied to the separation processing unit 10 together with the timing description 28 from a network 26 such as the Internet via a network controller (not shown).

  In the present specification, the application is a function that provides an interactive function to the user in synchronization with the reproduction of the moving image data, and includes, for example, a script file, a markup file, image data, text data, audio data, and the like. Is done. The image data, text data, and audio data realize, for example, buttons and characters displayed on the display, and sound effects (click sounds, etc.) when the buttons are selected. Application data includes script files, markup files, audio data, image data, text data, and the like that constitute an application. The application data may be, for example, advanced content of the HD DVD standard (except for primary video). The application data includes an “unconditional application” that is executed unconditionally and a “conditional application” that is executed only when a predetermined condition (execution condition) is satisfied. Whether or not the execution condition corresponding to the conditional application is satisfied is determined at a timing when the conditional application is to be loaded.

  The timing description 28 describes application data corresponding to each of a plurality of types of applications to be processed in synchronization with the reproduction of moving image data, and the timing for loading and processing each of the application data. The timing description 28 is described in accordance with the time of the moving image. For example, the timing description 28 can describe that the application A is loaded between time (moving image display time) t1 and t2 and started at t3. The timing description 28 may include, for example, a corresponding execution condition (for example, when the user selects a predetermined audio track or when the user selects a subtitle in a predetermined language) for each of a plurality of types of application data. Good. In this case, when the execution condition is satisfied (for example, when the user specifies the condition), the predetermined application described in the timing description 28 is processed (executed).

  The separation processing unit 10 separates the received data 22 into moving image data, audio data, and application data. The separated moving image data and audio data are supplied from the separation processing unit 10 to the AV playback unit 12. The separated application data is output from the separation processing unit 10 to the intermediate buffer 14 or the application data storage unit 16. The output destination of the application data is determined based on an inquiry from the separation processing unit 10 to the application processing unit 18 and a notification from the application processing unit 18 to the separation processing unit 10 in response to the inquiry.

  The AV playback unit 12 decodes the moving image data and audio data received from the separation processing unit 10. The decoded moving image data and audio data are output from the AV playback unit 12 to the output synthesis unit 20.

  The intermediate buffer 14 temporarily stores application data received from the separation processing unit 10. Further, the intermediate buffer 14 outputs the temporarily stored application data to the application data storage unit 16 based on the control by the separation processing unit 10.

  The application data storage unit 16 stores (accumulates) application data received from the separation processing unit 10. The application data storage unit 16 stores application data necessary for executing an application. The stored application data is supplied to the application processing unit 18.

  The application processing unit 18 activates the application in accordance with the synchronization signal from the AV playback unit 12, thereby causing the application to operate in synchronization with the playback of the moving image data. Application loading and activation (start of execution) are performed according to the timing description 28.

  The output combining unit 20 combines the moving image data received from the AV playback unit 12 and the application data received from the application processing unit 18, and generates an output image based on the synchronization signal from the AV playback unit 12. The output generation unit 20 synthesizes and outputs the audio data received from the AV playback unit 12 and the audio data received from the application processing unit.

  The separation processing unit 10, AV playback unit 12, application processing unit 18, and output composition unit 20 shown in FIG. 1 are basically executed by software executed by a CPU (not shown) in the playback device 100. Realized. However, some of these elements may be realized by hardware. For example, the decoding function of the AV playback unit 12 can be realized by hardware controlled by software.

  Further, the id of the application constituted by each application data in the data 22 and the id of the application described in the timing description 28 are described by the same identifier. For example, “id = 3” is described in the timing description 28, and “3” is described in the identifier field in the data 22. Therefore, the separation processing unit 10 and the application processing unit 18 can check the necessity of data by using the identifier.

  According to the playback device 100 including the intermediate buffer 14 as described above, when processing data in which moving image data and application data are multiplexed, processing time of separation processing and processing of application processing based on the time of moving image data are performed. The difference between time can be absorbed.

  Next, a reproduction method according to an embodiment of the present invention will be described with reference to FIG. FIG. 2 is a schematic diagram for explaining an outline of a reproduction method according to an embodiment of the present invention. More specifically, FIG. 2 is a schematic diagram for explaining an outline of processing performed between the separation processing unit 10 and the application processing unit 18. The reproduction method according to the present embodiment can be applied to the reproduction apparatus 100 shown in FIG. Hereinafter, for convenience of explanation, a case where the reproduction method according to the present embodiment is applied to the reproduction apparatus 100 will be described as an example. In FIG. 2, the same elements as those shown in FIG. 1 are denoted by the same reference numerals, and the description thereof is omitted.

  The application processing unit 18 operates in accordance with the video data playback time. In other words, the application processing unit 18 operates in accordance with the time when the video data is actually displayed. For this reason, the processing of the application processing unit 18 is delayed from the processing of the separation processing unit 10 by the time of processing by the AV playback unit 12. For example, it is assumed that the timing description 28 describes that the application A is activated at the fifth minute (t3 = 5 minutes) from the start of video data reproduction. In this case, the separation processing unit 10 processes the video data corresponding to the fifth minute from the start of reproduction at an earlier time than the video data of the fifth minute is actually displayed. Accordingly, there is always a time lag (indicated by δ in FIG. 2) between the processing of the separation processing unit 10 and the processing of the application processing unit 18 (video display or the like).

  When the separation processing unit 10 advances the separation processing of the data 22 and processes the data after t1, and detects the application data a of the application A, the separation processing unit 10 determines whether or not the application data a should be loaded. An inquiry is made to the application processing unit 18 (inquiry 40). However, as described above, at this point, the processing of the application processing unit 18 has not reached time t1, and the application processing unit 18 may not be able to determine whether or not the application data a is necessary. This is because whether or not the application data a is necessary may depend on conditions specified by the user (selection of an English audio track, selection of Japanese subtitles, etc.).

  The application processing unit 18 returns one of three types of answers to the inquiry 40 from the separation processing unit 10. The three types of answers are “necessary”, “unnecessary”, and “possibly necessary” for the application data a.

  When it is determined that the application data a is necessary, the separation processing unit 10 performs a process of loading the application data a into the application data storage unit 16.

  When it is determined that the application data a is unnecessary, the separation processing unit 10 discards the subsequent application data a. More specifically, data having the same identifier is discarded until the end.

  If it is not possible to determine whether or not the application data a is necessary and it is determined that the application data a may be necessary, the separation processing unit 10 stores the application data a in the intermediate buffer 14 (in FIG. 2). The unconfirmed application data a1) and the separation process are advanced. When the processing time of the application processing unit 18 reaches t1, the application processing unit 18 determines again whether or not the application data a is necessary. Note that when the display of moving image data reaches t1, the processing time of the application also reaches t1. The display time of the moving image data is described in the data 22.

  When it is determined that the application data a is necessary, the application processing unit 18 notifies the separation processing unit 10 that the application data a is necessary (notification 42). When the separation processing unit 10 receives the notification 42 from the application processing unit 18, the separation processing unit 10 moves the application data a from the intermediate buffer 14 to the application data storage unit 16 (confirmed application data a2 in FIG. 2), and continues the separation processing. .

  On the other hand, when it is determined that the application data a is unnecessary, the application processing unit 18 notifies the separation processing unit 10 that the application data a is unnecessary (notification 42). In response to this notification, the separation processing unit 10 discards the application data a from the intermediate buffer 14.

  Here, application data (conditional application) that can be determined to be unnecessary will be described. For example, there are Japanese and English audio tracks, and there is unnecessary application data when switching the application to be used based on the selection of the track, and when switching the application startup according to the language of the subtitle. Will do. In the former case, when the Japanese application corresponding to the Japanese track is activated, the application data of the English application corresponding to the English track becomes unnecessary.

  In addition, when the separation processing unit 10 cannot accurately acquire the application data a due to a change in the reproduction state of the data 22 (special reproduction such as fast-forwarding), the separation processing unit 10 notifies the application processing unit 18 of the application The fact that the data a cannot be acquired correctly is notified using the path of the inquiry 40. Upon receiving this notification, the application processing unit 18 requests the AV playback unit 12 to stop playback of moving image data and audio data. Further, the application processing unit 18 performs appropriate operations such as reading application data, and separately secures application data. More specifically, the application processing unit 18 performs a process of separately reading a file or the like from the storage medium 24, for example.

  Next, the reproduction method of the present embodiment will be described in more detail with reference to FIG. FIG. 3 is a flowchart for explaining processing performed between the separation processing unit 10, the intermediate buffer 14, the application data storage unit 16, and the application processing unit 18 in the reproduction method according to the present embodiment.

  First, the separation processing unit 10 inquires of the application processing unit 18 whether or not the separated application data a should be loaded (step S50) (corresponding to the inquiry 40 shown in FIG. 2). Upon receiving this inquiry, the application processing unit 18 determines whether or not the application data a is necessary (step S52). This determination is made based on, for example, conditions specified by the user (selecting an English audio track, selecting Japanese subtitles, and the like) and the timing description 28. The determination result is returned to the separation processing unit 10 (S54, S56, or S58). More specifically, the application processing unit 18 replies to the separation processing unit 10 that the application data a is necessary, unnecessary, or possibly necessary (S54, S56, or S58).

  When it is determined that the application data a is necessary, the separation processing unit 10 loads the application data a into the application data storage unit 16 (step S54).

  When it is determined that the application data a is unnecessary, the separation processing unit 10 discards the application data a (step S56).

  When it is determined that there is a possibility that the application data a is necessary (whether or not it is necessary), the separation processing unit 10 performs the separation process while storing the application data a in the intermediate buffer 14 ( Step S58). When the application data a is stored in the intermediate buffer 14, it is determined whether or not the processing time T of the application processing unit 18 has reached the time t1 described in the timing description (step S60). The process of step S60 is repeated until the processing time T of the application processing unit 18 reaches time t1. When the processing time T of the application processing unit 18 reaches time t1, the application processing unit 18 determines again whether or not the application data a is necessary (step S62).

  If it is determined that the application data a is necessary (YES in step S62), the application processing unit 18 notifies the separation processing unit 10 that the application data a is necessary (step S64). In response to this notification, the separation processing unit 10 moves (stores) the application data a temporarily stored in the intermediate buffer 14 to the application data storage unit 16 (step S66).

  On the other hand, when it is determined that the application data a is unnecessary (NO in step S62), the application processing unit 18 notifies the separation processing unit 10 that the application data a is unnecessary (step S68). Upon receiving this notification, the separation processing unit 10 discards the application data a temporarily stored in the intermediate buffer 14 (step S70).

  According to the playback method as described above, when handling data in which video data and application data are multiplexed, the difference between the processing time of the separation process and the processing time of the application process based on the time of the video data Can be absorbed.

  Next, a playback method according to another embodiment of the present invention will be described with reference to FIG. FIG. 4 is a schematic diagram for explaining an outline of a reproduction method according to another embodiment of the present invention. In this playback method, the application to be activated is determined according to the status of the preceding application or the playback status (for example, depending on the playback language).

  FIG. 4 shows application data b of application B and application data c of application C, application data b of application B and application data c of application C temporarily stored in the intermediate buffer 14, reproduced video data and audio Data, application B to be played back, and times t1, t2, t3, t4, t5 and t6 are shown. For example, the application data b may be configured by the application data 32 and 38 shown in FIG. 1, and the application data c may be configured by the application data 35 shown in FIG.

  In order to start the application B or C at the time t5, the data b and c need to be multiplexed as data (video stream) before the time t5. Here, for convenience of explanation, it is assumed that data b of application B needs to be multiplexed until time t3. The application to be activated at time t5 (application B or C) is selected depending on the playback status at time t6 or the execution status of the preceding application.

  First, the playback situation will be described. The playback situation at time t6 depends on the playback language, for example. More specifically, the playback status includes a plurality of audio tracks (Japanese, English, commentary, etc.) and one of them is being played back, and one of the subtitles in multiple languages. Is considered. There is also a case where t5 = t6. As an example of such a case, either application B or C may be selected depending on the audio track at the timing of executing the application.

  Next, the preceding application will be described. The preceding application is an application executed before in time. For example, in the case of selecting the application E to be executed from 10 minutes by performing an operation such as the user selecting a button for the application D executed from 5 minutes to 7 minutes from the start of reproduction, Application D corresponds to the preceding application.

  At times t1 and t2, the application processing unit 18 cannot determine which one of the application data b of the application B and the application data c of the application C is necessary. Therefore, when the separation processing unit 10 detects the application data b of the application B at time t1, the separation processing unit 10 makes an inquiry (see FIGS. 2 and 3) to the application processing unit 18 based on the ID and name of the data b. Upon receiving this inquiry, the application processing unit 18 determines whether or not the application data b is necessary, and returns the determination result to the separation processing unit 10. If it is determined that the application data b is unnecessary, the application data b is not retained. On the other hand, when it is determined that the application data b is necessary, the application data b is stored in the application data storage unit 16. If it is determined that the application data b may be necessary, the application data b of the application B is stored in the intermediate buffer 14. Even if it is determined that the application data b is necessary, the application data b may be temporarily stored in the intermediate buffer 14 because it may be discarded by special reproduction or the like.

  When application data c of application C is detected at time t2, an inquiry is made to application processing unit 18 based on the ID and name of application data c (see FIGS. 2 and 3). Upon receiving this inquiry, the application processing unit 18 determines whether or not the application data c is necessary, and returns the determination result to the separation processing unit 10. If it is determined that the application data c is unnecessary, the application data c is not retained. On the other hand, when it is determined that the application data c is necessary, the application data c is stored in the application data storage unit 16. When it is determined that the application data c may be necessary (it cannot be determined whether or not it is necessary), the application data c of the application C is stored in the intermediate buffer 14. Even if it is determined that the application data c is necessary, the application data c may be temporarily stored in the intermediate buffer 14 because it may be discarded by special reproduction or the like.

  At times t3 and t4 when the last of the multiplexed application data b and c is detected, an inquiry similar to the above is performed. For example, when it is determined that the application data b is necessary at time t <b> 3, the application data b is moved from the intermediate buffer 14 to the application data storage unit 16. If it is determined that the application data b is unnecessary, the application data b is discarded from the intermediate buffer 14. Even at time t3, when it is not possible to determine whether or not the application data b is necessary, the application data b is continuously held in the intermediate buffer 14. The same processing as described above is performed for the application data c at time t4.

  When it is determined at time t6 that the application data b of the application B is necessary based on the data reproduction condition and the timing description 28, the application processing unit 18 notifies the separation processing unit 10 that the application data b is necessary. To do. Upon receipt of this notification, the separation processing unit 10 moves (stores) the application data b of the application B held in the intermediate buffer 14 to the application data storage unit 16. On the other hand, the application data c of the application C is determined to be unnecessary and is discarded from the intermediate buffer 14.

  In addition, when special reproduction (fast forward etc.) is performed such that multiplexed data cannot be taken out between times t1 and t3, application data b of application B cannot be prepared. For this reason, the separation processing unit 10 notifies the application processing unit 18 that data cannot be prepared instead of the inquiry as described above.

  According to the playback method as described above, when application data to be used is determined according to the status of the preceding application or the playback status, the handling of data multiplexed before the actual determination time is intermediate. This can be determined using queries and notifications between the buffer 14 and associated modules.

  The playback method described above can also be realized as a program that can be executed by a computer.

  In addition, this invention is not limited to the said embodiment as it is. In the implementation stage, the present invention can be embodied by changing the components without departing from the scope of the invention.

  Moreover, various inventions can be formed by appropriately combining a plurality of constituent elements disclosed in the embodiment. For example, some components may be deleted from all the components shown in the embodiment. Furthermore, constituent elements over different embodiments may be appropriately combined.

It is a block diagram which shows the structure of the reproducing | regenerating apparatus by one Embodiment of this invention. It is the schematic for demonstrating the outline | summary of the reproducing | regenerating method by one Embodiment of this invention. It is a flowchart for demonstrating in detail the reproducing | regenerating method by the said embodiment. It is the schematic for demonstrating the outline | summary of the reproducing | regenerating method by another embodiment of this invention.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 10 ... Separation processing part, 12 ... AV reproduction part (reproduction processing part), 14 ... Intermediate buffer, 16 ... Application data storage part, 18 ... Application processing part, 20 ... Output composition part, 22 ... Data, 24 ... Recording medium, 26 ... Network, 28 ... Timing description, 40 ... Inquiry, 42 ... Notification.

Claims (15)

A separation processing unit that separates, into the moving image data and the application data, data obtained by multiplexing compressed and encoded moving image data and application data to be processed in synchronization with the reproduction of the moving image data;
An application data storage unit that stores the application data separated by the separation processing unit;
A reproduction processing unit for decoding the moving image data separated by the separation processing unit;
An application processing unit that processes the application data stored in the application data storage unit;
An output composition unit that synthesizes the moving image data decoded by the reproduction processing unit and the application data processed by the application processing unit to generate an output image;
The separation processing unit inquires of the application processing unit whether the application data is necessary, and receives the notification that the application data is necessary from the application processing unit. When storing in the storage unit and receiving a notification from the application processing unit that it is not possible to determine whether or not the application data is necessary, the application data is temporarily stored in an intermediate buffer, and the application data is transmitted from the application processing unit. After receiving a notification that it is not possible to determine whether or not it is necessary, if a notification that the application data is necessary is received, the application data temporarily stored in the intermediate buffer is updated. It is stored in the application data storage unit, the reproducing apparatus.   When the application processing unit cannot determine whether or not the application data is necessary, the application processing unit determines whether or not a condition corresponding to the application data is satisfied at a time point when the application data is to be processed. The playback apparatus according to claim 1, which returns a notification as to whether the application data is necessary based on the notification.   The playback apparatus according to claim 1, wherein the application data includes a script file, a markup file, image data, and text data that constitute an application that provides an interactive function to a user in synchronization with the playback of the moving image data. .   The reproducing apparatus according to claim 1, wherein the multiplexed data is supplied to the separation processing unit via a recording medium.   The playback apparatus according to claim 1, wherein the multiplexed data is supplied to the separation processing unit via a network. Separating the data obtained by multiplexing the compressed and encoded moving image data and the application data to be processed in synchronization with the reproduction of the moving image data into the moving image data and the application data;
Decoding the separated video data;
Determining whether the application data is necessary;
If it is determined that the application data is needed, processing the application data;
If it is not possible to determine whether the application data is needed, temporarily storing the application data in an intermediate buffer;
Determining whether application data temporarily stored in the intermediate buffer is needed;
If it is determined that the application data temporarily stored in the intermediate buffer is needed, processing the application data temporarily stored;
A step of synthesizing the decoded moving image data and the processed application data to generate an output image.   The step of determining whether or not application data temporarily stored in the intermediate buffer is necessary is whether or not a condition corresponding to the application data is satisfied at the time when the application data is to be processed. The reproduction method according to claim 6, further comprising: determining whether the application data is necessary based on the determination.   The reproduction method according to claim 6, wherein the application data includes a script file, a markup file, image data, and text data that constitute an application that provides an interactive function to a user in synchronization with reproduction of the moving image data. .   The reproduction method according to claim 6, wherein the multiplexed data is supplied via a recording medium.   The reproduction method according to claim 6, wherein the multiplexed data is supplied via a network. On the computer,
A function of separating data obtained by multiplexing compressed and encoded moving image data and application data to be processed in synchronization with reproduction of the moving image data into the moving image data and the application data;
A function of decoding the separated video data;
A function of determining whether the application data is necessary;
If it is determined that the application data is necessary, a function of processing the application data;
If it is not possible to determine whether or not the separated application data is necessary, a function of temporarily storing the application data in an intermediate buffer;
A function of determining whether application data temporarily stored in the intermediate buffer is necessary;
A function of processing the temporarily stored application data when it is determined that the application data temporarily stored in the intermediate buffer is necessary;
A program that realizes a function of synthesizing the decoded moving image data and the processed application data to generate an output image.   The function of determining whether or not application data temporarily stored in the intermediate buffer is necessary is whether or not a condition corresponding to the application data is satisfied at the time when the application data is to be processed. The program according to claim 11, further comprising: a function for determining whether the application data is necessary based on the determination.   The program according to claim 11, wherein the application data includes a script file, a markup file, image data, and text data that constitute an application that provides an interactive function to a user in synchronization with reproduction of the moving image data.   The program according to claim 11, wherein the multiplexed data is supplied via a recording medium.   12. The program according to claim 11, wherein the multiplexed data is supplied via a network.

Images