JP4511952B2 - Media playback device - Google Patents

Description

  The present invention relates to a media playback device that plays back media such as video and audio, and more particularly to a media playback device that plays back received media at a time to be presented.

  In recent years, multimedia (hereinafter referred to as media) such as video (including moving images, still images, CG images, and characters) and audio (including synthesized audio) has been distributed on the Internet (network). It was. Among such media, moving images, sounds, and the like are composed of information elements having a time axis. Therefore, when outputting, it is necessary to reproduce each element in synchronization. Thus, there is known a multimedia playback device that plays back at the time when the media should be presented while synchronizing between the media when there are a plurality of types of media such as moving images and sounds (see, for example, Patent Document 1). ).

  Prior to receiving each medium, the multimedia playback apparatus disclosed in Patent Document 1 transmits the information amount of each medium to be played in advance and the load state information of this playback apparatus to the transmission side via the network. The transmission side determines the playback time of each medium based on these pieces of information, and transmits it to the multimedia playback apparatus.

  In addition, a standard for associating a time at which a medium should be reproduced with the medium is known, and a time stamp for reproducing the medium is added to each medium (Non-Patent Document 1). With this time stamp, even when a so-called fluctuation occurs in transmission speed and arrival interval when data is transmitted, it is possible to reproduce the sound with reduced quality defects such as the sound being interrupted.

  Conventionally, as shown in FIG. 10, such a multimedia playback apparatus includes a demultiplexing unit 1010 that separates a plurality of received media, and a plurality (two in the figure) of media playback that plays back each separated media. Device 1030. A media playback device 1030 shown in FIG. 10 inputs time information to be played back by media and data such as a compressed stream, and decodes or synthesizes them to play back as video or audio. For example, as shown in FIG. 10, the media playback device 1030 stores received data (compressed stream) to absorb fluctuations in the arrival interval of data generated on the transmission path, and the input buffer 1031 Reproducing means 1032 for reproducing (decoding or synthesizing) the compressed stream output from the buffer 1031 is provided.

  Alternatively, as shown in FIG. 11, in addition to the input buffer 1031 and the playback unit 1032 described above, a media playback apparatus 1130 provided with an output buffer 1133 that accumulates video and audio played back by the playback unit 1032 until the time to be output is output. Also known is a multimedia playback device 1100 having

The reproduction means 1032 shown in FIG. 10 and FIG. 11 has a time required for reproduction (processing delay) rather than the reproduction time so that video and audio are output at the designated reproduction time (output time) of the medium. The playback process starts as soon as possible. That is, as shown above the time axis (t) shown in FIG. 12A, the reproducing means 1032 uses a predetermined delay time T _S with a fixed processing delay time (assumed delay time t ₁ ). The reproduction process is started at the start time T _P before the expected delay time t ₁ .

Further, the output buffer 1133 shown in FIG. 11 can wait for a predetermined time (reproduction margin time: maximum value t ₃ ) in a state where the stored video and audio can be instantaneously reproduced. Therefore, as shown in FIG. 12 (b), the reproduction means 1032 shown in FIG. 11 is earlier than the predetermined reproduction time T _S by the sum of the estimated delay time t ₁ and the reproduction allowance time t _3. The reproduction process is started at the start time T _P ′.
Japanese Patent No. 3179922 (paragraphs 0014 to 0062, FIG. 1) H. Schulzrinne, 3 others, "RTP: A Transport Protocol for Real-Time Applications", RFC3550 (Network Working Group Request for Comments: 3550), July 2003

However, on a general-purpose OS excluding a real-time processing system, generally, the maximum time (processing delay time) required for reproduction processing is not guaranteed. For this reason, the reproduction means 1032 shown in FIG. 10 requires the actual processing delay time t ₂ longer than the assumed delay time t ₁ for the reproduction processing, as shown below the time axis shown in FIG. when the output of video and audio, sometimes be started in the output time T _D that is delayed than the predetermined reproduction time T _S.

On the other hand, the multimedia playback device 1100 having the output buffer 1133 shown in FIG. 11 provides the actual processing delay time t ₂ by providing the playback allowance time t ₃ as shown in FIG. Even if it differs from the assumed delay time t ₁ , video and audio can be reproduced at a predetermined reproduction time T _S (output time T _S ), but the output buffer 1133 can store uncompressed video and audio. The storage device must have a large capacity.

  Further, the multimedia playback device disclosed in Patent Document 1 has a problem that the amount of media information must be calculated in advance and transmitted to the media transmission side.

  In addition, as long as it is executed on the general-purpose OS, the time (processing delay time) required for decoding the compressed stream and interpreting and synthesizing the script changes. Even when the time stamp is added, there is a problem that the media cannot be reproduced at the time of the added time stamp.

  The present invention has been made in view of the above problems, and provides a multimedia playback device capable of outputting multimedia at a predetermined playback time without increasing the buffer capacity. For the purpose.

The present invention was created to achieve the above object, and first, the media playback device according to claim 1 stores encoded media data for playing back the media at a predetermined playback time. A media playback device that plays back media based on the data, and an input buffer that plays back data output from the input buffer.
An output buffer, comprising: a processing delay calculation means, and a control unit, a starting time control means, the input buffer is provided with capacity control means, the output buffer is a structure in which Ru provided with a capacity control means.

  According to such a configuration, the media playback apparatus accumulates, as media data, for example, a stream or script transmitted via a network in the input buffer, and playback processing (decoding) of data output from the input buffer by the playback means. Processing or composition processing). For example, the reproduction means decodes an MPEG-2 stream to reproduce an image, or decodes an MPEG-1 Audio Layer stream to reproduce an audio. It also interprets TVML (TV program making language) scripts to synthesize video and audio, and interprets text scripts to synthesize video and audio.

Then, the media playback device stores the data output from the playback means in the output buffer, and holds the stored data for a predetermined time. Then, when the media playback apparatus detects the input of data from the input buffer to the playback means by the processing delay calculation means , the media playback apparatus calculates the processing delay time indicating the time required for data playback by the playback means by measurement . Here, the calculation of the processing delay time can be obtained from, for example, the count value when the clock counting is started and the output from the reproducing unit is detected when the input to the reproducing unit is detected. Then, the media playback device designates the time for starting the process for playing back the media in the next process based on the current process delay time calculated by the process delay calculation means by the control means , and inputs Specifies the amount of data stored in the buffer and output buffer, respectively . Thus, start time control means, the time specified by the control unit, starts a process for reproducing media.

Its to the media playback device, the capacity control means of the input buffer to control the amount of data accumulated, the amount of space specified by the control means.

Its to the media playback device, the capacity control means of the output buffer, and controls the amount of data accumulated, the amount of space specified by the control means.

Further, in the media reproducing apparatus, the control means, the data a time for processing for reproducing media, at a given than the reproduction time, the processing delay time of the time calculated by the processing delay calculating means and the output buffer Is specified as an earlier time in the next processing by the sum of the predetermined time for which is held .

Then, the start time controller starts the process for playing media at the specified time by the control unit, after the processing delay time at the output buffer, since it is held for a predetermined time, predetermined reproduction Media can be played back and output at the time. This predetermined time is a time during which data stored in the output buffer can be held in a state where it can be instantaneously reproduced.

Further , according to this configuration, the media playback device starts the playback process early when the current processing delay time becomes larger than a predetermined value. That is, in the next processing, the data accumulated in the input buffer is output early. As a result, the amount of data stored in the input buffer is small in the next processing . Therefore, when the current processing delay time becomes greater than a predetermined value , the media playback device performs control to reduce the amount of data stored in the input buffer in the next processing . On the other hand, when the current processing delay time becomes smaller than a predetermined value, the media playback device starts the playback processing later in the next processing . That is, in the next processing, the data accumulated in the input buffer is output later. As a result, the amount of data stored in the input buffer must be large in the next processing . Therefore, when the current processing delay time becomes smaller than a predetermined value , the media playback device performs control to increase the amount of data stored in the input buffer in the next processing . As a result, the media can be played back at a predetermined playback time without increasing the capacity of the input buffer.

Further, the media playback device, the control unit, when the current processing delay time calculated by the processing delay calculating means is larger than a predetermined value, in the next process, reduce the volume of data stored in the buffer And to increase the amount of data stored in the output buffer. Further, the control means designates to increase the capacity of data accumulated in the input buffer in the next process when the current processing delay time calculated by the processing delay calculation means is smaller than a predetermined value, In addition, it is specified to reduce the amount of data stored in the output buffer.

According to this configuration, the media playback device starts the playback process early in the next process when the current processing delay time becomes greater than a predetermined value. That is, in the next processing, the data stored in the input buffer is output earlier, so the capacity of the data stored in the input buffer is reduced. In this case, if not much time is required for the reproduction process, it is necessary to wait until a predetermined reproduction time. Therefore, in this next processing, control is performed to increase the capacity of data stored in the output buffer. On the other hand, when the current processing delay time becomes smaller than a predetermined value, the media playback device starts the playback processing later in the next processing . That is, in the next processing, the data stored in the input buffer is output later, so the capacity of the data stored in the input buffer is increased. In this case, even if the reproduction process ends early, the time until the predetermined reproduction time decreases because the time at which the reproduction process starts is late. Therefore, in this next processing, control is performed to reduce the amount of data stored in the output buffer. As a result, the media can be played back at a predetermined playback time without increasing the capacity of the output buffer.

According to a second aspect of the present invention, there is provided a media playback apparatus, wherein an input buffer for storing encoded media data for playing back media at a predetermined playback time, and playback for playing back the data output from the input buffer. Means for reproducing media based on the data, comprising: an output buffer; a load measuring means; a control means; and a start time control means. The input buffer includes a capacity control means. And the output buffer includes a capacity control means .

According to such a configuration, the media playback device stores the data output from the playback means in the output buffer, and holds the stored data for a predetermined time. Then, the media playback device measures a processing load, which is a processing load for playing back the media, by the load measuring means. Here, the processing load is, for example, a CPU load factor indicated by a real value of 0.0 when there is no load and 1.0 at the maximum load in the CPU of the media playback device. In the media playback device, the control means designates the time to start the process for playing back the media in the next process based on the current processing load fluctuation measured by the load measuring means. Thus, start time control means, the time specified by the control unit, starts a process for reproducing media.

Further , according to this configuration, the media playback device designates the capacity of data stored in the input buffer and the output buffer based on the processing load measured by the load measuring unit by the control unit. Then, the media playback device controls the capacity of the stored data to the capacity designated by the control means by the capacity control means of the input buffer.

Its to the media playback device, the capacity control means of the output buffer, and controls the amount of data accumulated, the amount of space specified by the control means.

Further, the media playback device, the control unit, when the current processing load measured by the load measuring means is greater than a predetermined value, in the next process, so as to reduce the volume of data stored in the buffer And to increase the amount of data stored in the output buffer. In addition, when the current processing load measured by the load measuring means is smaller than a predetermined value, in the next processing, it is specified to increase the amount of data stored in the input buffer and stored in the output buffer. Specifies that the amount of data to be reduced is reduced.

  According to such a configuration, the media playback device starts the playback process early when the processing load becomes larger than a predetermined value. That is, the data stored in the input buffer is output early. As a result, the amount of data stored in the input buffer can be small. Therefore, the media playback device performs control to reduce the volume of data stored in the input buffer. On the other hand, the media playback device starts the playback process later when the processing load becomes smaller than a predetermined value. That is, the data accumulated in the input buffer is output later. As a result, the amount of data stored in the input buffer must be large. Therefore, the media playback device performs control to increase the capacity of data stored in the input buffer. As a result, the media can be played back at a predetermined playback time without increasing the capacity of the input buffer.

Further , according to this configuration, the media playback device starts the playback process early in the next process when the current processing load becomes greater than a predetermined value. That is, in the next processing, the data stored in the input buffer is output earlier, so the capacity of the data stored in the input buffer is reduced. In this case, if not much time is required for the reproduction process, it is necessary to wait until a predetermined reproduction time. Therefore, in this next processing, control is performed to increase the capacity of data stored in the output buffer. On the other hand, when the current processing load becomes smaller than a predetermined value, the media playback device starts the playback processing later in the next processing . That is, in the next processing, the data stored in the input buffer is output later, so the capacity of the data stored in the input buffer is increased. In this case, even if the reproduction process ends early, the time until the predetermined reproduction time decreases because the time at which the reproduction process starts is late. Therefore, in this next processing, control is performed to reduce the volume of data stored in the output buffer. As a result, the media can be played back at a predetermined playback time without increasing the capacity of the output buffer.

According to the first or second aspect of the present invention, in the media playback apparatus, the time when the control means starts the process for playing back the media is flexibly changed, so that the processing load is changed and played back. Even if the time required for this process fluctuates, video and audio can be reproduced and output at a predetermined reproduction time. Further, since the processing delay time or processing load required for the control means is generated in the media playback device, it is necessary to perform complicated control such as receiving information from the media transmission side in advance as in Patent Document 1. do not do. Therefore, by describing the time stamps defined in RFC3550 in a plurality of media, it becomes possible to reproduce and output these media in synchronization with a predetermined time without increasing the buffer capacity.

Further, according to the invention described in claim 1 or claim 2, in the media reproducing apparatus, the control means may control the amount of data stored in the buffer, the capacity of the input buffer larger ones There is no need. As a result, the manufacturing cost can be reduced.

Further, according to the invention described in claim 1 or claim 2, in the media reproducing apparatus, when controlling the amount of data that the control unit is stored in the output buffer, the capacity of the output buffer larger ones There is no need. As a result, the manufacturing cost can be reduced.

Further, according to the invention described in claim 1 or claim 2, in the media reproducing apparatus, the control means may advance the time to start the process for playing media, video Ya to a predetermined reproduction time Audio can be reproduced and output.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.
(First Reference Embodiment)
[Configuration of multimedia playback device]
FIG. 1 is a block diagram of a multimedia playback apparatus according to the first reference embodiment.
The multimedia playback apparatus 1 is a multi-media such as video (including moving images, still images, CG images, and characters) and audio (including synthesized audio) that is stream-transmitted via a communication line such as an IP (Internet Protocol) network. A medium (hereinafter referred to as a medium) is reproduced at a time indicated by an input time stamp.

As shown in FIG. 1, the multimedia playback device 1 includes a demultiplexing unit 50 and a plurality (two in the figure) of media playback devices 100.
The demultiplexing unit (demultiplexing means) 50 separates the time information to be reproduced by the media and multimedia data (hereinafter simply referred to as data) such as a stream in which video and audio are compressed. Thereby, a plurality of media and the clock signal ck are separated.
The media playback device 100 plays back the media separated by the demultiplexing unit 50 at the time when they should be presented while synchronizing with the clock signal ck. The media playback device 100 includes an input buffer 101, playback means 102, processing delay calculation means 103, and control means 104.

  The input buffer 101 accumulates data such as a compressed stream and outputs it to the reproduction means 102. This input buffer 101 accumulates data such as a compressed stream, so that fluctuations in the arrival interval of data occurring on the transmission path can be absorbed. The input buffer 101 performs sequence control and error correction of IP packets that have transmitted a compressed stream, takes out a time stamp, and outputs a stream (or script) in which payload portions are combined. The input buffer 101 includes a capacity control unit 111. This capacity control unit (capacity control means) 111 controls the capacity of data stored in the input buffer 101 to the capacity specified by the control means 104.

  The playback unit 102 performs decoding processing of the compressed stream output from the input buffer 101, or interpretation and synthesis processing (hereinafter collectively referred to as playback processing) of the script, and outputs the output device 150 including a monitor, a speaker, and the like. Output. The reproduction means 102 includes a start time control unit 112. The start time control unit 112 starts a reproduction process of video or audio to be reproduced at a certain time at a time (hereinafter referred to as a decoding start time) that is earlier than the time by a predetermined time designated by the control unit 104. Is to control. Although the output device 150 is separated from the multimedia playback device 1, it is needless to say that the output device 150 may be integrated.

The processing delay calculation unit 103 calculates a time required for the reproduction process by the reproduction unit 102 (hereinafter referred to as a processing delay time) and notifies the control unit 104 of the time. When the processing delay calculation unit 103 detects the input of the stream to the reproduction unit 102, for example, the processing delay calculation unit 103 starts to count the clock by a Query Performance Counter function that is a Windows (registered trademark) API (Application Program Interface), and video and audio are output. The processing time until measurement is measured, and the measurement result is set as the processing delay time.
As will be described later, the control means 104 designates the capacity of data stored in the input buffer 101 and the decoding start time of the reproduction means 102 based on the processing delay time notified from the processing delay calculation means 103. It is.

[Operation of multimedia playback device]
Next, referring to FIG. 2 (see FIG. 1 as appropriate), the operation of the multimedia playback apparatus according to the first reference embodiment will be described. FIG. 2 is a flowchart of the control means of the multimedia playback apparatus according to the first reference embodiment of the present invention.

  First, the control means 104 sets the initial value of the processing delay time to D and holds this value (step S1). For example, if the processing delay time of 1 GOP (Group of Picture) of MPEG-2 is 10 ms on average, this value (10 ms) is held as D. Then, the control unit 104 determines whether or not a processing delay time is input (D_IN) from the processing delay calculation unit 103 (step S2). Normally, this processing delay time D_IN changes according to the change in the processing load of the multimedia playback device 1, and is a value around 10 ms, for example. Subsequently, the control unit 104 compares the input processing delay time D_IN with the held processing delay time D to determine whether it is large or small (step S3).

When the input processing delay time D_IN is larger than the held processing delay time D (D_IN> D), that is, the processing load that is the processing load for reproducing the media is larger than the previous time (initial value is the first time). In this case, the control means 104 designates a time that the decoding start time T _Q is advanced by (D_IN) from the reproduction time T _S (step S4), and sets the capacity of data stored in the input buffer 101 to (D_IN). -D) The capacity reduced by xRate is designated (step S5). Here, Rate is the encoding rate of the received stream. Thus, the start time controller 112 of the reproducing means 102, advancing the decoding start time T _Q, also, the capacity control section 111 of the input buffer 101, reduces the volume of data stored in the buffer 101.
Following step S5, the control means 104 sets the value of the input processing delay time (D_IN) to D again (step S6) and returns to step S2. Steps S4 and S5 may be processed in the reverse order or in parallel.

In step S3, when the input processing delay time (D_IN) is smaller than the held processing delay time D (D_IN <D), that is, when the processing load is smaller than the previous time (initial value is the first time). Similarly to step S4, the control means 104 designates a time earlier than the reproduction start time T _Q by (D_IN) than the reproduction time T _S (step S7), and the capacity of data stored in the input buffer 101 Is specified by (D−D_IN) × Rate (step S8), and the process proceeds to step S6. Thus, the start time controller 112 of the reproducing means 102, advancing the decoding start time T _Q, also, the capacity control section 111 of the input buffer 101 increases the amount of data stored in the buffer 101. Steps S7 and S8 may be processed in the reverse order or in parallel.

  In step S3, when the input processing delay time (D_IN) is equal to the held processing delay time D (D_IN = D), that is, when the processing load is the same as the previous time (initial value is the first time), the control means 104 returns to step S2 and waits until the next input (D_IN).

According to the multimedia playback apparatus of the first reference embodiment, the processing delay time by the playback means 102 can always be monitored, and the decoding start time can be advanced by this processing delay time. Further, the capacity accumulated in the input buffer 101 can be controlled based on the processing delay time reflecting the processing load. As a result, even if the processing load fluctuates, it is possible to reproduce and output video and audio decoded or synthesized at a predetermined reproduction time while maintaining a necessary and sufficient capacity in the input buffer 101.

(Second Reference Embodiment)
[Configuration of multimedia playback device]
FIG. 3 is a block diagram of a multimedia playback apparatus according to the second reference embodiment.
As shown in FIG. 3, the multimedia playback device 2 includes a demultiplexing unit 50, a plurality (two in the figure) of media playback devices 200, and a load measuring unit 210. In addition, the same code | symbol is attached | subjected to the same structure as FIG. 1, and description is abbreviate | omitted.

  The load measuring unit 210 measures a processing load (load information) and outputs it to the media playback device 200 at a predetermined timing. Here, the load information is a CPU load factor, and is represented by a real value of, for example, 0.0 when there is no CPU load and 1.0 at the maximum load of the CPU. Further, the load measuring unit 210 outputs the load information to the media playback device 200 every 20 ms, for example.

  The media playback device 200 includes an input buffer 101, playback means 102, and control means 204. As will be described later, the control unit 204 determines the data stored in the input buffer 101 based on the load information output from the load measurement unit 210 and the processing delay time that is the time required for the reproduction processing by the reproduction unit 102. The capacity and the decoding start time by the reproducing means 102 are designated.

[Operation of multimedia playback device]
Next, referring to FIG. 4 (refer to FIG. 3 as appropriate), the operation of the multimedia playback apparatus according to the second reference embodiment will be described. FIG. 4 is a flowchart of the control means of the multimedia playback apparatus according to the second reference embodiment of the present invention.

  First, the control means 204 sets the initial value of the processing delay time to D, holds this value, sets the initial value of load information to L, and holds it (step S11). Here, for example, the initial value of the load information is set to L as 0.0 and held. And the control means 204 discriminate | determines whether there existed the input (L_IN) of the present load information from the load measurement means 210 (step S12). Subsequently, the control unit 204 compares the input load information L_IN with the stored load information L to determine the magnitude (step S13).

  When the input load information L_IN is larger than the held load information L (L_IN> L), that is, when the processing load is larger than the previous time (initial value is the first time), the control unit 204 holds the load information L_IN. Based on the processing delay time D and the load information L_IN, L, the adjustment time D ′ is calculated by the following equation (1) (step S14). Here, α varies depending on the processing capacity of the CPU and the encoding rate of the stream, and is a constant determined in advance by simulation or the like.

  D ′ = D + α × (L_IN−L) (1)

Then, the control unit 204, than the reproduction time T _S the decoding start time T _Q, designates a time at which advanced by adjusting time D 'represented by the formula (1) with (step S15), and is stored in the buffer 101 A capacity obtained by reducing the capacity of data to be reduced by β × (L_IN−L) is designated (step S16). Here, β varies depending on the processing capacity of the CPU and the encoding rate of the stream, and is a constant determined in advance by simulation or the like. Thus, the start time controller 112 of the reproducing means 102, advancing the decoding start time T _Q, also, the capacity control section 111 of the input buffer 101, reduces the volume of data stored in the buffer 101.

  Subsequent to step S16, the control unit 204 sets the value of the input load information (L_IN) to L again and holds it, and sets the value of the adjustment time D ′ to D again and holds it (step S17). ), The process returns to step S12. Note that step S15 and step S16 may be processed in reverse order or in parallel.

  In step S13, if the input load information (L_IN) is smaller than the held load information L (L_IN <L), that is, if the processing load is smaller than the previous time (initial value is the first time), control is performed. The means 204 calculates an adjustment time D ′ by the following equation (2) based on the stored processing delay time D and load information L_IN, L (step S18). Here, α is the same as that of the above-described formula (1).

  D ′ = D−α × (L−L_IN) (2)

Then, the control means 204 designates a time that the decoding start time T _Q is earlier than the reproduction time T _S by the adjustment time D ′ shown in the equation (2) (step S19) and is stored in the input buffer 101. A capacity obtained by increasing the capacity of data to be increased by β × (L−L_IN) is designated (step S20), and the process proceeds to step S17. Thus, the start time controller 112 of the reproducing means 102, advancing the decoding start time T _Q, also, the capacity control section 111 of the input buffer 101 increases the amount of data stored in the buffer 101. Note that step S19 and step S20 may be processed in the reverse order or in parallel.

  In step S13, when the input load information (L_IN) is equal to the stored load information L (L_IN = L), that is, the processing load of the multimedia playback device 1 is the same as the previous time (initial value is the first time). In that case, the control means 204 proceeds to step S17.

According to the multimedia playback device of the second reference embodiment, the decoding start time can be advanced by the adjustment time D ′ based on the processing load variation. Further, the capacity accumulated in the input buffer 101 can be controlled based on the fluctuation of the processing load. As a result, even if the processing load fluctuates, it is possible to reproduce and output video and audio decoded or synthesized at a predetermined reproduction time while maintaining a necessary and sufficient capacity in the input buffer 101.

(First Embodiment)
[Configuration of multimedia playback device]
FIG. 5 is a block diagram of the multimedia playback apparatus according to the first embodiment.
As shown in FIG. 5, the multimedia playback device 3 includes a demultiplexing unit 50 and a plurality (two in the figure) of media playback devices 300. In addition, the same code | symbol is attached | subjected to the same structure as FIG. 1, and description is abbreviate | omitted.
The media playback device 300 has the same configuration as the media playback device 100 of the first reference embodiment shown in FIG. 1 except that it includes an output buffer 310 and the control unit 304 has a different function. .

The output buffer 310 accumulates the video and audio reproduced by the reproducing means 102 until the time to be reproduced. The output buffer 310 waits for a predetermined time (reproduction allowance time) in a state where the stored video and audio can be instantaneously reproduced. Then, the video and audio stored in the output buffer 310 are immediately output at the reproduction time indicated by the time stamp. In addition, the output buffer 310 includes a capacity control unit 311. The capacity control unit (capacity control means) 311 controls the capacity of data stored in the output buffer 310 to a capacity designated by the control means 304.
As will be described later, the control unit 304, based on the processing delay time notified from the processing delay calculation unit 103, the capacity of data stored in the input buffer 101, the capacity of data stored in the output buffer 310, The decoding start time by the reproduction means 102 is designated.

[Operation of multimedia playback device]
Next, referring to FIG. 6 (see properly Figure 5), the operation of the multimedia reproducing apparatus according to the first embodiment. FIG. 6 is a flowchart of the control means of the multimedia playback apparatus according to the first embodiment of the present invention.

  First, the control unit 304 sets the initial value of the processing delay time to D and holds this value (step S21). In addition, the control unit 304 sets and retains the reproduction allowance time of the output buffer 310 as the initial value of the variable S. Then, the control unit 304 determines whether or not a processing delay time is input (D_IN) from the processing delay calculation unit 103 (step S22). Subsequently, the control unit 304 compares the input processing delay time D_IN with the held processing delay time D to determine whether it is large or small (step S23).

When the input processing delay time D_IN is larger than the held processing delay time D (D_IN> D), that is, when the processing load becomes larger than the previous time (initial value is the first time), the control unit 304 starts decoding. time T _Q than the playback time T _S to specify the time at which advanced by (S + D_IN) (step S24). Further, in this case, a capacity obtained by reducing the capacity of data stored in the input buffer 101 by (D_IN−D) × Rate is designated (step S25). In this case, a capacity obtained by increasing the capacity of data stored in the output buffer 310 by (D_IN−D) × Org is designated (step S26). Here, Rate is the encoding rate of the received stream, and Org is the amount of information per unit time of uncompressed video and audio. Thus, the start time controller 112 of the reproducing means 102, advancing the decoding start time T _Q, also, the capacity control section 111 of the input buffer 101, reduces the volume of data stored in the buffer 101, the output buffer A capacity control unit 311 of 310 increases the capacity of data stored in the output buffer 310. Note that steps S24 to S26 may be processed in an arbitrary order or may be processed in parallel.
Following step S26, the control means 304 sets the value of the input processing delay time (D_IN) to D again and holds it, and sets the value of S + (D_IN-D) to S again and holds it. (Step S27), the process returns to Step S22.

In step S23, when the input processing delay time (D_IN) is smaller than the held processing delay time D (D_IN <D), that is, when the processing load is smaller than the previous time (initial value is the first time). Similarly to step S24, the control unit 304 designates a time that is earlier than the reproduction time T _S by (D_IN) the decoding start time T _Q (step S28). In this case, a capacity obtained by increasing the capacity of data stored in the input buffer 101 by (D−D_IN) × Rate is designated (step S29). Further, in this case, a capacity obtained by reducing the capacity of data stored in the output buffer 310 by (D−D_IN) × Org is designated (step S30). Thus, the start time controller 112 of the reproducing means 102, advancing the decoding start time T _Q, also, the capacity control section 111 of the input buffer 101 increases the amount of data stored in the buffer 101, the output buffer A capacity control unit 311 of 310 reduces the capacity of data stored in the output buffer 310. Steps S28 to S30 may be processed in an arbitrary order or may be processed in parallel.

Further, following step S30, the control unit 304 proceeds to step S27.
In step S23, when the input processing delay time (D_IN) is equal to the held processing delay time D (D_IN = D), that is, the processing load of the multimedia playback device 1 is the previous time (initial value is the first time). If the same, the control means 304 proceeds to step S27.

According to the multimedia playback apparatus of the first embodiment, the processing delay time by the playback means 102 can be constantly monitored, and the decoding start time can be advanced by the amount of the processing delay time and the playback allowance time. Further, the capacity accumulated in the input buffer 101 and the output buffer 310 can be controlled based on the processing delay time reflecting the processing load. As a result, even if the processing load fluctuates, it is possible to reproduce and output video and audio decoded or synthesized at a predetermined reproduction time while maintaining the necessary and sufficient capacity in the input buffer 101 and the output buffer 310.

(Second Embodiment)
[Configuration of multimedia playback device]
FIG. 7 is a block diagram of the multimedia playback apparatus according to the second embodiment.
As shown in FIG. 7, the multimedia playback device 4 includes a demultiplexing unit 50, a plurality (two in the figure) of media playback devices 400, and a load measuring unit 210. In addition, the same code | symbol is attached | subjected to the same structure as FIG. 3, and description is abbreviate | omitted.

The media playback device 400 has the same configuration as that of the media playback device 200 of the second reference embodiment shown in FIG. 3 except that the output buffer 310 is provided and the function of the control unit 404 is different. .
As will be described later, the control unit 404 stores the capacity of data stored in the input buffer 101 based on the processing load output from the load measuring unit 210 and the processing delay time indicating the time required for reproduction by the reproduction unit 102. And the capacity of data stored in the output buffer 310 and the decoding start time by the reproducing means 102 are designated.

[Operation of multimedia playback device]
Next, referring to FIG. 8 (refer to FIG. 7 as appropriate), the operation of the multimedia playback apparatus according to the second embodiment will be described. FIG. 8 is a flowchart of the control means of the multimedia playback apparatus according to the second embodiment of the present invention.

  First, the control unit 404 sets the initial value of the processing delay time to D, holds this value, sets the initial value of the load information to L, and holds it (step S31). In addition, the control unit 404 sets and retains the reproduction allowance time of the output buffer 310 as the initial value of the variable S. Then, the control unit 404 determines whether or not the current load information is input (L_IN) from the load measuring unit 210 (step S32). Subsequently, the control unit 404 compares the input load information L_IN with the stored load information L to determine the magnitude (step S33).

  When the input load information L_IN is larger than the held load information L (L_IN> L), that is, when the processing load is larger than the previous time (initial value is the first time), the control unit 404 holds the load information L_IN. Based on the processing delay time D and the load information L_IN, L, the adjustment time D ′ is calculated by the following equation (3) (step S34). Here, α ′ varies depending on the processing capacity of the CPU and the encoding rate of the stream, and is a constant determined in advance by simulation or the like.

  D ′ = D + α ′ × (L_IN−L) (3)

Then, the control unit 404 designates a time that is earlier than the reproduction time T _S by (S + D ′) the decoding start time T _Q (step S35). In this case, a capacity obtained by reducing the capacity of data stored in the input buffer 101 by β ′ × (L_IN−L) is designated (step S36). In this case, a capacity obtained by increasing the capacity of data stored in the output buffer 310 by γ ′ × (L_IN−L) is designated (step S37). Here, β ′ and γ ′ differ according to the processing capability of the CPU and the encoding rate of the stream, and are constants determined in advance by simulation or the like. Thus, the start time controller 112 of the reproducing means 102, advancing the decoding start time T _Q, also, the capacity control section 111 of the input buffer 101, reduces the volume of data stored in the buffer 101, the output buffer A capacity control unit 311 of 310 increases the capacity of data stored in the output buffer 310. Steps S35 to S37 may be processed in an arbitrary order or may be processed in parallel.

  Subsequent to step S37, the control unit 404 sets the value of the input load information (L_IN) to L again and holds it, sets the value of the adjustment time D ′ to D again, and holds it. The value of {S + γ ′ × (L_IN−L) / Org} is obtained, and this value is again set and held in the variable S (step S38), and the process returns to step S32. Note that Org is the amount of information per unit time of uncompressed video and audio.

  In step S33, if the input load information (L_IN) is smaller than the stored load information L (L_IN <L), that is, if the processing load is smaller than the previous time (initial value is the first time), control is performed. The means 404 calculates the adjustment time D ′ by the following equation (4) based on the stored processing delay time D and load information L_IN, L (step S39). Here, α ′ is the same as that in the formula (3).

  D ′ = D−α ′ × (L−L_IN) (4)

Then, as in step S35, the control means 404 designates a time that is earlier than the reproduction time T _S by (S + D ′) the decoding start time T _Q (step S40). In this case, a capacity obtained by increasing the capacity of data stored in the input buffer 101 by β ′ × (L−L_IN) is designated (step S41). In this case, a capacity obtained by reducing the capacity of data stored in the output buffer 310 by γ ′ × (L−L_IN) is designated (step S42). Here, β ′ and γ ′ differ according to the processing capability of the CPU and the encoding rate of the stream, and are constants determined in advance by simulation or the like. Thus, the start time controller 112 of the reproducing means 102, advancing the decoding start time T _Q, also, the capacity control section 111 of the input buffer 101 increases the amount of data stored in the buffer 101, the output buffer A capacity control unit 311 of 310 reduces the capacity of data stored in the output buffer 310. Steps S40 to S42 may be processed in an arbitrary order or may be processed in parallel. Following step S42, the control unit 404 proceeds to step S38.

  In step S33, when the input load information (L_IN) is equal to the stored load information L (L_IN = L), that is, when the processing load is the same as the previous time (initial value is the first time), the control unit 404 The process proceeds to step S38.

According to the multimedia playback device of the second embodiment, the decoding start time can be advanced by the time corresponding to the adjustment time D ′ based on the fluctuation of the processing load and the playback allowance time. Further, it is possible to control the capacity accumulated in the input buffer 101 and the output buffer 310 based on processing load fluctuations. As a result, even if the processing load fluctuates, it is possible to reproduce and output video and audio decoded or synthesized at a predetermined reproduction time while maintaining the necessary and sufficient capacity in the input buffer 101 and the output buffer 310.

As mentioned above, although embodiment of this invention was described, this invention is not limited to 1st and 2nd embodiment. For example, in the first and second embodiments, the multimedia playback device includes a plurality of the same media playback devices and can play back a plurality of media. However, a single media playback corresponding to a single media is possible. Only the device may be provided.
In the present invention, for example , the media playback devices in the first reference embodiment, the second reference embodiment, the first embodiment, and the second embodiment may be mixed. A multimedia playback apparatus in this case will be described as a third reference embodiment.

( Third reference embodiment)
[Configuration of multimedia playback device]
FIG. 9 is a block diagram of a multimedia playback apparatus according to the third reference embodiment.
As shown in FIG. 9, the multimedia playback device 5 includes a demultiplexing unit 50, media playback devices 100 to 400 (see FIGS. 1, 3, 5, and 7) and a load measuring unit 210. ing. In addition, the same code | symbol is attached | subjected to the structure same as the multimedia reproduction apparatuses 1-4, and description is abbreviate | omitted.

Here, MPEG-2 video, MPEG-1 Audio Layer, TVML script, and text script are assumed as media (streams and scripts) transmitted through the IP network.
Then, the media playback device 100 decodes the MPEG-2 stream and plays back the video. In this media playback apparatus 100, the processing delay time of 1 GOP of MPEG-2 is 10 ms, for example, and this value (10 ms) is set as the initial value of the processing delay time.
Further, the media playback device 200 plays back audio by decoding a stream encoded by MPEG-1 Audio Layer. In this media playback apparatus 200, the processing delay time of one audio frame of MPEG-1 Audio Layer is 3 ms, for example, and this value (3 ms) is set as the initial value of the processing delay time.

Further, the media playback device 300 interprets the TVML script and synthesizes video and audio. In this media playback device 300, the processing delay time required to interpret and synthesize a certain amount of TVML script is, for example, 900 ms, and this value (900 ms) is set as the initial value of the processing delay time.
Further, the media playback device 400 interprets the text script and synthesizes video and audio. In this media playback device 400, the processing delay time required for interpretation and synthesis of a certain amount of text script is 600 ms, for example, and this value (600 ms) is set as the initial value of the processing delay time.

According to the multimedia playback device of the third reference embodiment, each media such as video and audio reproduces video and audio at the time of the time stamp assigned to each media, thereby synchronizing the plurality of media. Playback output can be performed. In addition, which media is used as which encoding method is arbitrary, and which stream (or script) is reproduced (decoded or synthesized) by any media playback device 100 to 400 is arbitrary. Of course, the numerical value of the processing delay time is an example in a certain system.

It is a block diagram which shows the structure of the multimedia reproduction apparatus which concerns on 1st reference embodiment of this invention. It is a flowchart of the control means of the multimedia reproduction apparatus which concerns on 1st reference embodiment of this invention. It is a block diagram which shows the structure of the multimedia reproduction apparatus which concerns on 2nd reference embodiment of this invention. It is a flowchart of the control means of the multimedia reproduction apparatus which concerns on 2nd reference embodiment of this invention. It is a block diagram which shows the structure of the multimedia reproduction apparatus which concerns on the 1st Embodiment of this invention. It is a flowchart of the control means of the multimedia reproducing | regenerating apparatus which concerns on the 1st Embodiment of this invention. It is a block diagram which shows the structure of the multimedia reproduction apparatus which concerns on the 2nd Embodiment of this invention. It is a flowchart of the control means of the multimedia reproduction apparatus which concerns on the 2nd Embodiment of this invention. It is a block diagram which shows the structure of the multimedia reproduction apparatus which concerns on 3rd reference embodiment of this invention. It is a block diagram which shows the structure of the conventional multimedia reproducing | regenerating apparatus (when there is no output buffer). It is a block diagram which shows the structure of the conventional multimedia reproducing | regenerating apparatus (when there exists an output buffer). It is explanatory drawing of processing delay time, (a) shows the case where there is no output buffer, (b) shows the case where there is an output buffer.

Explanation of symbols

1-5 Multimedia playback apparatus 50 Demultiplexer (demultiplexer means)
DESCRIPTION OF SYMBOLS 100 Media playback apparatus 101 Input buffer 102 Playback means 103 Processing delay calculation means 104 Control means 111 Capacity control part (capacity control means)
112 Start time control unit (start time control means)
Reference Signs List 150 output device 200 media playback device 204 control means 210 load measurement means 300 media playback device 304 control means 310 output buffer 311 capacity control unit (capacity control means)
400 Media playback device 404 Control means

Claims (2)

An input buffer for storing encoded media data for playing back the media at a predetermined playback time; and playback means for playing back the data output from the input buffer, and playing back the media based on the data A media playback device,
An output buffer for accumulating data output from the reproducing means and holding the accumulated data for a predetermined time;
When detecting the input of data into the playback unit from the input buffer, and processing delay calculating means for calculating a measured processing delay time indicating the time required for reproduction of the data in said reproducing means,
Based on the current processing delay time calculated by the processing delay calculation means, in the next process, as well as specify the time to start the process for playing the media, respectively accumulated in the input buffer and the output buffer Control means for specifying the amount of data to be processed ;
The time specified by the pre-SL control means, and a start time controlling means for starting the processing for reproducing the media,
The input buffer includes capacity control means for controlling the capacity of accumulated data to a capacity designated by the control means,
The output buffer includes capacity control means for controlling the capacity of accumulated data to a capacity designated by the control means,
The control means includes
The time for starting the process for playing back the media is set to the current processing delay time calculated by the processing delay calculation means and the predetermined time for holding data in the output buffer, relative to the predetermined playback time. Only the sum is specified as an earlier time in the next process,
When the current processing delay time calculated by the processing delay calculation means is greater than a predetermined value, the next processing specifies to reduce the amount of data stored in the input buffer, and the output Specify to increase the amount of data stored in the buffer,
When the current processing delay time calculated by the processing delay calculation means is smaller than a predetermined value, the next processing specifies to increase the amount of data stored in the input buffer, and the output A media playback device characterized by specifying to reduce the capacity of data stored in a buffer . An input buffer for storing encoded media data for playing back the media at a predetermined playback time; and playback means for playing back the data output from the input buffer. A media playback device for playback,
An output buffer for accumulating data output from the reproducing means and holding the accumulated data for a predetermined time;
Load measuring means for measuring a processing load which is a processing load for reproducing the media;
Based on the fluctuation of the current processing load measured by the load measuring means, in the next processing, the time for starting the processing for reproducing the media is designated and stored in the input buffer and the output buffer, respectively. Control means for specifying the amount of data to be processed ;
The time specified by the pre-SL control means, and a start time controlling means for starting the processing for reproducing the media,
The input buffer includes capacity control means for controlling the capacity of accumulated data to a capacity designated by the control means,
The output buffer includes capacity control means for controlling the capacity of accumulated data to a capacity designated by the control means,
The control means includes
The processing delay time set this time as the time required for reproducing the data by the reproducing means is set to the time at which the processing for reproducing the media is started, compared to the predetermined reproduction time, and the measured current processing The sum of the adjustment time calculated by the load and a predetermined coefficient and the predetermined time that data is held in the output buffer is designated as the time that is advanced in the next processing,
When the current processing load measured by the load measuring means is larger than a predetermined value, in the next processing, it is specified to reduce the amount of data stored in the input buffer, and the output buffer Specify to increase the amount of accumulated data,
When the current processing load measured by the load measuring means is smaller than a predetermined value, the next processing specifies that the capacity of data stored in the input buffer is increased, and the output buffer A media playback device characterized in that it specifies to reduce the volume of stored data .

Images