WO2009090728A1

WO2009090728A1 - Media reproducer

Info

Publication number: WO2009090728A1
Application number: PCT/JP2008/050352
Authority: WO
Inventors: Yasuaki Takimoto; Masahiro Abukawa; Shiyuu Murayama; Shinji Akatsu; Kazuma Kaneko
Original assignee: Mitsubishi Electric Corporation
Priority date: 2008-01-15
Filing date: 2008-01-15
Publication date: 2009-07-23
Also published as: JPWO2009090728A1

Abstract

A media reproducer comprises an STC reproducing unit (2) for reproducing an STC, an STC correcting unit (3) for correcting the STC and generating a corrected STC, and a video decoder unit (4) and an audio decoder unit (5) for sequentially storing MPEG-TS signal data to be reproduced and reproducing the MPEG-TS signal data at a timing when PTSs of the corrected STC and the MPEG-TS signal data are synchronized. The STC correcting unit (3) corrects the STC according to the storage state of continuous media data and the PTS of the MPEG-TS signal data stored in the decoder units (4, 5).

Description

Media playback device

The present invention relates to a media playback apparatus such as an MPEG (Moving Picture Coding Experts Group) decoder.

For example, in a conventional media playback device such as an MPEG decoder disclosed in Patent Document 1, a PTS (Presentation Timp Stamp) is extracted from the header of an MPEG-TS (Transport Stream) signal, and an STC (System Time Clock) is obtained from the PTS value. ) Is subtracted. Thereafter, the conventional MPEG decoder compares the absolute value of the difference obtained by subtracting the STC value from the PTS value with a predetermined threshold value.

Here, when the absolute value of the difference obtained by subtracting the STC value from the PTS value is larger than a predetermined threshold, the conventional media playback device can repeat the same frame a predetermined number of times if the PTS value is larger than the STC value. Is repeatedly played back, and if the value of PTS is smaller than the value of STC, the same frame is skipped up to a predetermined number of times and played back. In this way, the conventional media reproducing apparatus executes the reproducing process so that the absolute value of the difference obtained by subtracting the STC value from the PTS value is within the predetermined threshold. This makes it possible to establish synchronization between the PTS value and the STC value and to decode the normal image sound (see, for example, Patent Document 1).

JP 2002-204404 A

In the conventional media playback device, when skipping or repeating within the predetermined number of times does not fall within the threshold, the playback is switched to asynchronous playback. For this reason, when the PTS value and the STC value are steadily far apart from each other, even if the relative relationship of PTS between media (video, audio, etc.) is normal, decoding is performed while maintaining synchronization between the media. There was a problem that it was not possible.

For example, when the PTS value and the STC value are far apart and the STC value has a constant offset with respect to the PTS of each medium, in normal playback, in the buffer for storing the media signal to be played back The media signal may cause overflow or underflow.

The present invention has been made to solve the above-described problems, and is time information that serves as a reference for synchronizing the value of time information (for example, PTS) indicating the presentation time of media data and the presentation time of the media data. It is an object of the present invention to obtain a media playback device that can play back with synchronization between media even when the value of (for example, STC) is constantly far apart.

A media playback device according to the present invention includes a reference time playback unit that plays back a reference time that serves as a reference for synchronizing the presentation time of continuous media data to be played back, and a correction unit that corrects the reference time to generate a corrected reference time. A continuous media data storage unit, and a media playback unit that plays back the continuous media data at a timing when the correction reference time and the presentation time of the continuous media data are synchronized. The reference time is corrected according to the storage status and the presentation time of the continuous media data stored in the media playback unit.

By having this configuration, even when the value of time information indicating the presentation time of media data and the value of time information serving as a reference for synchronization of the presentation time of the media data are constantly largely separated from each other, There is an effect that it is possible to reproduce while maintaining synchronization.

It is a block diagram which shows the structure of the media reproduction apparatus by Embodiment 1 of this invention. It is a flowchart which shows the flow of the STC correction process by the STC correction | amendment part in FIG. It is a block diagram which shows the structure of the media reproduction apparatus by Embodiment 2 of this invention. It is a flowchart which shows the flow of the STC correction process by the STC correction | amendment part in FIG.

Hereinafter, in order to describe the present invention in more detail, the best mode for carrying out the present invention will be described with reference to the accompanying drawings.
Embodiment 1 FIG.
FIG. 1 is a block diagram showing the configuration of a media playback apparatus according to Embodiment 1 of the present invention, and shows an MPEG decoder conforming to the MPEG-2 standard as an example. 1, the MPEG decoder according to Embodiment 1 includes a Demux unit 1, an STC playback unit 2, an STC correction unit 3, a video decoder unit 4, and an audio decoder unit 5. The Demux unit 1 is connected to an input signal line for inputting the MPEG-TS signal a from the outside, the STC reproduction unit 2, the audio decoder unit 4, and the video decoder unit 5. The MPEG-TS signal a is continuous media data in which stream data of each medium (video, audio) is multiplexed.

Also, the Demux unit 1 separates a PCR (Program Clock Reference) signal b, a video PES (Packetized Element Arty Stream) signal c, and an audio PES signal d from an MPEG-TS signal a input from the outside. The PCR signal b is a signal indicating a reference time when the MPEG-TS signal a is encoded, and is sent from the Demux unit 1 to the STC reproduction unit 2. The video PES signal c is a video encoded data signal to be reproduced, and is sent from the Demux unit 1 to the video decoder unit 4. The audio PES signal d is an audio encoded data signal to be reproduced, and is sent from the Demux unit 1 to the audio decoder unit 5.

The STC playback unit (reference time playback unit) 2 is connected to the Demux unit 1 and the STC correction unit 3. In addition, the STC reproduction unit 2 reproduces the STC signal e using the value of the PCR signal b input from the Demux unit 1. For example, a PLL (Phase Locked Loop) circuit is constructed as the STC reproducing unit 2, and the STC signal e is reproduced from the PCR signal b by this PLL circuit. The STC signal e is a signal indicating a reference time for synchronization with the media playback start time.

The STC correction unit (correction unit) 3 is connected to the STC playback unit 2, the video decoder unit 4, and the audio decoder unit 5, respectively. The STC correction unit 3 receives the STC signal e from the STC reproduction unit 2. Further, the STC correction unit 3 outputs the corrected STC signal f to the video decoder unit 4 and the audio decoder unit 5, while receiving the video storage status g from the video decoder unit 4 and the audio storage status h from the audio decoder unit 5. input.

The STC correction unit 3 calculates an offset by a determination process using the video accumulation state g, the audio accumulation state h, and the STC signal e. This determination process will be described later with reference to FIG. The STC correction unit 3 obtains a corrected STC signal f (corrected STC) by adding the offset value to the value of the STC signal e (STC) according to the following equation (1). The value of the corrected STC signal f becomes a reference time that serves as a reference for synchronizing the media playback start time used in the video decoder unit 4 and the audio decoder unit 5.
Correction STC = STC + Offset (1)

The video decoder unit (media playback unit) 4 is connected to the Demux unit 1 and the STC correction unit 3 and to a video output signal line for outputting the video signal i. In addition, the video decoder unit 4 includes a buffer that sequentially stores the video PES signal c input from the Demux unit 1. The video decoder unit 4 decodes each video frame in the video PES signal c, and outputs the video PES signal c to the buffer while the value of the PTS attached to each video frame is smaller than the value of the corrected STC signal f. accumulate. On the other hand, the video decoder unit 4 outputs the decoding result as the video signal i at the timing when the value of the PTS attached to each video frame matches the value of the corrected STC signal f. Note that the PTS value of each media (video, audio) stream is the playback start time of each media stream.

Also, the video decoder unit 4 generates a video storage status g composed of the storage amount of the video PES signal c stored in the buffer and the PTS value extracted from the first frame of the video PES signal c. This video accumulation state g is information used for STC correction processing by the STC correction unit 3 and is fed back from the video decoder unit 4 to the STC correction unit 3.

The audio decoder unit (media playback unit) 5 is connected to the Demux unit 1 and the STC correction unit 3 and to an audio output signal line for outputting an audio signal j. In addition, the audio decoder unit 5 includes a buffer that sequentially stores the audio PES signal d input from the Demux unit 1. The audio decoder unit 5 decodes each audio frame in the audio PES signal d, and outputs the audio PES signal d to the buffer while the value of the PTS attached to each audio frame is smaller than the value of the corrected STC signal f. accumulate. On the other hand, the audio decoder unit 5 outputs the decoding result as an audio signal j at a timing when the value of the PTS attached to each audio frame matches the value of the corrected STC signal f.

Also, the audio decoder unit 5 generates an audio accumulation state h including the accumulation amount of the audio PES signal d accumulated in the buffer and the PTS value extracted from the first frame of the audio PES signal d. The voice accumulation state h is information used for the STC correction process by the STC correction unit 3 and is fed back from the voice decoder unit 5 to the STC correction unit 3.

Next, the operation will be described.
FIG. 2 is a flowchart showing the flow of STC correction processing by the STC correction unit in FIG. 1, and the correction processing of the STC signal e will be described in detail with reference to this drawing.
First, the STC correction unit 3 initializes an offset value used in the STC correction process according to the above equation (1) (step ST1). Here, Offset = 0 is set.

Subsequently, the STC correction unit 3 determines whether or not there is an external reproduction end request, and if there is a request, ends the STC correction process (step ST1-1). If there is no request, the STC correction unit 3 determines whether or not the accumulation amount of the video PES signal c input from the video decoder unit 4 as the video accumulation state g is equal to or less than a predetermined threshold A1, and as the audio accumulation state h. It is determined whether or not the accumulated amount of the audio PES signal d input from the audio decoder unit 5 is equal to or less than a predetermined threshold A2. The STC correction unit 3 determines whether each buffer of the

decoder units

4 and 5 is in a near overflow state according to the logical sum of these determination results (step ST2).

The threshold values A1 and A2 are obtained from the following formula (2) and the following formula (3), for example. However, the storage capacity of the video PES is the storage capacity of the video PES signal c of the buffer in the video decoder unit 4. The audio PES accumulative capacity is an accumulative capacity of the audio PES signal d in the buffer in the audio decoder unit 5.
A1 = capacity of video PES storage × 0.1 (2)
A2 = Accumulable capacity of voice PES × 0.1 (3)

In step ST2, if the logical sum of the determination result for the video PES signal c and the determination result for the audio PES signal d is false (indicated by NO in FIG. 2), the STC correction unit 3 performs step ST3. Move on to processing. Here, the logical sum is false when the accumulation amount of the video PES signal c exceeds the threshold value A1 and the accumulation amount of the audio PES signal d exceeds the threshold value A2. At this time, it is determined that both buffers of the

decoder units

4 and 5 are not in the near overflow state.

In step ST2, when the logical sum of the determination result for the video PES signal c and the determination result for the audio PES signal d is true (indicated by YES in FIG. 2), the STC correction unit 3 The process proceeds to step ST6. Here, the logical sum is true when the accumulated amount of one of the video PES signal c and the audio PES signal d is equal to or smaller than the threshold value, or the accumulated amount of the video PES signal c is equal to or smaller than the threshold value A1. And the accumulated amount of the audio PES signal d is equal to or less than the threshold value A2. At this time, it is determined that at least one of the buffers of the

decoder units

4 and 5 is in a near overflow state.

In step ST3, the STC correction unit 3 determines whether or not the PTS value (PTS value extracted from the first frame of the video PES signal c) input from the video decoder unit 4 as the video accumulation status g is equal to or less than the value of the corrected STC signal f. Determine whether. Further, the STC correction unit 3 determines whether or not the PTS value (the PTS value extracted from the first frame of the audio PES signal d) input from the audio decoder unit 5 as the audio accumulation state h is equal to or less than the value of the correction STC signal f. judge. The STC correction unit 3 determines whether or not the reproduction start condition is satisfied according to the logical sum of these determination results. Note that the reproduction start condition refers to a case where the PTS values for the video PES signal c and the audio PES signal d match the value of the corrected STC signal f.

Here, if the logical sum of the determination result for the video PES signal c and the determination result for the audio PES signal d is false (indicated by NO in FIG. 2), the STC correction unit 3 performs step ST2. Return to processing. The logical sum is false because the PTS value for the video PES signal c exceeds the value of the corrected STC signal f, and the PTS value for the audio PES signal d exceeds the value of the corrected STC signal f. It is a case. At this time, in both the video PES signal c and the audio PES signal d, it is determined that the time indicated by the value of the corrected STC signal f is ahead of the time indicated by the PTS value (STC advance state). The

On the other hand, if the logical sum of the determination result for the video PES signal c and the determination result for the audio PES signal d is true (indicated by YES in FIG. 2), the STC correction unit 3 performs the process of step ST4. Migrate to Here, the logical sum is true when the PTS value of one of the video PES signal c and the audio PES signal d is equal to or smaller than the value of the corrected STC signal f, or the PTS for the video PES signal c. This is a case where the value is equal to or smaller than the value of the corrected STC signal f and the PTS value for the audio PES signal d is equal to or smaller than the value of the corrected STC signal f. At this time, at least one of the video PES signal c and the audio PES signal d is determined to be in a state in which the time indicated by the value of the corrected STC signal f is delayed from the time indicated by the PTS value (STC delay state). Is done.

The corrected STC signal f obtained by this STC correction process is output from the STC correction unit 3 to the video decoder unit 4 and the audio decoder unit 5.

When the logical sum of the determination results of the video PES signal c and the audio PES signal d is true in step ST3, the STC correction unit 3 has an accumulation amount of the video PES signal c in the video accumulation state g that is equal to or greater than a predetermined threshold B1. It is determined whether or not there is, and it is determined whether or not the accumulation amount of the audio PES signal d in the audio accumulation state h is equal to or greater than a predetermined threshold B2. The STC correction unit 3 determines whether or not each buffer of the

decoder units

4 and 5 is in the near underflow state according to the logical sum value of these determination results (step ST4).

The threshold values B1 and B2 are obtained from the following formula (4) and the following formula (5), for example.
B1 = Capacity of accumulation of video PES × 0.9 (4)
B2 = Accumulable capacity of voice PES × 0.9 (5)

In step ST4, when the logical sum of the determination result for the video PES signal c and the determination result for the audio PES signal d is false (indicated by NO in FIG. 2), the STC correction unit 3 performs step ST6. The process proceeds to -1. Here, the logical sum is false when the accumulation amount of the video PES signal c is less than the threshold value B1 and the accumulation amount of the audio PES signal d is less than the threshold value B2. At this time, it is determined that both buffers of the

decoder units

4 and 5 are not in the near underflow state.

In step ST4, when the logical sum of the determination result for the video PES signal c and the determination result for the audio PES signal d is true (indicated by YES in FIG. 2), the STC correction unit 3 The process proceeds to step ST5. Here, the logical sum is true when the accumulated amount of one of the video PES signal c and the audio PES signal d is equal to or greater than the threshold value, or the accumulated amount of the video PES signal c is equal to or greater than the threshold value B1. And the accumulated amount of the audio PES signal d is greater than or equal to the threshold B2. At this time, it is determined that at least one buffer of the

decoder units

4 and 5 is in the near underflow state.

In step ST5, the STC correction unit 3 performs an offset calculation process in the near underflow state of the buffer. Specifically, the STC correction unit 3 performs an operation according to the following formula (6) and the following formula (7) with respect to the smallest PTS value among the PTS value in the video accumulation situation g and the PTS value in the audio accumulation situation h. To calculate the offset (Offset). By adding this offset to the value of the STC signal e, a condition for delaying the start of reproduction in the

decoder units

4 and 5 can be obtained. When the STC correction unit 3 obtains the offset in the near underflow state of the buffer, the STC correction unit 3 executes the processing from step ST2 with this offset value. The assumed video PES input rate is an assumed input rate of the video PES signal c input to the video decoder unit 4, and is assumed from a transmission rate or the like according to the specification of the video PES signal c. Further, the assumed audio PES input rate is an input rate of an assumed audio PES signal d input to the audio decoder unit 5, and is assumed from a transmission rate or the like in the specification of the audio PES signal d.
Offset = PTS−STC−α (6)
α = (A1 / assumed video PES input rate + A2 / assumed audio PES input rate) / 2
... (7)

Further, in the offset calculation described above, an input rate measuring unit (not shown) for measuring the input rate of each medium (video, audio) is provided in the configuration of FIG. May be. The measured video PES input rate is the input rate of the video PES signal c from the Demux unit 1 to the video decoder unit 4 measured by the input rate measuring unit. The measured audio PES input rate is the input rate of the audio PES signal d from the Demux unit 1 to the audio decoder unit 5 measured by the input rate measuring unit.
α = (A1 / measurement video PES input rate + A2 / measurement audio PES input rate) / 2
... (8)

In step ST6, the STC correction unit 3 performs an offset calculation process in the buffer near overflow state. Specifically, the STC correction unit 3 performs an operation according to the following equation (9) on the largest PTS value among the PTS value in the video accumulation situation g and the PTS value in the audio accumulation situation h, thereby offset ( Offset) is calculated. By adding this offset to the value of the STC signal e, it is possible to obtain a condition for starting reproduction at the decoder units 4 and 5 (condition for immediately starting reproduction). When the STC correction unit 3 determines the offset in the near overflow state of the buffer, the STC correction unit 3 proceeds to the process of step ST6-1.
Offset = PTS−STC (9)

In step ST6-1, the STC correction unit 3 determines whether or not there is an external reproduction end request. If there is a request, the STC correction process ends. If there is no request, the STC correction unit 3 proceeds to step ST7 and determines whether or not the storage amount of the video PES signal c in the video storage state g is the storable capacity of the buffer of the video decoder unit 4, It is determined whether or not the accumulation amount of the audio PES signal d in the audio accumulation state h is the accumulation capacity of the buffer of the audio decoder unit 5. The STC correction unit 3 determines whether each buffer of the

decoder units

4 and 5 is in an overflow state according to the logical sum value of these determination results.

Here, if the logical sum of the determination result for the video PES signal c and the determination result for the audio PES signal d is false (indicated by NO in FIG. 2), the STC correction unit 3 performs step ST8. Transition to processing. Here, the logical sum is false when the accumulation amount of the video PES signal c does not reach the storable capacity and the accumulation amount of the audio PES signal d does not reach the storable capacity. is there. At this time, it is determined that both buffers of the

decoder units

4 and 5 are not in an overflow state.

Further, in step ST7, the STC correction unit 3 determines that the logical sum of the determination result for the video PES signal c and the determination result for the audio PES signal d is true (indicated by YES in FIG. 2). The process proceeds to step ST9. Here, the logical sum is true when the accumulation amount of one of the video PES signal c and the audio PES signal d is the accumulation capacity or the accumulation amount of the video PES signal c is the accumulation. This is a case where the storage capacity is the capacity and the storage amount of the audio PES signal d is the storage capacity. At this time, it is determined that at least one buffer of the

decoder units

4 and 5 is in an overflow state.

If the STC correction unit 3 determines that both buffers of the

decoder units

4 and 5 are not in an overflow state, the STC correction unit 3 determines whether or not both buffers of the

decoder units

4 and 5 are in an underflow state (step ST8). Specifically, the STC correction unit 3 determines whether or not the accumulation amount of the video PES signal c in the video accumulation situation g is 0, and the accumulation amount of the audio PES signal d in the audio accumulation situation h is 0. It is determined whether or not each buffer of the

decoder units

4 and 5 is in an underflow state according to the logical sum value of these determination results.

Here, if the logical sum of the determination result for the video PES signal c and the determination result for the audio PES signal d is false (indicated by NO in FIG. 2), the STC correction unit 3 performs step ST6- Return to the process of 1. Here, the logical sum is false when the accumulation amount of the video PES signal c is not zero and the accumulation amount of the audio PES signal d is not zero. At this time, it is determined that both buffers of the

decoder units

4 and 5 are not in the underflow state.

If the logical sum of the determination result for the video PES signal c and the determination result for the audio PES signal d is true (indicated by YES in FIG. 2) in step ST8, the STC correction unit 3 The process returns to step ST2. Here, the logical sum is true when the accumulation amount of one of the video PES signal c and the audio PES signal d is 0, or the accumulation amount of the video PES signal c is 0, and This is a case where the accumulated amount of the audio PES signal d is zero. At this time, it is determined that at least one buffer of the

decoder units

4 and 5 is in an underflow state.

On the other hand, when the STC correction unit 3 determines in step ST7 that the buffer is in the overflow state, the STC correction unit 3 performs an offset calculation process in the overflow state (step ST9). Specifically, the STC correction unit 3 performs an operation according to the following formula (10) and the following formula (11) with respect to the largest PTS value among the PTS value in the video accumulation situation g and the PTS value in the audio accumulation situation h. To calculate the offset (Offset). By adding this offset to the value of the STC signal e, the

decoder units

4 and 5 can make a condition for skipping some frames in the media stream. When the STC correction unit 3 obtains the offset in the buffer overflow state, the STC correction unit 3 executes the processing from step ST7 with this offset value.
Offset = PTS−STC + β (10)
β = (B1 / assumed video PES input rate + B2 / assumed audio PES input rate) / 2
(11)

Further, in the above-described offset calculation, β is obtained by the following equation (12) by providing an input rate measuring unit (not shown) for measuring the input rate of each medium (video, audio) in the configuration of FIG. May be.
α = (B1 / measurement video PES input rate + B2 / measurement audio PES input rate) / 2
(12)

The STC correction unit 3 repeats the processing from step ST2 to step ST9 until the reproduction start condition is satisfied. The corrected STC signal f obtained in this way is output from the STC correction unit 3 to the video decoder unit 4 and the audio decoder unit 5. The video decoder unit 4 and the audio decoder unit 5 execute a reproduction process using the value of the corrected STC signal f, and output a video signal i and an audio signal j.

As described above, according to the first embodiment, the STC reproduction unit 2 that reproduces the STC signal e serving as a reference for the synchronization of the PTS of the MPEG-TS signal data to be reproduced, and the STC signal e are corrected and corrected. The STC correction unit 3 for generating the STC signal f and the MPEG-TS signal data are sequentially accumulated, and the MPEG-TS signal data is reproduced at the timing when the value of the corrected STC signal f and the PTS value of the MPEG-TS signal data are synchronized. The STC correction unit 3 stores the continuous media data storage status in the video decoder unit 4 and the audio decoder unit 5 and the video decoder unit 4 and the audio decoder unit 5. The STC signal e is corrected according to the value of the PTS of the MPEG-TS signal data being processed.
By doing so, even if the PTS value and the STC value are far apart and the STC value has a constant offset with respect to the PTS of each medium, synchronization between the media is performed. The MPEG-TS signal data can be reproduced while maintaining it. In addition, the STC signal is corrected so that the buffers of the

decoder units

4 and 5 do not enter the buffer over state or the buffer under state in consideration of the accumulation state of the MPEG-TS signal data. The occurrence of overflow and buffer underflow can be suppressed.

Embodiment 2. FIG.
FIG. 3 is a block diagram showing the configuration of a media playback apparatus according to Embodiment 2 of the present invention, and shows an MPEG decoder conforming to the MPEG-2 standard as an example. 3, the MPEG decoder according to the second embodiment includes a demux unit 1, an STC playback unit 2, an STC correction unit 3a, a video decoder unit 4, an audio decoder unit 5, and a speed conversion unit 6. The Demux unit 1, the STC playback unit 2, the video decoder unit 4 and the audio decoder unit 5 have the same configuration as that of FIG. 1 described in the first embodiment.

The STC correction unit (correction unit) 3a is connected to the STC reproduction unit 2, the video decoder unit 4, the audio decoder unit 5 and the speed conversion unit 6, respectively. The STC correction unit 3a receives the STC signal e from the STC reproduction unit 2. The STC correction unit 3 a outputs the corrected STC signal f to the video decoder unit 4 and the audio decoder unit 5, while receiving the video storage status g from the video decoder unit 4 and the audio storage status h from the audio decoder unit 5. input. Further, the STC correction unit 3 a outputs a speed instruction signal k to the speed conversion unit 6.

The STC correction unit 3a calculates an offset by a determination process using the video accumulation state g, the audio accumulation state h, and the STC signal e. This determination process will be described later with reference to FIG. The STC correction unit 3a obtains the corrected STC signal f (corrected STC) by adding the offset value to the value of the STC signal e (STC) according to the equation (1).

In the offset calculation process described above, when the STC correction unit 3a needs to increase the offset within a predetermined threshold, the STC correction unit 3a sends a speed instruction signal k that increases the presentation speed of the video signal i and the audio signal j to the speed conversion unit 6. Outputs and gradually increases the offset value. On the other hand, when it is necessary to reduce the offset within the above threshold, the STC correction unit 3a outputs a speed instruction signal k for reducing the presentation speed of the video signal i and the audio signal j to the speed conversion unit 6, and Decrease the value gradually.

The speed conversion unit 6 is connected to the STC correction unit 3a via a signal line for inputting a speed instruction signal k, and connected to the video decoder unit 4 via a video output signal line for outputting a video signal i. And connected to the audio decoder unit 5 through an audio output signal line for outputting the audio signal j. The speed converter 6 is connected to an output signal line for outputting the speed converted video signal l, and is connected to an output signal line for outputting the speed converted audio signal m.

The speed converter 6 calculates a speed-converted video signal l obtained by converting the presentation speed of the video signal i into a value designated by the speed instruction signal k, and the presentation speed of the audio signal j is instructed by the speed instruction signal k. The speed-converted audio signal m converted to the calculated value is calculated. As the presentation speed conversion of the video signal i, for example, a method of repeating or skipping a video frame or a method of generating and outputting an intermediate image of the video frame can be considered. As for the presentation speed conversion of the audio signal j, for example, there are a method of converting the sampling frequency of the audio signal j and a method of inserting and deleting in units of periods after detecting the period of the audio signal j.

Next, the operation will be described.
FIG. 4 is a flowchart showing the flow of STC correction processing by the STC correction unit in FIG. 3, and the correction processing of the STC signal e will be described in detail with reference to FIG.
First, the STC correction unit 3a initializes the value of the offset (Offset) used in the STC correction process according to the above equation (1) and the value of the presentation speed V indicated by the speed instruction signal k (step ST1a). Here, Offset = 0 and presentation speed V = 0 are set.

Subsequently, the STC correction unit 3a determines whether or not there is an external reproduction end request, and if there is a request, ends the STC correction process (step ST1a-1). If there is no request, the STC correction unit 3a determines whether or not the accumulation amount of the video PES signal c input from the video decoder unit 4 as the video accumulation status g is equal to or less than a predetermined threshold A1, and as the audio accumulation status h. It is determined whether or not the accumulated amount of the audio PES signal d input from the audio decoder unit 5 is equal to or less than a predetermined threshold A2. The STC correction unit 3a determines whether or not each buffer of the

decoder units

4 and 5 is in a near overflow state according to the logical sum value of these determination results (step ST2a). The threshold values A1 and A2 are obtained from the above formula (2) and the above formula (3).

If the logical sum of the determination result for the video PES signal c and the determination result for the audio PES signal d is false (indicated by NO in FIG. 4) in step ST2a, the STC correction unit 3a Move on to processing. If the logical sum of the determination result for the video PES signal c and the determination result for the audio PES signal d is true (indicated by YES in FIG. 4), the STC correction unit 3a performs the process of step ST6a. Migrate to

In step ST3a, the STC correction unit 3a determines whether the PTS value (PTS value extracted from the first frame of the video PES signal c) input from the video decoder unit 4 as the video accumulation status g is equal to or less than the value of the corrected STC signal f. Determine whether. In addition, the STC correction unit 3a determines whether or not the PTS value (PTS value extracted from the first frame of the audio PES signal d) input from the audio decoder unit 5 as the audio accumulation state h is equal to or less than the value of the correction STC signal f. judge. The STC correction unit 3a determines whether or not the reproduction start condition is satisfied according to the logical sum of these determination results. Note that the reproduction start condition refers to a case where the PTS values for the video PES signal c and the audio PES signal d match the value of the corrected STC signal f.

Here, if the logical sum of the determination result for the video PES signal c and the determination result for the audio PES signal d is false (indicated by NO in FIG. 4), the STC correction unit 3a performs step ST2a. Return to processing. At this time, in both the video PES signal c and the audio PES signal d, it is determined that the time indicated by the value of the corrected STC signal f is ahead of the time indicated by the PTS value (STC advance state). The

On the other hand, when the logical sum of the determination result for the video PES signal c and the determination result for the audio PES signal d is true (indicated by YES in FIG. 4), the STC correction unit 3a performs the process of step ST4a. Migrate to At this time, at least one of the video PES signal c and the audio PES signal d is determined to be in a state in which the time indicated by the value of the corrected STC signal f is delayed from the time indicated by the PTS value (STC delay state). Is done.

The corrected STC signal f obtained by this STC correction process is output from the STC correction unit 3a to the video decoder unit 4 and the audio decoder unit 5.

When the logical sum of the determination results of the video PES signal c and the audio PES signal d is true in step ST3a, the STC correction unit 3a has an accumulation amount of the video PES signal c in the video accumulation state g that is equal to or greater than a predetermined threshold B1. It is determined whether or not there is, and it is determined whether or not the accumulation amount of the audio PES signal d in the audio accumulation state h is equal to or greater than a predetermined threshold B2. The STC correction unit 3a determines whether or not each buffer of the

decoder units

4 and 5 is in the near underflow state according to the logical sum value of these determination results (step ST4a). The threshold values B1 and B2 are obtained from the above formula (4) and the above formula (5).

In step ST4a, the STC correction unit 3a determines that the logical sum of the determination result for the video PES signal c and the determination result for the audio PES signal d is false (indicated by NO in FIG. 4). The process proceeds to -1. At this time, it is determined that both buffers of the

decoder units

4 and 5 are not in the near underflow state.

In step ST4a, if the logical sum of the determination result for the video PES signal c and the determination result for the audio PES signal d is true (indicated by YES in FIG. 4), the STC correction unit 3a The process proceeds to step ST5a. At this time, it is determined that at least one buffer of the

decoder units

4 and 5 is in the near underflow state.

In step ST5a, the STC correction unit 3a performs an offset calculation process in the near underflow state of the buffer. Specifically, the STC correction unit 3a performs the above expression (6) and the above expression (7) (or the PTS value in the video accumulation situation g and the PTS value in the audio accumulation situation h with respect to the smallest PTS value. An offset is calculated by performing an operation according to the above equation (8). When the STC correction unit 3a obtains the offset in the near underflow state of the buffer, the STC correction unit 3a executes the processing from step ST2a with this offset value.

In step ST6a, the STC correction unit 3a performs an offset calculation process in the buffer near overflow state. Specifically, the STC correction unit 3a performs an operation according to the above formula (9) on the largest PTS value among the PTS value in the video accumulation situation g and the PTS value in the audio accumulation situation h, thereby offset ( Offset) is calculated. When the STC correction unit 3a obtains the offset in the near overflow state of the buffer, the STC correction unit 3a proceeds to the process of step ST7a.

In step ST6a-1, the STC correction unit 3a determines whether or not there is an external reproduction end request, and if there is a request, ends the STC correction process. If there is no request, the STC correction unit 3a moves to step ST7a and performs a process of adding the product of the presentation speed V and the elapsed time Δt to the offset with respect to the elapsed time Δt from the previous process. Thereafter, the STC correction unit 3a determines whether or not the accumulation amount of the video PES signal c in the video accumulation situation g is the storable capacity of the buffer of the video decoder unit 4, and the audio PES signal d in the audio accumulation situation h. It is determined whether or not the storage amount is the bufferable storage capacity of the audio decoder unit 5. The STC correction unit 3a determines whether or not each buffer of the

decoder units

4 and 5 is in a buffer overflow state according to the logical sum value of these determination results (step ST8a).

Here, if the logical sum of the determination result for the video PES signal c and the determination result for the audio PES signal d is false (indicated by NO in FIG. 4), the STC correction unit 3a performs step ST9a. Transition to processing. At this time, it is determined that both buffers of the

decoder units

4 and 5 are not in an overflow state. On the other hand, if the logical sum of the determination result for the video PES signal c and the determination result for the audio PES signal d is true (indicated by YES in FIG. 4), the STC correction unit 3a performs the process of step ST16a. Migrate to At this time, it is determined that at least one buffer of the

decoder units

4 and 5 is in an overflow state.

If the STC correction unit 3a determines that both buffers of the

decoder units

4 and 5 are not in the buffer overflow state, the STC correction unit 3a determines whether or not both buffers of the

decoder units

4 and 5 are in the buffer under-state (step ST9a). Specifically, the STC correction unit 3a determines whether or not the accumulation amount of the video PES signal c in the video accumulation situation g is 0, and the accumulation amount of the audio PES signal d in the audio accumulation situation h is 0. It is determined whether or not each buffer of the

decoder units

Here, if the logical sum of the determination result for the video PES signal c and the determination result for the audio PES signal d is false (indicated by NO in FIG. 4), the STC correction unit 3a performs step ST11a. Transition to processing. At this time, it is determined that both buffers of the

decoder units

4 and 5 are not in the underflow state.

In step ST9a, if the logical sum of the determination result for the video PES signal c and the determination result for the audio PES signal d is true (indicated by YES in FIG. 4), the STC correction unit 3a The process proceeds to step ST10a. At this time, it is determined that at least one buffer of the

decoder units

4 and 5 is in an underflow state.

In step ST10a, the STC correction unit 3a initializes the value of the presentation speed V indicated by the speed instruction signal k. Here, the presentation speed V = 0 is set. Thereafter, the STC correction unit 3a returns to step ST2a and repeats the above-described processing from step ST2a.

In step ST11a, the STC correction unit 3a determines whether or not the accumulation amount of the video PES signal c input from the video decoder unit 4 as the video accumulation status g is equal to or less than a predetermined threshold A1, and the audio accumulation status h is audio. It is determined whether or not the accumulated amount of the audio PES signal d input from the decoder unit 5 is equal to or less than a predetermined threshold A2. The STC correction unit 3a determines whether or not each buffer of the

decoder units

4 and 5 is in a near overflow state according to the logical sum value of these determination results.

Here, if the logical sum of the determination result for the video PES signal c and the determination result for the audio PES signal d is false (indicated by NO in FIG. 4), the STC correction unit 3a performs step ST12a. Transition to processing. If the logical sum of the determination result for the video PES signal c and the determination result for the audio PES signal d is true (indicated by YES in FIG. 4), the STC correction unit 3a performs the process of step ST14a. Migrate to

In step ST12a, the STC correction unit 3a determines that the logical sum of the determination result for the video PES signal c and the determination result for the audio PES signal d is false (indicated by NO in FIG. 4). Move on to processing. At this time, it is determined that both buffers of the

decoder units

4 and 5 are not in the near-under state. On the other hand, when the logical sum of the determination result for the video PES signal c and the determination result for the audio PES signal d is true (indicated by YES in FIG. 4), the STC correction unit 3a performs the process of step ST15a. Migrate to At this time, it is determined that at least one buffer of the

decoder units

4 and 5 is in the near underflow state.

The STC correction unit 3a executes processing for bringing the presentation speed close to 0 in step ST13a. Specifically, if the presentation speed V is a positive value, the STC correction unit 3a subtracts a predetermined value γ from the presentation speed V, and returns to 0 when the presentation speed V falls below zero. On the other hand, if the presentation speed V is a negative value, the predetermined value γ is added to the presentation speed V, and when the presentation speed V exceeds 0, the process returns to 0. The predetermined value γ may be a positive value (γ> 0). Also, a different value may be set as the predetermined value γ in each process of step ST13a, step ST14a described later, and step ST15a. Thus, after the value of the presentation speed instruct | indicated with the speed instruction | indication signal k approaches 0, the STC correction | amendment part 3a returns to the process of step ST7a.

If it is determined in step ST11a that the buffers of the

decoder units

4 and 5 are in the near overflow state, the STC correction unit 3a adds the predetermined value γ to the value of the presentation speed indicated by the speed instruction signal k (step ST14a). Thereafter, the STC correction unit 3a returns to step ST7a and repeats the above-described processing from step ST7a.

If it is determined in step ST12a that the buffers of the

decoder units

4 and 5 are in the near underflow state, the STC correction unit 3a subtracts the predetermined value γ from the value of the presentation speed indicated by the speed instruction signal k ( Step ST15a). Thereafter, the STC correction unit 3a returns to step ST7a and repeats the above-described processing from step ST7a.

On the other hand, in step ST16a, the STC correction unit 3a follows the above formula (10) and the above formula (11) with respect to the largest PTS value among the PTS value in the video accumulation situation g and the PTS value in the audio accumulation situation h. By performing the calculation, an offset is calculated. In addition, the STC correction unit 3a initializes the value of the presentation speed V indicated by the speed instruction signal k. Here, the presentation speed V = 0 is set. Thereafter, the STC correction unit 3a returns to step ST7a and repeats the above-described processing from step ST7a.

The STC correction unit 3a repeats the processing from step ST2a to step ST16a until the reproduction start condition is satisfied. The corrected STC signal f obtained in this way is output from the STC correction unit 3a to the video decoder unit 4 and the audio decoder unit 5. The video decoder unit 4 and the audio decoder unit 5 execute a reproduction process using the value of the corrected STC signal f, and output a video signal i and an audio signal j.

The STC correction unit 3a instructs the speed instruction signal k as described above when the buffer is in a near overflow state or a near underflow state during media reproduction by the video decoder unit 4 and the audio decoder unit 5. The presentation speed is adjusted, and the offset value is sequentially corrected according to the presentation speed. As a result, the speed converter 6 that has received the speed instruction signal k converts the video signal i and the audio signal j to the presentation speed instructed by the speed instruction signal k.

As described above, according to the second embodiment, the speed conversion unit 6 that converts the presentation speed of the media data reproduced by the video decoder unit 4 and the audio decoder unit 5 into a value corresponding to the instruction is provided, and STC correction is performed. Since the unit 3a instructs the speed conversion unit 6 on the value of the presentation speed adjusted according to the MPEG-TS signal data accumulation status in the video decoder unit 4 and the audio decoder unit 5, the image sound is interrupted during the offset adjustment. The occurrence of buffer overflow and underflow can be suppressed.

In the first and second embodiments, the MPEG decoder has been described as an example. However, the present invention is not limited to MPEG encoded data. For example, the present invention can be applied to an apparatus that handles AVI data as a reproduction target.

As described above, the media playback apparatus according to the present invention is suitable for use in an MPEG decoder such as a digital television receiver because it can stably perform media playback while maintaining synchronization between media.

Claims

A reference time playback unit that plays back a reference time that serves as a reference for synchronizing the presentation time of continuous media data to be played back;
A correction unit that corrects the reference time to generate a corrected reference time;
A media playback unit that sequentially stores the continuous media data, and plays back the continuous media data at a timing at which the correction reference time and the presentation time of the continuous media data are synchronized;
The media playback device that corrects the reference time according to a storage status of continuous media data in the media playback unit and a presentation time of the continuous media data stored in the media playback unit.
The correction unit
Determining the accumulation status of the continuous media data based on a comparison result between the accumulation amount of the continuous media data and a predetermined threshold value corresponding to the accumulation capacity of the media playback unit;
Based on the comparison result of the value of the presentation time and the value of the reference time, the deviation state of the presentation time is determined,
An offset value to be added to the reference time is calculated based on the accumulation state of the continuous media data of the determination result and the deviation state of the presentation time. The media playback apparatus according to claim 1, wherein the media playback apparatus outputs to a playback unit.
Continuous media data is data encoded in the MPEG format,
The correction unit uses an accumulated amount of the continuous media data and a PTS (Presentation Timp Stamp) added to the first frame of the accumulated continuous media data as a presentation time, and an STC (System Time Clock) value indicating a reference time The media reproducing apparatus according to claim 2, wherein:
The correction unit
Based on the comparison result between the storage amount of continuous media data and a predetermined threshold value corresponding to the storable capacity of the media playback unit, the continuous media data indicates the near underflow state, the near overflow state, and the normal storage state of the continuous media data. Judge as the data accumulation status,
A state in which the reference time is advanced from the presentation time, a state in which the reference time is delayed from the presentation time, and the reference time and the presentation time from the magnitude relationship between the value of the presentation time and the value of the reference time Is determined to match,
When the determination result that the continuous media data is in a near underflow state, the reference time and the presentation time match or the reference time is ahead of the presentation time is obtained, Calculate the offset value so that the playback start of the continuous media data is delayed,
When the determination result that the continuous media data is in the near overflow state and the reference time is behind the presentation time is obtained, the offset is set so that the reproduction start of the continuous media data becomes the current time. 3. The media reproducing apparatus according to claim 2, wherein a value is calculated.
Continuous media data is data encoded in the MPEG format,
The correction unit uses an accumulated amount of the continuous media data and a PTS (Presentation Timp Stamp) added to the first frame of the accumulated continuous media data as a presentation time, and an STC (System Time Clock) value indicating a reference time The media reproducing apparatus according to claim 4, wherein the medium is corrected.
The correction unit
Based on the comparison result between the storage amount of continuous media data and a predetermined threshold value corresponding to the storable capacity of the media playback unit, the continuous media data indicates the near underflow state, the near overflow state, and the normal storage state of the continuous media data. Judge as the data accumulation status,
A state in which the reference time is advanced from the presentation time, a state in which the reference time is delayed from the presentation time, and the reference time and the presentation time from the magnitude relationship between the value of the presentation time and the value of the reference time Is determined to match,
When the determination result that the continuous media data is in a near underflow state, the reference time and the presentation time match or the reference time is ahead of the presentation time is obtained, Calculate the offset value so that the playback start of the continuous media data is delayed,
When the determination result that the continuous media data is in the near overflow state and the reference time is behind the presentation time is obtained, the offset is set so that the reproduction start of the continuous media data becomes the current time. A first operating state for calculating a value;
Based on a comparison result between a storage amount of the continuous media data and a predetermined threshold according to a storage capacity of the media playback unit, an underflow state and an overflow state are determined as the storage state of the continuous media data,
When a determination result that the continuous media data is in an overflow state is obtained, a second operation state in which the offset value is increased so as to skip the playback of the continuous media data by the media playback unit,
In the first operation state, when the continuous media data is not in either a near underflow state or the continuous media data is in a near overflow state, and the reference time and the presentation time coincide with each other, Transition to the operating state 2
3. The media reproducing apparatus according to claim 2, wherein, in the second operation state, when a determination result that the continuous media data is in an underflow state is obtained, the media reproduction device shifts to the first operation state.
Continuous media data is data encoded in the MPEG format,
The correction unit uses an accumulated amount of the continuous media data and a PTS (Presentation Timp Stamp) added to the first frame of the accumulated continuous media data as a presentation time, and an STC (System Time Clock) value indicating a reference time The media reproducing apparatus according to claim 6, wherein the medium is corrected.
Multiple media playback units
The correction unit
A state in which at least one of the plurality of media playback units is in a near underflow state of continuous media data and the reference time and the presentation time are coincident or the reference time is ahead of the presentation time When the determination result is obtained, the offset value is calculated so that the reproduction start of the continuous media data having the largest presentation time value among the plurality of continuous media data is delayed,
When a determination result is obtained that at least one of the plurality of media playback units is in a near overflow state of continuous media data and the reference time is delayed from the presentation time, the plurality of media playback units 5. The media playback apparatus according to claim 4, wherein an offset value is calculated so that playback of the continuous media data having the smallest presentation time value among the continuous media data proceeds.
Continuous media data is data encoded in the MPEG format,
The correction unit uses an accumulated amount of the continuous media data and a PTS (Presentation Timp Stamp) added to the first frame of the accumulated continuous media data as a presentation time, and an STC (System Time Clock) value indicating a reference time The media reproducing apparatus according to claim 8, wherein the medium is corrected.
A speed conversion unit that converts the presentation speed of the media data reproduced by the media reproduction unit into a value according to the instruction;
2. The media playback apparatus according to claim 1, wherein the correction unit instructs the speed conversion unit of a presentation speed value adjusted in accordance with a state of continuous media data accumulation in the media playback unit.
Continuous media data is data encoded in the MPEG format,
The correction unit uses an accumulated amount of the continuous media data and a PTS (Presentation Timp Stamp) added to the first frame of the accumulated continuous media data as a presentation time, and an STC (System Time Clock) value indicating a reference time The media reproducing apparatus according to claim 10, wherein the medium is corrected.