JP2008067179A - Receiver and coded data reproduction method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a receiver and a coded data reproduction method which output a visually successful reproduced image even when a transmission rate of coded data sharply deviates from regulation in coding by lack of bands of a transmission path or temporary delay. <P>SOLUTION: An offset setting part 10 input buffer occupancy from a video buffer 8, when the buffer occupancy is equal to or less than a preset lower limit threshold, increases offset and when the buffer occupancy is equal to or more than a preset upper limit threshold, decreases the offset. In a decoding control part 15, time obtained by adding the offset to display time (PTS) included in the coded data is treated as the PTS to control decoding and display. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to a receiving apparatus and encoded data reproduction method, and more specifically, a decoding method for moving image encoded data that has been encoded with high efficiency, and when decoding as prescribed cannot be performed due to a delay in a transmission path or the like In particular, the present invention relates to an encoded data reproduction apparatus and an encoded data reproduction method for displaying a visually improved reproduction image.

  In recent years, video signal distribution means have been diversified, and not only broadcast waves but also video distribution using the Internet has been performed. In general, high-efficiency encoding is performed in order to use a limited broadcast band or line band in moving image distribution. For example, audio and video are encoded with high efficiency, time division multiplexed, and distributed as an MPEG2 (Moving Pictures Experts Group 2) transport stream (hereinafter referred to as MPEG2-TS).

  In the case of the MPEG2-TS system, a PTS (Presentation Time Stamp) representing the display time of audio and video, a DTS (Decoding Time Stamp) representing the decoding time, and a PCR (Program Clock Reference) representing the reference time information are multiplexed. The time information is used to control playback time and to synchronize audio and video. Among these, PCR is time information for reproducing a clock called STC (System Time Clock), and is defined so that an STC can be accurately reproduced when a delay of a transmission path is constant. The receiving apparatus reproduces and displays and decodes audio frames and video frames corresponding to the times indicated by the PTS and DTS with reference to the correctly reproduced STC.

  In moving image distribution using the Internet, a sufficient transmission speed may not be obtained for distributed data. In this case, distribution data including PCR is transmitted with a delay, and the STC may not be reproduced accurately. When the STC cannot be reproduced accurately, there is a problem that the reproduced image itself is deteriorated due to the output deviation or the deterioration of the synchronization signal of the video output.

  In order to solve such a problem, Japanese Patent Application Laid-Open No. H10-228688 obtains a good reproduced image even when the PCR reception timing is not as specified. That is, in Patent Document 1, the number of video frames in the received reproduction data is counted, the reception data for every 30 frames is accumulated in the buffer, the data amount for 30 frames is measured, and the reproduction data for 30 frames is stored. The data is supplied to the decoder at a uniform rate over a period of 30 frames.

FIG. 5 shows the input / output timing of the buffer in Patent Document 1. FIG. 5A shows the input timing, and FIG. 5B shows the output timing to the decoder.
As shown in FIG. 5 (A), the playback data for 30 frames is not constant, so the time required for input is not constant. In the receiving apparatus, the transfer rate of the reproduction data for 30 frames is obtained from the code amount for 30 frames, and is output to the decoder at regular intervals over a period of 30 frames as shown in FIG.

In this way, in Patent Document 1, a delay occurs in the transmission path, and the transfer rate to the decoder does not change even when the input timing to the receiving apparatus changes. That is, even if there is a delay in the PCR, the reproduced data is made uniform so as to absorb the delay and transferred to the decoding.
JP 2006-128857 A

  However, in the distribution of moving images via the Internet, extreme transmission delays and steady transmission line transfer rate shortages are assumed. Under such circumstances, even with the technique of Patent Document 1, sufficient reproduction data cannot be supplied to the decoder.

FIG. 6 shows an operation when the transfer rate of the transmission line is constantly insufficient in Patent Document 1, FIG. 6 (A) is the input timing, and FIG. 6 (B) is the output timing to the decoder. It is.
When the transfer rate of the transmission path is steadily insufficient, as shown in FIG. 6A, it takes a longer time than 30 frames to input the reproduction data for 30 frames. As shown in B), when 30 frames of reproduction data is output to the decoder in a 30 frame period, a period during which the reproduction data cannot be output occurs.

  As shown in FIG. 6B, when there is a period during which reproduction data is not input to the decoder, DTS and PTS are greatly delayed with respect to STC, and normal reproduction and display cannot be performed by processing as expected at the time of encoding. Comes out. In addition, since there is a period during which no encoded data is supplied to the decoder, there is a problem in that the reproduced image is not updated at all during this period.

  The present invention has been made in consideration of the above-described circumstances, and the transmission rate of encoded data is greatly deviated from the encoding specification due to insufficient bandwidth of the transmission path or temporary delay. In the present invention, it is an object to provide a receiving apparatus and an encoded data reproduction method for outputting a visually reproduced image.

In order to solve the above-described problems, the receiving apparatus and the encoded data reproduction method of the present invention are configured as follows.
The receiving apparatus includes at least one of audio data and video data, and generates a reference clock based on the reference time information from encoded data including reference time information and display time information of the audio data or the video data. A receiving device that reproduces and reproduces the audio data or the video data corresponding to the display time information at a time when the reference clock and the display time information coincide with each other, the audio data, the video data, and the encoding A buffer for storing at least one of the data, a PTS extracting means for inputting the audio data or the video data and extracting display time information, a buffer occupancy of the buffer, and a preset upper threshold And the offset to set the offset by comparing with the lower threshold And a decoding control means for controlling the display time of the audio data or the video data based on the offset set by the offset setting means and the display time information extracted by the PTS extraction means. is there.

Here, the offset setting means sets a positive offset when the buffer occupancy of the buffer falls below the lower limit threshold, sets a negative offset when the buffer occupancy exceeds the upper limit threshold, and the decoder The control means may add an offset to the display time information, delay the display time when the offset is positive, and advance the display time when the offset is negative.
Further, when the buffer occupancy rate of the buffer becomes equal to or lower than the lower limit threshold value, a negative offset is set, and when the buffer occupancy rate becomes equal to or higher than the upper limit threshold value, a positive offset is set. By subtracting the offset from the information, the display time may be delayed when the offset is negative, and the display time may be advanced when the offset is positive.

  Further, the offset may be a constant multiple of the display frame interval.

  The encoded data reproduction method includes at least one of audio data and video data, and the reference time information is obtained from encoded data including reference time information and display time information of the audio data or the video data. An encoded data reproduction method for reproducing the audio data or the video data corresponding to the display time information at a time when the reference clock and the display time information coincide with each other. , Storing at least one of the video data and the encoded data in a buffer, setting a lower limit threshold and an upper limit threshold of the buffer occupancy in the buffer, and when the buffer occupancy is equal to or lower than the lower limit threshold, Add a positive offset to the display time information to delay the display time, and the buffer occupancy By adding a negative offset may be quickly display time on the display time information and becomes more serial upper threshold,

The encoded data reproduction method includes at least one of audio data and video data, and the reference time information is obtained from encoded data including reference time information and display time information of the audio data or the video data. An encoded data reproduction method for reproducing the audio data or the video data corresponding to the display time information at a time when the reference clock and the display time information coincide with each other. , Storing at least one of the video data and the encoded data in a buffer, and setting a lower limit threshold and an upper limit threshold of the buffer occupancy in the buffer, and when the buffer occupancy is equal to or lower than the lower limit threshold, The display time is delayed by adding a negative offset to the reference time information, and the buffer occupancy is It may be configured to quickly display time by adding a positive offset to the reference time information and equal to or larger than the upper limit threshold value.
Further, the offset may be a constant multiple of the display frame interval.

  Further, as each means of the receiving apparatus having the above-described configuration and the encoded data reproduction method, a program for causing the computer to function is created, or the program is recorded on a computer-readable recording medium. The above problem can also be solved by executing the program on a computer.

  According to the present invention, since the PTS or STC is adjusted based on the occupation ratio of the encoded data buffer, audio data buffer, or video data buffer, the decoding process may be controlled according to the adjusted PTS or STC. . Therefore, even when the input data rate is greatly deviated from the assumption at the time of encoding, there is an effect that the entire control itself can be made common with normal processing.

  Also, by adjusting the PTS or STC, the occupancy of the encoded data buffer, audio data buffer, and video data buffer is maintained appropriately, so that overflow and underflow do not occur, and the decoder has no frame. There is also an effect that audio data and video data in units can be input reliably.

  DESCRIPTION OF EXEMPLARY EMBODIMENTS Hereinafter, exemplary embodiments according to a receiving device and an encoded data reproduction method of the invention will be described in detail with reference to the drawings.

  FIG. 1 is a functional block diagram of a receiving apparatus according to the present embodiment, in which the receiving apparatus includes a demultiplexer unit 1, a PCR detection unit 2, a subtraction circuit 3, an STC unit 4, a VCXO (Voltage Controlled Xtal Oscillator). : Voltage controlled crystal oscillator) 5, audio PTS extraction unit 6, audio buffer 7, video buffer 8, video PTS extraction unit 9, offset setting unit 10, addition circuit 11, addition circuit 12, audio decoder 13, video decoder 14, decoding The control unit 15 is configured.

  Hereinafter, an operation when MPEG2-TS is input to the receiving apparatus according to the present embodiment will be described.

First, the demultiplexer unit 1 separates the input MPEG2-TS data into audio data, video data, and PCR data.
The separated video data is stored in the video buffer 8 and supplied to the video PTS extraction unit 9. The video buffer 8 outputs the amount of video data stored with respect to the video buffer size, that is, the buffer occupancy rate.
Audio data is stored in the audio buffer 7 and supplied to the audio PTS extraction unit 6. The audio buffer 7 may also output the buffer occupation amount like the video buffer 8.
Further, the PCR data is supplied to the PCR detection unit 2.

The PCR detection unit 2 extracts PCR from the supplied PCR data.
The STC unit 4 receives, for example, a 27 MHz clock from the VXCO 5, generates an STC value, and inputs this STC value to the subtraction circuit 3.
The subtraction circuit 3 performs subtraction between the STC value and the value detected by the PCR detection unit 2, feeds back the subtraction result to the VCXO 5, and the feedback compensated clock is input to the STC unit 4. In the STC unit 4, the clock assumed on the encoding side is accurately reproduced by this feedback circuit.

The audio PTS extraction unit 6 receives the audio data separated by the demultiplexer unit 1, extracts the audio PTS, and supplies it to the adder circuit 11.
The video PTS extraction unit 9 receives the video data separated by the demultiplexer unit 1, extracts the video PTS, and supplies it to the adder circuit 12.

  The offset setting unit 10 compares the buffer occupancy of the video buffer 8 with preset upper and lower thresholds, sets an offset value, and supplies the offset to the addition circuits 11 and 12. The addition circuits 11 and 12 perform addition with the audio PTS and video PTS, respectively, and the added audio PTS and video PTS are supplied to the decode control unit 15. The operation of the offset setting unit 10 will be described later.

  The decode control unit 15 controls reading of audio data from the audio buffer 7, audio decoding and reproduction by the audio decoder 13, and reading of video data from the video buffer 8 and video decoding and reproduction by the video decoder 14.

  PCR data may be multiplexed with video data or audio data. In this case, video data and audio data including PCR are input to the video buffer 8 and the audio buffer 7 and input to the PCR detection unit 2 in an overlapping manner.

  FIG. 2 shows a setting example of the transition of the buffer occupancy ratio and the offset of the video buffer 8. The vertical axis in FIG. 2 indicates the buffer occupancy rate, the dotted line in (A) indicates the buffer occupancy rate of 100% when the buffer is FULL, and the dotted lines in (B) and (C) are the upper thresholds of the buffer occupancy rate, respectively. And the lower limit threshold. Further, the horizontal axis in FIG. 2 represents the time axis.

When the buffer occupancy deviates from the range indicated by the lower limit threshold value (C) and the upper limit threshold value (B), the offset setting unit 10 adjusts the offset to fall within this range.
Also, t ₀ to t _{7 on} the time axis in FIG. 2 are times when video data is displayed, which are times indicated by the PTS detected by the video PTS detector 9.

2, stores the video data when starting the decoding, to decode the first video data at time t _0. In the buffer occupation rate transition model shown in FIG. 2, for simplification, the video data output from the video buffer 8 is described as being instantaneous.
From the time t ₂ for a period of t ₇ the dotted line (D) is, if a transition in the case where a sufficient line speed was to have been maintained, a solid line (E) is temporarily line speed is lowered enough video data The transition when no is input is shown.

In the transition of the solid (E), below the threshold (C) of the lower limit at _{t 3,} is below the lower limit of the threshold value (C) also display timing _{t 4} and _{t 5} after. In this case, without performing the decoding process, the offset is increased until the buffer occupancy exceeds the lower limit threshold (C). Here, one unit of offset corresponds to one frame period.

Next, the operation of the decode control unit 15 when the offset is increased will be described using FIG. 2 as an example.
When the offset increases from 0 to 1 at time t ₄ , t _{(4 + 1)} is input as the PTS to the decoding control unit 15 so that the video data originally reproduced at time t ₄ is reproduced at time t _5. Be controlled. Further, when the offset increases from 1 to 2 at time t ₅ , t _{(4 + 2)} is input as the PTS to the decoding control unit 15 so that the video data originally reproduced at time t ₄ is reproduced at time t _6. Controlled. Also controlled so video data to be reproduced originally t ₅ are reproduced at time t _7.

  Next, FIG. 3 shows a setting example of the transition of the buffer occupation rate and the offset of the video buffer 8 when the line speed is restored. Similar to FIG. 2, the vertical axis in FIG. 3 indicates the buffer occupancy, the dotted line in (A) indicates the buffer occupancy 100% in which the buffer is FULL, and the dotted lines in (B) and (C) indicate the buffer occupancy, respectively. The upper limit threshold and the lower limit threshold of the occupation ratio are shown.

Also, the horizontal axis of FIG. 3 indicates the time axis, and t ₇ to t _{13 on} the time axis are times when video data is displayed, and are times indicated by the PTS detected by the video PTS detector 9.
The dotted line in the period up to time t ₁₁ (D) is, if a transition in the case where a sufficient line speed was to have been maintained, a solid line (E) is or are entered enough video data is over-supplied The transition is shown.

In FIG. 3, since the buffer occupancy is within the range between the lower limit threshold (C) and the upper limit threshold (B) from time t ₇ to t ₉ , decoding is continued with the offset being 2. That is, the state of being played back at a timing delayed by 2 frames from the standard is continued.
At time t _10, since the buffer occupancy is greater than or equal to the upper threshold (B), reduced 1 offset, which is the buffer occupancy than the upper threshold (B) even more time t ₁₁ Therefore, the offset is further reduced by one.

Next, FIG. 3 as an example, the operation of the decoding control unit 15 between time t ₇ t ₁₃ will be described.
Since the offset is 2 until time t ₉ , for example, at time t ₈ , t _{(8 + 2)} is input to the decoding control unit 15 as PTS, so that video data originally reproduced at time t ₈ is reproduced at time t _10. Controlled.

When the offset decreases from 2 to 1 at time t ₁₀ , t _{(10 + 1)} is input as the PTS to the decoding control unit 15, and control is performed so that video data originally reproduced at time t ₁₀ is reproduced at time t _11. Is done.
At this time, control is performed so that the video data originally reproduced at time t ₈ is reproduced, and the video data of the next frame is deleted from the video buffer 8.

Furthermore, when the offset decreases from 1 to 0 at time t ₁₁ , the original t ₁₁ is input to the decode control unit 15 as the PTS. Here, the video data originally reproduced at time t ₁₀ is reproduced, and the video data reproduced at time t ₁₁ is deleted from the video buffer 8.

  If the video data is highly efficient encoded, deleting the video data from the buffer as it is may affect the previous and next video frames. The encoded data may be removed from the buffer.

Next, operations of offset setting and decoding control will be described using the flowchart of FIG.
First, the offset is reset to 0, and an upper limit threshold and a lower limit threshold of the buffer occupancy are set (step S0401).
It is determined whether or not the clock generated by the STC unit 4 and the video PTS extracted by the video PTS extraction unit 9 are equal (picture reproduction time). If it is not the picture reproduction time (step S0402 / NO), picture reproduction is performed. On the other hand, if it is the picture playback time (step S0402 / YES), it is determined whether or not the buffer occupancy is equal to or greater than the upper limit threshold (step S0403).

If the buffer occupancy is equal to or greater than the upper limit threshold (step S0403 / YES), 1 is subtracted from the offset (step S0404), and the process proceeds to step S0407.
On the other hand, if the buffer occupancy is less than the upper limit threshold (step S0403 / NO), it is determined whether the buffer occupancy is less than or equal to the threshold for adjustment (step S0405).

If the buffer occupancy is equal to or lower than the lower limit threshold (step S0405 / YES), 1 is added to the offset (step S0406), and the process proceeds to step S0407.
On the other hand, if the buffer occupancy is less than the lower limit threshold (step S0405 / NO), the process proceeds to step S0407.

  Then, the decoder control unit 15 performs decoding and display control based on the offset PTS (step S0407), determines whether or not the decoding is completed (step S0408), and repeats steps S0402 to S0408 until the decoding is completed.

In the embodiment described above, the offset is increased or decreased by a value corresponding to one frame period, but may be a constant multiple of a value corresponding to one frame period.
Further, although it is determined whether the buffer occupancy is equal to or higher than the upper limit threshold or lower than the lower limit threshold, the buffer occupancy may exceed the upper limit threshold or be lower than the lower limit threshold.
Further, although the determination is made using the buffer occupation rate of the video buffer, an audio buffer may be used.
Further, a buffer for storing the encoded data (MPEG2-TS) as it is may be provided, and the determination may be performed using the occupation rate of the buffer.

  Furthermore, in the above-described embodiment, an offset is added to the PTS, but a reverse offset may be added to the STC.

  As described above, by configuring the present embodiment, the PTS or STC is adjusted based on the occupation ratio of the encoded data buffer, the audio data buffer, or the video data buffer, so that the decoding process control is adjusted. Therefore, even when the input data rate is greatly deviated from the assumption at the time of encoding, the entire control itself can be shared with normal processing.

  Also, by adjusting the PTS or STC, the occupancy of the encoded data buffer, audio data buffer, and video data buffer is maintained appropriately, so that overflow and underflow do not occur, and the decoder has no frame. There is also an effect that audio data and video data in units can be input reliably.

  The embodiment of the present invention can also be realized by converting the functions of the respective units of the above receiving apparatus into a computer program, and installing and executing the computer program on the receiving apparatus. Further, the computer program can be recorded on a detachable recording medium or downloaded via a network, so that the transfer can be simplified and implemented easily.

  Note that the present invention is not limited to the above-described embodiment, and various modifications and corrections can be made without departing from the scope of the present invention.

It is a functional block diagram of the receiver which concerns on embodiment. It is a figure which shows control of a buffer occupation rate and offset when a line speed is slow. It is a figure which shows control of a buffer occupation rate and offset when a line speed is high. 5 is a flowchart showing an offset setting operation and a decoding control operation in the embodiment. It is a figure which shows the relationship between the input data and output data of the buffer in a prior art. In prior art, it is a figure which shows the relationship between the input data and output data of a buffer when a line speed is slow.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 ... Demultiplexer part, 2 ... PCR detection part, 3 ... Subtraction circuit, 4 ... STC part, 5 ... VCXO, 6 ... Audio PTS extraction part, 7 ... Audio buffer, 8 ... Video buffer, 9 ... Video PTS extraction part, DESCRIPTION OF SYMBOLS 10 ... Offset setting part, 11 ... Adder circuit, 12 ... Adder circuit, 13 ... Audio decoder, 14 ... Video decoder, 15 ... Decode control part.

Claims (10)

A reference clock is reproduced based on the reference time information from encoded data including at least one of audio data and video data, and including reference time information and display time information of the audio data or the video data. A receiving device that reproduces the audio data or the video data corresponding to the display time information at a time when a clock and the display time information match;
A buffer for storing at least one of the audio data, the video data, and the encoded data;
PTS extraction means for inputting the audio data or the video data and extracting display time information;
An offset setting means for setting an offset by comparing a buffer occupation rate of the buffer with a preset upper limit threshold and a lower limit threshold;
And a decoding control means for controlling the display time of the audio data or the video data based on the offset set by the offset setting means and the display time information extracted by the PTS extraction means. apparatus.   The receiving apparatus according to claim 1, wherein the offset setting means sets a positive offset when the buffer occupancy rate of the buffer is equal to or lower than the lower limit threshold value, and is negative when the buffer occupancy rate is equal to or higher than the upper limit threshold value. An offset is set, and the decoder control means adds the offset to the display time information, delays the display time when the offset is positive, and advances the display time when the offset is negative. apparatus.   The receiving apparatus according to claim 1, wherein the offset setting means sets a negative offset when the buffer occupancy rate of the buffer is equal to or lower than the lower limit threshold value, and is positive when the buffer occupancy rate is equal to or higher than the upper limit threshold value. An offset is set, and the decoder control means subtracts the offset from the display time information, delays the display time when the offset is negative, and advances the display time when the offset is positive. apparatus.   4. The receiving apparatus according to claim 1, wherein the offset is a constant multiple of a display frame interval. A reference clock is reproduced based on the reference time information from encoded data including at least one of audio data and video data, and including reference time information and display time information of the audio data or the video data. An encoded data reproduction method for reproducing the audio data or the video data corresponding to the display time information at a time when a clock and the display time information coincide with each other,
Storing at least one of the audio data, the video data, and the encoded data in a buffer;
A lower limit threshold and an upper limit threshold for the buffer occupancy rate are set in the buffer, and when the buffer occupancy rate is equal to or lower than the lower limit threshold value, a positive offset is added to the display time information to delay the display time, and the buffer occupancy rate is An encoded data reproduction method characterized in that a negative offset is added to the display time information to make the display time earlier when the upper limit threshold is exceeded. A reference clock is reproduced based on the reference time information from encoded data including at least one of audio data and video data, and including reference time information and display time information of the audio data or the video data. An encoded data reproduction method for reproducing the audio data or the video data corresponding to the display time information at a time when a clock and the display time information coincide with each other,
Storing at least one of the audio data, the video data, and the encoded data in a buffer;
A lower limit threshold and an upper limit threshold of the buffer occupancy rate are set in the buffer, and when the buffer occupancy rate falls below the lower limit threshold value, a negative offset is added to the reference time information to delay the display time, and the buffer occupancy rate A coded data reproduction method characterized in that a positive offset is added to the reference time information to make the display time earlier when the upper limit threshold is exceeded.   7. The encoded data reproduction method according to claim 5, wherein the offset is a constant multiple of a display frame interval.   The program for functioning a computer as each means of the receiver in any one of Claims 1 thru | or 4.   A program for causing a computer to execute the encoded data reproduction method according to any one of claims 5 to 7.   A computer-readable recording medium on which the program according to claim 8 or 9 is recorded.

Images