JP2005348160A - Stream decoder - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To synchronize audio and video without using synchronization information. <P>SOLUTION: In a stream decoder, it is provided with a video decoder which decodes video streams and outputs a first piece of decoding information according to the size of a part which can not normally be decoded, etc. in the video streams and decoded video data, an audio decoder which decodes audio streams and outputs a second piece of decoding information according to the size of a part which can not normally be decoded, etc. in the audio streams and decoded video data, a video output buffer which stores and outputs the decoded video data, an audio output buffer which stores and outputs the decoded audio data and an output control part which controls the video output buffer and the audio output buffer so that the decoded video data and the decoded audio data are synchronized and outputted according to the first and second pieces of decoding information. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to a stream decoding apparatus that decodes and outputs a bit stream, and more particularly to a technique for synchronizing decoded video data and audio data.

  In recent years, with the spread of broadband Internet, large data size files have been actively exchanged. High-quality, high-quality video files have been transferred particularly frequently after broadband. There are various video file formats such as WMV (Windows Media Video), ASF (Advanced Systems Format), MPEG (Moving Picture Experts Group), and AVI (Audio Video Interleaving).

  For example, for the AVI format, the algorithm for compressing the file is independent of the system, so that various compression techniques can be used by preparing various compression codecs. That is, it is not necessary to unify the compression codec used for audio and video compression, and for example, MP3 (MPEG Audio Layer 3) for audio and DivX compression technology for video can be used in combination.

However, since this format has a high degree of freedom, the AVI format file often does not contain information about synchronization between audio and video. For this reason, at the time of reproduction, if the synchronization between the audio and the video is once shifted, it cannot be synchronized again. Therefore, when the decoding process of either audio or video is likely to be delayed, the technology that degrades the quality of the decoding process that is likely to be delayed, that is, the process is performed using a light process as the decoding process. There is known a technique for preventing a delay and preventing a synchronization error (see, for example, Patent Document 1).
Japanese Patent Laid-Open No. 11-112936

  FIG. 9 is a block diagram illustrating a configuration example of a conventional stream decoding apparatus. It is assumed that a bit stream STRM for transmitting an AVI format file is input to the input stream buffer 911. Of these, the video stream is input to the video decoder 922 via the video stream buffer 912, and the audio stream is input to the audio decoder 924 via the audio stream buffer 914.

  The video decoder 922 decodes the video stream and outputs the result VOUT, and the audio decoder 924 decodes the audio stream and outputs the result AOUT. Since the AVI format file does not include a synchronization signal for synchronizing audio and video, the audio and video decoding results are sequentially output as they are.

  FIG. 10A is an explanatory diagram illustrating an example of a video stream input to the video decoder 922 and an audio stream input to the audio decoder 924 in FIG. FIG. 10B is an explanatory diagram showing an example of video data and audio data obtained by decoding the bit stream of FIG.

  As shown in FIG. 10A, when there is no error in the video stream and there is an error in the audio stream, the audio decoder 924 skips the error part and decodes it. For this reason, when audio and video are output sequentially as they are, as shown in FIG. 10 (b), after the point where there is an error, the audio advances ahead of the video, and between the audio and video. Will be out of sync.

  Thus, in a stream that transmits a file that does not include synchronization information, such as an AVI format file, if there is an error in either the audio or video stream itself, the synchronization between the audio and video is performed during decoding. Will shift. And since there is no synchronization information, when the synchronization between the audio and the video is out of sync, the timing of both cannot be corrected and synchronized again.

  An object of the present invention is to provide a stream decoding device that can synchronize audio and video again without using synchronization information even if the synchronization between the audio and video is lost.

  In order to solve the above-mentioned problem, the means taken by the invention of claim 1 is a stream decoding apparatus that decodes a bit stream having an audio stream and a video stream, at least one of which does not include synchronization information. A video decoder that decodes the video stream, and outputs first decoding information according to a size of a part that can be normally decoded or a part that cannot be decoded normally, and decoded video data; An audio decoder that decodes the audio stream, and outputs second decoding information according to a size of a part that can be normally decoded or a part that cannot be decoded normally; and the decoded audio data; and the decoding Video output to store and output recorded video data A buffer; an audio output buffer for storing and outputting the decoded audio data; and the decoded video data and the decoded audio data are output in synchronization according to the first and second decoding information. As described above, an output control unit that controls the video output buffer and the audio output buffer is provided.

  According to this, even if there is a part that could not be normally decoded due to an error included in the bitstream, even if the synchronization between the video and the audio is out of sync, the first and second decoding information causes the video and Since the time difference between the audio and the audio can be known, the video and audio can be synchronized.

  According to a second aspect of the present invention, in the stream decoding device according to the first aspect, the first decoding information is a size of a portion of the video stream that has not been successfully decoded, and the second decoding information Is the size of the portion of the audio stream that could not be decoded normally.

  According to a third aspect of the present invention, in the stream decoding device according to the first aspect, the first decoding information is a time required for reproducing a portion of the video stream that has not been normally decoded, The second decoding information is a time required for reproducing a portion of the audio stream that could not be normally decoded.

  According to a fourth aspect of the present invention, in the stream decoding device according to the third aspect, the output control unit, based on a difference between the first decoded information and the second decoded information, The audio output buffer is controlled.

  According to a fifth aspect of the present invention, in the stream decoding device according to the first aspect, the first decoding information is a time required to reproduce a portion of the video stream that has been successfully decoded, The second decoding information is a time required for reproducing a portion of the audio stream that has been successfully decoded.

  According to a sixth aspect of the present invention, in the stream decoding device according to the fifth aspect, the output control unit, based on a difference between the first decoded information and the second decoded information, The audio output buffer is controlled.

  According to a seventh aspect of the present invention, in the stream decoding device according to the first aspect, the output control unit is a portion of the decoded video data and the decoded audio data that cannot be normally decoded during decoding. The video output buffer and the audio output buffer are controlled so as to delay the output of the one having the error.

  According to an eighth aspect of the present invention, in the stream decoding device according to the first aspect, the output control unit is a portion of the decoded video data and the decoded audio data that cannot be normally decoded at the time of decoding. The video output buffer and the audio output buffer are controlled so as to speed up the output of which there is no signal.

  According to a ninth aspect of the present invention, in the stream decoding device according to the first aspect, the output control unit determines the progress of the output that is advanced between the decoded video data and the decoded audio data. The video output buffer and the audio output buffer are controlled so as to be paused.

  According to a tenth aspect of the present invention, in the stream decoding device according to the first aspect, the output control unit skips the delayed one of the decoded video data and the decoded audio data. The video output buffer and the audio output buffer are controlled.

  In the invention of claim 11, in the stream decoding device according to claim 1, the output control unit reproduces the advanced one of the decoded video data and the decoded audio data at a low speed. Thus, the video output buffer and the audio output buffer are controlled.

  According to a twelfth aspect of the present invention, in the stream decoding device according to the first aspect, the output control unit reproduces the delayed one of the decoded video data and the decoded audio data at high speed. Thus, the video output buffer and the audio output buffer are controlled.

  The invention according to claim 13 is the stream decoding device according to claim 1, further comprising a stream buffer that stores the bit stream and outputs the bit stream to the video decoder and the audio decoder, wherein the output control unit includes the first decoder. And when detecting that a time difference required for reproduction between the decoded video data and the decoded audio data is larger than a predetermined value based on the second decoding information, the stream buffer, The video output buffer and the audio output buffer are cleared.

  According to a fourteenth aspect of the present invention, in the stream decoding device according to the first aspect, information to be obtained as the first or second decoding information when there is no portion that cannot be normally decoded is stored as reference information. A reference information storage unit that outputs the reference information, and the output control unit controls the video output buffer according to the reference information and the first decode information, and the reference information and the second decode The audio output buffer is controlled according to the information.

  According to this, since the video and audio outputs can be controlled based on the case where the video stream or the audio stream can be normally decoded, the synchronization between the video and the audio can be maintained. it can.

  According to a fifteenth aspect of the present invention, in the stream decoding device according to the fourteenth aspect, the reference information is information representing a data size.

  According to a sixteenth aspect of the present invention, in the stream decoding device according to the fourteenth aspect, the reference information is information representing a time required for data reproduction.

  In the invention of claim 17, in the stream decoding device according to claim 14, the first decoding information is a size of a part of the video stream that can be normally decoded, and the second decoding information The decode information is the size of the portion of the audio stream that has been successfully decoded.

  In the invention according to claim 18, in the stream decoding device according to claim 14, the first decode information is a time required to reproduce a portion of the video stream that has been successfully decoded, The second decoding information is a time required for reproducing a portion of the audio stream that has been successfully decoded.

  According to a nineteenth aspect of the present invention, in the stream decoding device according to the fourteenth aspect, the output control unit determines the progress of the output that is advanced between the decoded video data and the decoded audio data. The video output buffer and the audio output buffer are controlled so as to be paused.

  According to a twentieth aspect of the present invention, in the stream decoding device according to the fourteenth aspect, the output control unit skips a delayed one of the decoded video data and the decoded audio data. The video output buffer and the audio output buffer are controlled.

  According to a twenty-first aspect of the invention, in the stream decoding device according to the fourteenth aspect, the output control unit reproduces the advanced one of the decoded video data and the decoded audio data at a low speed. Thus, the video output buffer and the audio output buffer are controlled.

  According to a twenty-second aspect of the present invention, in the stream decoding device according to the fourteenth aspect, the output control section reproduces the delayed one of the decoded video data and the decoded audio data at a high speed. Thus, the video output buffer and the audio output buffer are controlled.

  The invention according to claim 23 is the stream decoding device according to claim 14, further comprising a stream buffer that stores the bit stream and outputs the bit stream to the video decoder and the audio decoder, wherein the output control unit includes the reference information And the stream buffer, the video output buffer, and the audio output buffer are cleared when it is detected that the difference between the first decoding information and the second decoding information is larger than a predetermined value. .

  According to the present invention, even if the audio and video are out of synchronization, they can be synchronized again without using synchronization information. Therefore, when decoding a bitstream that does not have a signal for synchronization, synchronization between audio and video can be maintained.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

(First embodiment)
FIG. 1 is a block diagram showing a configuration example of a stream decoding apparatus according to the first embodiment of the present invention. The stream decoding apparatus of FIG. 1 includes a video decoder 22, an audio decoder 24, a video output buffer 26, an audio output buffer 28, decode information storage units 32 and 34, and an output control unit 42, and serves as a stream buffer. , An input stream buffer 11, a video stream buffer 12, and an audio stream buffer 14.

  It is assumed that a bit stream STRM for transmitting an AVI format file is input to the input stream buffer 11. The bit stream STRM includes an audio stream and a video stream, and none of them includes synchronization information. The input stream buffer 11 stores the bit stream STRM by a predetermined size, and the video stream buffer 12 and the audio stream buffer 14 store the video stream and the audio stream of the stored bit stream STRM, respectively.

  The video decoder 22 reads and decodes the video stream from the video stream buffer 12 and outputs the result to the video output buffer 26. Further, the video decoder 22 outputs the decode information obtained at the time of decoding to the decode information storage unit 32.

  The audio decoder 24 reads and decodes the audio stream from the audio stream buffer 14 and outputs the result to the audio output buffer 28. Further, the audio decoder 24 outputs the decode information obtained at the time of decoding to the decode information storage unit 34.

  For example, the video decoder 22 obtains the size of the part of the input video stream that could not be decoded normally, that is, the part that causes an error (hereinafter referred to as the error data size) as decoding information. Output. Similarly, the audio decoder 24 obtains an error data size that is the size of a portion of the input audio stream that could not be normally decoded and outputs it as decode information.

  Each of the decode information storage units 32 and 34 stores the received decode information and outputs it to the output control unit 42. The output control unit 42 controls the video output buffer 26 and the audio output buffer 28 according to the decode information output from the decode information storage units 32 and 34. The video output buffer 26 and the audio output buffer 28 output video data VOUT and audio data AOUT, respectively, under the control of the output control unit 42.

  FIG. 2A is an explanatory diagram illustrating an example of a video stream input to the video decoder 22 and an audio stream input to the audio decoder 24 in FIG. FIG. 2B is an explanatory diagram illustrating an example of video data and audio data obtained by decoding the bit stream of FIG. FIG. 2C is an explanatory diagram showing an example of the video data VOUT output from the video output buffer 26 and the audio data AOUT output from the audio output buffer 28 in FIG.

  As shown in FIG. 2A, if there is no error in the video stream and there is an error in the audio stream, the audio decoder 24 skips the error portion and decodes it. For this reason, the outputs of the video decoder 22 and the audio decoder 24 are as shown in FIG. 2 (b). After the point where the error has occurred, the audio advances ahead of the video. Synchronization will be lost. In FIG. 2B, the time required for reproducing the decoded video data is Y, and the time required for reproducing the decoded audio data is X (X <Y).

  Therefore, the output control unit 42, in accordance with the decode information output from the video decoder 22 and the audio decoder 24, outputs the video output buffer 26 and the video output buffer 26 and the decoded audio data in synchronization with each other. The audio output buffer 28 is controlled.

  That is, the output control unit 42 divides the decode information (video error data size) output from the decode information storage unit 32 by the video bit rate, and the time required to reproduce the data whose size is the error data size (error Reproduction time) is obtained for the video, and the decoding information (audio error data size) output from the decoding information storage unit 34 is divided by the audio bit rate to obtain the audio error reproduction time. It is assumed that the output control unit 42 has already obtained the video and audio bit rates from the header information of the bitstream STRM.

  Further, the output control unit 42 obtains the difference between the video error reproduction time and the audio error reproduction time, and in other words, the delayed one of the video and audio, in other words, the decoded video data and audio data. The video output buffer 26 and the audio output buffer 28 are controlled so that the output of the data having no error is made faster and output in a time shorter than the time required to output the data as it is.

  In the case of FIG. 2B, since the video without error is delayed, the output control unit 42 causes the video output buffer 26 to skip the video data by the determined time difference, and the video and audio. Synchronize with. In other words, the decoded video data is reproduced during time X. Then, the video data VOUT and the audio data AOUT output from the video output buffer 26 and the audio output buffer 28 are as shown in FIG. 2C, and synchronization between video and audio can be maintained.

  Instead of skipping the video output, the output control unit causes the video output buffer 26 to reproduce the video at a high speed so that, for example, the output is performed in a time shorter than the time required for the output as it is. (That is, to shorten the video frame interval).

  FIG. 3 is an explanatory diagram showing another example of the video data VOUT output from the video output buffer and the audio data AOUT output from the audio output buffer in FIG. The output control unit in FIG. 1 may be configured as follows.

  That is, the output control unit obtains the difference between the video error playback time and the audio error playback time, and there is an error in the video and audio, in other words, the more advanced video and audio. The video output buffer 26 and the audio output buffer 28 may be controlled so that the data output is delayed and output in a time longer than the time required to output the data as it is.

  In the case of FIG. 2B, since the audio having the error is advanced, the output control unit causes the audio output buffer 28 to temporarily stop the progress of the audio output by the determined time difference, and Synchronize with audio. In other words, the decoded audio data is reproduced during time Y. Then, the video data VOUT and the audio data AOUT output from the video output buffer 26 and the audio output buffer 28 are as shown in FIG. 3, and synchronization between video and audio can be maintained.

  As described above, according to the stream decoding device of FIG. 1, even when video and audio are not synchronized, without using synchronization information and without adding data to decoded video data or audio data, The synchronization between video and audio can be kept again.

  Instead of temporarily stopping the progress of audio output, the output control unit, for example, outputs audio to the audio output buffer 28 so that the audio output buffer 28 outputs the audio data in a time difference that is longer than the time required to output the audio data as it is. You may make it reproduce at low speed.

  Further, in the present embodiment, the video decoder 22 and the audio decoder 24 have been described as outputting error data sizes as decode information, but instead of this, normal decoding of the input stream may be performed. The size of the completed portion (hereinafter referred to as normal reproduction data size) may be output as decode information.

  In addition, it is assumed that the video decoder and the audio decoder have already obtained the video and audio bit rates from the header information of the bitstream STRM, respectively, and the video and audio cannot be normally decoded. Each time required may be obtained and output as decode information. That is, the error data sizes of video and audio may be divided by the respective bit rates, and the obtained values (video and audio error reproduction times) may be output as decode information.

  Further, it is assumed that the video decoder and the audio decoder have already obtained the bit rates of video and audio from the header information of the bit stream STRM, respectively, and it is necessary to reproduce the part that has been successfully decoded for video and audio. Each time may be obtained and output as decode information. That is, the normal playback data sizes of video and audio are obtained, the respective normal playback data sizes are divided by the respective bit rates, and the obtained values (normal playback times of video and audio) are output as decode information. Also good.

(Modification of the first embodiment)
FIG. 4 is a block diagram showing a configuration example of a stream decoding apparatus according to a modification of the first embodiment of the present invention. The stream decoding device in FIG. 4 includes the decode information storage unit 232 in place of the decode information storage units 32 and 34 in the stream decoding device in FIG. 1, and instead of the video decoder 22, the audio decoder 24, and the output control unit 42. , A video decoder 222, an audio decoder 224, and an output control unit 242. Other components are the same as those described with reference to FIG.

  From the header information of the bitstream STRM, the video decoder 222 obtains the video bit rate, and the audio decoder 224 obtains the audio bit rate. The video decoder 222 reads and decodes the video stream from the video stream buffer 12 and outputs the result to the video output buffer 26. In addition, the video decoder 222 outputs the decode information obtained at the time of decoding to the decode information storage unit 232.

  The audio decoder 224 reads and decodes the audio stream from the audio stream buffer 14 and outputs the result to the audio output buffer 28. Also, the audio decoder 224 outputs the decode information obtained at the time of decoding to the decode information storage unit 232.

  For example, the video decoder 222 obtains the error data size of the input video stream, divides this by the video bit rate, and outputs the obtained value (video error reproduction time) as decode information. The audio decoder 224 obtains the error data size of the input audio stream, divides this by the audio bit rate, and outputs the obtained value (audio error reproduction time) as decode information.

  Alternatively, the video decoder obtains the normal data size of the video, divides this by the video bit rate, outputs the obtained value (normal video playback time) as decode information, and The audio decoder may obtain the normal data size of audio, divide this by the audio bit rate, and output the obtained value (normal audio reproduction time) as decode information.

  The decode information storage unit 232 receives the decode information from the video decoder 22 and the audio decoder 24, obtains and stores the difference between the decode information of the video decoder 22 and the decode information of the audio decoder 24, and outputs this to the output control unit 242. To do. The value stored in the decode information storage unit 232 is a value indicating the time difference between video and audio.

  Based on the decode information output from the decode information storage unit 232, the output control unit 242 outputs the video output buffer 26 and the audio output so that the decoded video data and the decoded audio data are output in synchronization. The buffer 28 is controlled. The details of the control by the output control unit 242 are the same as those of the output control unit 42 of FIG. 1 after the time difference between the video error playback time and the audio error playback time is obtained. Omitted.

(Second Embodiment)
FIG. 5 is a block diagram showing a configuration example of a stream decoding apparatus according to the second embodiment of the present invention. 5 includes an input stream buffer 311, a video stream buffer 312, an audio stream buffer 314, a video decoder 22, an audio decoder 24, a video output buffer 326, an audio output buffer 328, and decoding information. Storage units 332 and 334 and an output control unit 342 are provided.

  The input stream buffer 311, the video stream buffer 312, the audio stream buffer 314, the video output buffer 326, the audio output buffer 328, and the decode information storage units 332 and 334 are stored according to the reset signal output from the output control unit 342, respectively. The configuration is such that the data being deleted is erased from the input stream buffer 11, the video stream buffer 12, the audio stream buffer 14, the video output buffer 26, the audio output buffer 28, and the decode information storage unit 32 of FIG. 1. , 34 are different from each other. The video decoder 22 and the audio decoder 24 are the same as those described with reference to FIG.

  The video decoder 22 and the audio decoder 24 output the error reproduction data size or the error reproduction time as decode information to the decode information storage units 332 and 334, respectively. Each of the decode information storage units 332 and 334 stores the received decode information and outputs it to the output control unit 342. The output control unit 342 is required for reproduction between the decoded video data and the decoded audio data based on the decode information received from the decode information storage unit 332 and the decode information received from the decode information storage unit 334. Find the time difference. Here, when the decoding information is the error reproduction data size, the difference is obtained after dividing each error reproduction data size by the respective bit rates of video and audio, and when the decoding information is the error reproduction time, Find the difference in input error playback time.

  When it is detected that the obtained value is larger than the predetermined value, the output control unit 342 performs a flash process. That is, the output control unit 342 outputs a reset signal to the input stream buffer 311, the video stream buffer 312, the audio stream buffer 314, the video output buffer 326, the audio output buffer 328, and the decode information storage units 332 and 334, Clear the buffer etc. so that the data already obtained is not output. Other points are the same as those of the stream decoding apparatus of FIG.

  Thereafter, a new portion of the bit stream STRM to be processed next is input to the input stream buffer 311 and decoding is started. For this reason, even if there is a time difference between the video and the audio, it is not accumulated, and the video and the audio can be output again in synchronization.

  It is also possible to use the normal reproduction data size or the normal reproduction time as the decode information.

(Third embodiment)
FIG. 6 is a block diagram showing a configuration example of a stream decoding apparatus according to the third embodiment of the present invention. The stream decoding device in FIG. 6 includes a video decoder 422, an audio decoder 424, and an output control unit 442 in place of the video decoder 22, the audio decoder 24, and the output control unit 42 in the stream decoding device in FIG. In addition, reference information storage units 36 and 38 are further provided. The other components are the same as those described with reference to FIG. 1, and thus the same reference numerals are assigned and description thereof is omitted.

  The video decoder 422 and the audio decoder 424 calculate in advance decoding information expected to be obtained when normal decoding can be performed without error from the header information of the bitstream STRM, and use the reference information storage unit as reference information. 36 and 38, respectively. The decode information here is a normal reproduction data size, but may be a normal reproduction time.

  The video decoder 422 reads and decodes the video stream from the video stream buffer 12 and outputs the result to the video output buffer 26. In addition, the video decoder 422 outputs the decode information obtained at the time of decoding to the decode information storage unit 32.

  The audio decoder 424 reads and decodes the audio stream from the audio stream buffer 14 and outputs the result to the audio output buffer 28. Also, the audio decoder 424 outputs the decode information obtained at the time of decoding to the decode information storage unit 34.

  The output control unit 442 obtains a difference between the decode information stored in the decode information storage unit 32 and the reference information stored in the reference information storage unit 36, and controls the video output buffer 26 according to the obtained value. For example, when the decode information has a normal reproduction data size and the value of the decode information stored in the decode information storage unit 32 is smaller than the value of the reference information, it is considered that there is a portion that is not normally reproduced in the video. Therefore, the output control unit 442 divides the difference between the two by the video bit rate, and delays the output of the video data VOUT by the obtained time.

  Similarly, for the audio, the output control unit 442 obtains the difference between the decode information stored in the decode information storage unit 34 and the reference information stored in the reference information storage unit 38, and outputs the audio according to the obtained value. The buffer 28 is controlled.

  As described above, according to the stream decoding apparatus of FIG. 6, synchronization between video and audio is performed by controlling the output of video and audio on the basis of the case where the bit stream can be normally decoded. Can keep.

(Modification of the third embodiment)
FIG. 7 is a block diagram showing a configuration example of a stream decoding apparatus according to a modification of the third embodiment of the present invention. The stream decoding device in FIG. 7 includes the reference information storage unit 536 in place of the reference information storage units 36 and 38 in the stream decoding device in FIG. 6, and instead of the video decoder 422, the audio decoder 424, and the output control unit 442. , A video decoder 522, an audio decoder 524, and an output control unit 542. Other components are the same as those described with reference to FIG.

  The video decoder 522 calculates in advance from the header information of the bitstream STRM the decoding information that is expected to be obtained when it can be normally decoded without error, and stores it in the reference information storage unit 536 as reference information. . The decode information is assumed to be a normal reproduction time here as an example, but may be an error reproduction time. Since the reference information has the same value for audio as that for video, the audio decoder 524 may calculate the reference information and store it in the reference information storage unit 536 instead of the video decoder 522.

  From the header information of the bitstream STRM, the video decoder 522 obtains the video bit rate, and the audio decoder 524 obtains the audio bit rate. The video decoder 522 reads the video stream from the video stream buffer 12, decodes it, and outputs the result to the video output buffer 26. In addition, the video decoder 522 outputs the decode information obtained at the time of decoding to the decode information storage unit 32.

  The audio decoder 524 reads and decodes the audio stream from the audio stream buffer 14 and outputs the result to the audio output buffer 28. Also, the audio decoder 524 outputs the decode information obtained at the time of decoding to the decode information storage unit 34.

  Here, the video decoder 522 obtains the normal reproduction data size of the input video stream, divides this by the video bit rate, and outputs the obtained value (normal video reproduction time) as decoding information. The audio decoder 524 obtains the normal reproduction data size of the input audio stream, divides this by the audio bit rate, and outputs the obtained value (audio normal reproduction time) as decode information.

  The reference information storage unit 536 receives and stores the decoding information from the video decoder 522 or the audio decoder 524, and outputs this to the output control unit 542.

  The output control unit 542 obtains a difference between the decode information stored in the decode information storage unit 32 and the reference information stored in the reference information storage unit 536, and controls the video output buffer 26 according to the obtained value. For example, if the decode information is a normal reproduction time and the value of the decode information stored in the decode information storage unit 32 is smaller than the value of the reference information, it is considered that there was a part that was not normally reproduced in the video. The output control unit 542 delays the output of the video data VOUT by the difference between the two.

  Similarly, for the audio, the output control unit 542 obtains a difference between the decode information stored in the decode information storage unit 34 and the reference information stored in the reference information storage unit 536, and outputs the audio according to the obtained value. The buffer 28 is controlled.

(Fourth embodiment)
FIG. 8 is a block diagram showing a configuration example of a stream decoding apparatus according to the fourth embodiment of the present invention. 8 includes an input stream buffer 311, a video stream buffer 312, an audio stream buffer 314, a video decoder 422, an audio decoder 424, a video output buffer 326, an audio output buffer 328, and decoding information. Storage units 332 and 334, reference information storage units 636 and 638, and an output control unit 642 are provided.

  The input stream buffer 311, video stream buffer 312, audio stream buffer 314, video output buffer 326, audio output buffer 328, and decode information storage units 332 and 334 are the same as those described with reference to FIG. The video decoder 422 and the audio decoder 424 are the same as those described with reference to FIG. The reference information storage units 636 and 638 are different from the reference information storage units 36 and 38 of FIG. 6 in that the reference information storage units 636 and 638 are cleared according to a reset signal output from the output control unit 642.

  The video decoder 422 and the audio decoder 424 output the normal reproduction data size or the normal reproduction time as decode information to the decode information storage units 332 and 334, respectively. Each of the decode information storage units 332 and 334 stores the received decode information and outputs it to the output control unit 642.

  The output control unit 642 obtains a difference between the decode information received from the decode information storage unit 332 and the reference information received from the reference information storage unit 636, and detects that the obtained value is larger than a predetermined value. Performs the flash process. That is, the output control unit 642 includes the input stream buffer 311, the video stream buffer 312, the audio stream buffer 314, the video output buffer 326, the audio output buffer 328, the decode information storage units 332 and 334, and the reference information storage units 636 and 638. A reset signal is output to clear these buffers and the like. Other points are the same as those of the stream decoding apparatus of FIG.

  Further, the output control unit 642 obtains a difference between the decode information received from the decode information storage unit 334 and the reference information received from the reference information storage unit 638, and detects that the obtained value is larger than a predetermined value. In this case, the flash process is performed in the same manner.

  Thereafter, a new portion of the bit stream STRM to be processed next is input to the input stream buffer 311 and decoding is started, so that the video and audio can be synchronized again and output.

  In the above embodiment, the bit stream STRM has been described as a bit stream for transmitting an AVI format file. However, the bit stream has an audio stream and a video stream, and at least one of them does not include synchronization information. A stream can be described similarly.

  As described above, since the present invention can synchronize audio and video without using synchronization information, it is useful as a stream decoding apparatus.

It is a block diagram which shows the structural example of the stream decoding apparatus which concerns on the 1st Embodiment of this invention. (A) is explanatory drawing which shows the example of the audio stream input into the video stream input into a video decoder, and an audio decoder in FIG. (B) is explanatory drawing which shows the example of the video data and audio data which were obtained by decoding the bit stream of (a). FIG. 3C is an explanatory diagram illustrating an example of the video data VOUT output from the video output buffer and the audio data AOUT output from the audio output buffer in FIG. 1. In FIG. 1, it is explanatory drawing which shows the other example of the video data VOUT output from a video output buffer, and the audio data AOUT output from an audio output buffer. It is a block diagram which shows the structural example of the stream decoding apparatus which concerns on the modification of the 1st Embodiment of this invention. It is a block diagram which shows the structural example of the stream decoding apparatus which concerns on the 2nd Embodiment of this invention. It is a block diagram which shows the structural example of the stream decoding apparatus which concerns on the 3rd Embodiment of this invention. It is a block diagram which shows the structural example of the stream decoding apparatus which concerns on the modification of the 3rd Embodiment of this invention. It is a block diagram which shows the structural example of the stream decoding apparatus which concerns on the 4th Embodiment of this invention. It is a block diagram which shows the structural example of the conventional stream decoding apparatus. FIG. 9A is an explanatory diagram illustrating an example of a video stream input to the video decoder and an audio stream input to the audio decoder in FIG. 9. (B) is explanatory drawing which shows the example of the video data and audio data which were obtained by decoding the bit stream of (a).

Explanation of symbols

11, 311 Input stream buffer 12, 312 Video stream buffer 14, 314 Audio stream buffer 22, 222, 422, 522 Video decoder 24, 224, 424, 524 Audio decoder 26, 326 Video output buffer 28, 328 Audio output buffer 32, 34, 232, 332, 334 Decode information storage unit 36, 38, 536, 636, 638 Reference information storage unit 42, 242, 342, 442, 542, 642 Output control unit

Claims (23)

A stream decoding apparatus that decodes a bit stream that includes an audio stream and a video stream, and at least one of which does not include synchronization information,
A video decoder that decodes the video stream, and outputs first decoding information according to a size of a part that can be normally decoded or a part that cannot be decoded normally, and decoded video data;
An audio decoder that decodes the audio stream, and outputs second decoding information corresponding to a size of a part that has been successfully decoded or a part that has not been correctly decoded, and decoded audio data;
A video output buffer for storing and outputting the decoded video data;
An audio output buffer for storing and outputting the decoded audio data;
An output control unit for controlling the video output buffer and the audio output buffer so that the decoded video data and the decoded audio data are output in synchronization according to the first and second decoding information. A stream decoding apparatus. The stream decoding device according to claim 1,
The first decoding information is:
The size of the portion of the video stream that could not be successfully decoded;
The second decoding information is:
A stream decoding apparatus characterized in that it is the size of a portion of the audio stream that could not be normally decoded. The stream decoding device according to claim 1,
The first decoding information is:
A time required to reproduce a portion of the video stream that could not be normally decoded;
The second decoding information is:
A stream decoding apparatus characterized by a time required for reproducing a portion of the audio stream that could not be normally decoded. The stream decoding device according to claim 3,
The output control unit
A stream decoding apparatus, wherein the video output buffer and the audio output buffer are controlled based on a difference between the first decoding information and the second decoding information. The stream decoding device according to claim 1,
The first decoding information is:
A time required to reproduce a portion of the video stream that has been successfully decoded;
The second decoding information is:
A stream decoding apparatus characterized by a time required for reproducing a portion of the audio stream that has been successfully decoded. The stream decoding device according to claim 5,
The output control unit
A stream decoding apparatus, wherein the video output buffer and the audio output buffer are controlled based on a difference between the first decoding information and the second decoding information. The stream decoding device according to claim 1,
The output control unit
The video output buffer and the audio output buffer are controlled so as to delay the output of the decoded video data and the decoded audio data that has a portion that cannot be normally decoded during decoding. A stream decoding apparatus characterized by being a device. The stream decoding device according to claim 1,
The output control unit
The video output buffer and the audio output buffer are controlled so as to speed up the output of the decoded video data and the decoded audio data in which there is no portion that cannot be normally decoded during decoding. A stream decoding apparatus characterized by being a device. The stream decoding device according to claim 1,
The output control unit
The video output buffer and the audio output buffer are controlled so as to temporarily stop the progress of the output of the decoded video data and the decoded audio data. A stream decoding device. The stream decoding device according to claim 1,
The output control unit
A stream decoding apparatus for controlling the video output buffer and the audio output buffer so as to skip a delayed one of the decoded video data and the decoded audio data . The stream decoding device according to claim 1,
The output control unit
A stream characterized by controlling the video output buffer and the audio output buffer so that the advanced one of the decoded video data and the decoded audio data is reproduced at a low speed. Decoding device. The stream decoding device according to claim 1,
The output control unit
A stream for controlling the video output buffer and the audio output buffer so that the delayed one of the decoded video data and the decoded audio data is reproduced at high speed. Decoding device. The stream decoding device according to claim 1,
A stream buffer for storing the bit stream and outputting the bit stream to the video decoder and the audio decoder;
The output control unit
Based on the first and second decoding information, when it is detected that the time difference required for reproduction between the decoded video data and the decoded audio data is larger than a predetermined value, A stream decoding apparatus for clearing a stream buffer, the video output buffer, and the audio output buffer. The stream decoding device according to claim 1,
A reference information storage unit that stores and outputs information to be obtained as the first or second decoding information as reference information when there is no portion that cannot be normally decoded;
The output control unit
The video output buffer is controlled according to the reference information and the first decode information, and the audio output buffer is controlled according to the reference information and the second decode information. A characteristic stream decoding apparatus. The stream decoding device according to claim 14, wherein
The reference information is
A stream decoding apparatus characterized by being information representing a data size. The stream decoding device according to claim 14, wherein
The reference information is
A stream decoding apparatus characterized in that the stream decoding apparatus is information indicating a time required for data reproduction. The stream decoding device according to claim 14, wherein
The first decoding information is:
The size of the portion of the video stream that was successfully decoded;
The second decoding information is:
A stream decoding apparatus having a size of a portion of the audio stream that has been successfully decoded. The stream decoding device according to claim 14, wherein
The first decoding information is:
A time required to reproduce a portion of the video stream that has been successfully decoded;
The second decoding information is:
A stream decoding apparatus characterized by a time required for reproducing a portion of the audio stream that has been successfully decoded. The stream decoding device according to claim 14, wherein
The output control unit
The video output buffer and the audio output buffer are controlled so as to temporarily stop the progress of the output of the decoded video data and the decoded audio data. A stream decoding device. The stream decoding device according to claim 14, wherein
The output control unit
A stream decoding apparatus for controlling the video output buffer and the audio output buffer so as to skip a delayed one of the decoded video data and the decoded audio data . The stream decoding device according to claim 14, wherein
The output control unit
A stream characterized by controlling the video output buffer and the audio output buffer so that the advanced one of the decoded video data and the decoded audio data is reproduced at a low speed. Decoding device. The stream decoding device according to claim 14, wherein
The output control unit
A stream for controlling the video output buffer and the audio output buffer so that the delayed one of the decoded video data and the decoded audio data is reproduced at high speed. Decoding device. The stream decoding device according to claim 14, wherein
A stream buffer for storing the bitstream and outputting the bitstream to the video decoder and the audio decoder;
The output control unit
When it is detected that the difference between the reference information and the first or second decoding information is larger than a predetermined value, the stream buffer, the video output buffer, and the audio output buffer are cleared. A stream decoding apparatus characterized by being a device.

Images