JP2005236641A - Reproduction method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To highly precisely synchronize audio data with video data to output them. <P>SOLUTION: In the reproduction method, encoded video data and encoded audio data are reproduced and decoded, the decoded audio data are written in a DMA transfer buffer, the decoded audio data are read from the DMA transfer buffer to output them to an audio output device and to output the decoded video data to a video output device. On the basis of an output value of a synchronous timer for counting clocks of higher frequency than a system timer for reproduction processing and an output delay time obtained from a state of the audio data stored in the DMA buffer, outputs of the audio data and video data are controlled so that reproduction elapsed time of the audio data to be outputted from the audio output device and reproduction elapsed time to be outputted from the video output device synchronize with each other. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

  The present invention relates to a video and audio playback method.

Conventionally, in video data playback software that combines video data and audio data, a method for synchronously playing back video and audio is based on time information obtained from the system elapsed time using the system timer of the operation system. There is a method of synchronizing the playback time of audio and audio (for example, see Patent Document 1). Further, when audio DMA transfer to an audio device is used in audio reproduction, an audio reproduction process (or thread) writes audio data in an audio DMA transfer buffer, and audio is output by executing DMA transfer. In such a case, in the audio reproduction process (process) for writing the audio reproduction data to the DMA transfer buffer, a series of audio reproduction loop processes are performed until the DMA transfer buffer is written. It is difficult to reflect the delay time in synchronization with video playback.
JP 2001-324998 A

  The video and audio synchronous playback method using time information by the system timer as described above has a problem that video and audio cannot be synchronously played back with a smaller granularity than the system timer. For example, when the granularity of the system timer is 10 ms, the audio and video playback process does not switch in a time shorter than 10 ms, so the accuracy of synchronous playback does not become 10 ms or less. Furthermore, when data transfer by DMA to audio devices is used for audio playback, if the delay time until audio is actually output increases due to buffering of audio transfer DMA, the system timer time granularity is accurate. The video and audio playback that reflects the delay time cannot be synchronized.

  An object of the present invention is to solve such a problem and to enable audio data and video data to be output in synchronization with high accuracy.

  The present invention provides a reproduction step of reproducing encoded video data and encoded audio data according to an output of a system timer, and audio decoding for decoding the reproduced encoded audio data and writing it to a DMA transfer buffer Independent of the audio decoding step, the video decoding step for decoding the reproduced encoded video data, and the reproduction processing of the audio data by the reproduction step, and outputting the decoded audio data from the DMA transfer buffer. An audio output step for outputting to the device; a video output step for outputting the video data decoded in the video decoding step to a video output device; an output value of a synchronous timer for counting a clock having a frequency higher than the system timer; and the DMA Bag Based on the output delay time obtained from the state of the audio data stored in the memory, the playback elapsed time of the audio data output by the audio output device and the playback elapsed time of the video data output by the video output device, A synchronization control step for controlling the audio output step and the video output step so as to synchronize with each other.

  According to the present invention, audio data and video data can be synchronized and output with higher accuracy.

  FIG. 1 is a diagram showing an example of a reproduction system to which the present invention is applied. Reference numeral 101 denotes moving image data storage means, which represents means for storing moving image data in which encoded video data and audio data are combined and reading them out during moving image reproduction.

  Reference numeral 102 denotes reproduction data synchronization means. An apparatus and software for reading video data and audio data from 101 during moving image reproduction. Reference numeral 102 can extract encoded video data and encoded audio data that match an arbitrary playback elapsed time.

  Reference numeral 103 denotes audio reproduction means. A process (or thread) for decoding encoded audio data during video playback is shown. Reference numeral 105 denotes an audio output means. Output audible audio from audio data.

  In the audio reproduction, the reproduction data synchronization unit 102 reads the encoded audio data from the moving image data storage unit 101 and passes it to the audio reproduction unit 103. The audio reproduction means 103 decodes the encoded audio data and transfers it to the audio output means 105. Finally, 105 audio output means outputs moving image audio.

  Reference numeral 112 denotes that audio data decoded using DMA is transferred from 103 to 105.

  Reference numeral 104 denotes video playback means. A process (or thread) for decoding encoded video data during video playback is shown. Reference numeral 106 denotes a video output means, which is a display device such as a liquid crystal display.

  In the video reproduction, the reproduction data synchronization unit 102 reads the encoded video data from the moving image data storage unit 101 and passes it to the video reproduction unit 104. The video reproduction means 104 decodes the encoded data and transfers it to the 106 video output means. Finally, the video output means 106 outputs a moving image video.

  Reference numeral 107 denotes playback control means of the moving image playback system of the present invention, which represents a process (or thread) that issues control instructions 102, 103, and 104.

  Reference numeral 108 denotes a reproduction operation receiving means. When a user who performs video playback performs the playback operation shown in 109 (playback, pause, fast forward, rewind, etc.), the playback operation receiving unit 108 indicates what playback operation has been performed on the 107 playback control unit. Send. Reference numeral 107 identifies the type of operation, and executes control instructions for 102, 103, and 104.

  The moving image playback system operates according to the time granularity of the system timer 110. The system timer 110 is an operation system timer in which the moving image playback system operates. In the present invention, a timer 111 used exclusively for reproduction synchronization is used. This timer is called a synchronous timer. The reproduction data synchronization means 102, the audio reproduction means 103, and the video reproduction means 104 use a synchronization timer.

  Here, an embodiment of the synchronization timer will be described. The synchronous timer is created so that timer interrupt processing occurs with a finer precision than the system timer. For example, if the system timer granularity is 10 ms, the synchronization timer must be at least 1 ms granularity. An example of the synchronization timer is a timer using a hardware period timer. The number of clocks with a fixed period is set in the hardware period timer. When started, it counts the clocks synchronized with the internal clock source or the external clock source, and generates a hardware timer interrupt with the set number of clocks. A synchronization timer is created using this. The synchronous timer can read the elapsed time from the start at an arbitrary timing, and can rewrite the elapsed time from the start at an arbitrary timing.

  In video playback synchronization, if the video frame rate is 10 fps, it is ideal to display and output video frames every 100 ms from the start of playback. However, if it is 15 fps, it is 1000/15 ms, and if it is 30 fps, it is ideal to output 1000/30 ms as a cycle. For this purpose, a timer with an accuracy of at least about 0.1 ms is required. Since the system timer is used by the operation system of the moving image playback system, if the granularity is too small, timer interrupt processing increases. Therefore, it is difficult to reduce the granularity of the system timer time interval.

  Therefore, if the system timer is used for reproduction synchronization, the accuracy of synchronous reproduction is deteriorated.

  Therefore, the number of clocks of the hardware interrupt interval is calculated and set to the synchronous timer so that 1000/15 ms for 15 fps and 1000/30 ms for 30 fps become the interrupt period of the hardware period timer. The synchronous timer calculates and holds an operation elapsed time I when an interrupt occurs in the timer interrupt process. In this way, the synchronization timer holds the operation elapsed time inside.

  The operation elapsed time of the synchronous timer read from the software process (or thread) at an arbitrary timing is a time calculated from the operation elapsed time I and the number of clocks from the previous interrupt.

  Next, an embodiment of reproduction synchronization using a synchronization timer will be described. The reproduction data synchronization means 102 shown in FIG. 1 starts a synchronization timer simultaneously with the start of reproduction. In order to synchronize and play back the audio and video of the video, the elapsed time of the synchronous timer is used. Adjust the synchronization timer operation elapsed time to match the audio playback elapsed time calculated by the audio playback means, and adjust the video data playback elapsed time calculated by the video playback means to match the synchronization timer operation elapsed time. Thus, synchronized playback is possible by matching the elapsed playback time of audio and video as much as possible.

  FIG. 2 is a diagram schematically showing that the audio playback elapsed time and the video playback elapsed time are matched by using the synchronization timer. 201 represents the elapsed operation time of the synchronization timer, and 202 represents the elapsed audio reproduction time. The audio playback elapsed time is set to fall within a specified synchronization time range (204) before and after the synchronization timer operation elapsed time. When the audio playback elapsed time is out of the range of 204 during moving image playback, the synchronization timer is set or the audio playback processing is paused so that the operation elapsed time of the synchronization timer becomes the audio playback elapsed time.

  Reference numeral 203 represents a video elapsed time, and 205 represents a timing for displaying a video frame. Similarly, the video playback elapsed time is set to fall within the specified synchronization time range (204) before and after the synchronization timer operation elapsed time. By delaying the display of the next video frame when the elapsed video playback time has advanced from the range of 204 during video playback, or by skipping the display of the next video frame when it is delayed from the range of 204 The video playback is controlled so as to follow the elapsed operation time of the synchronization timer.

  In the DMA transfer of audio data as indicated by 112 in FIG. 1, the decoded audio data written in the DMA transfer buffer by the 103 audio reproducing means is transferred to the 105 audio output means. The DMA transfer buffer has a FIFO (First In First Out) structure, and is sequentially transferred from the previously written data. Therefore, there is a delay due to buffering in the DMA transfer buffer until the audio output. In general, when output data from a software process is transferred to an output device by DMA transfer, the software process cannot know the exact time of output due to a delay due to buffering in the DMA transfer buffer. In order to solve this problem, a method for taking into account the delay time generated by buffering of the audio output DMA buffer in the reproduction synchronization processing according to claim 1 will be described.

  FIG. 3 shows a playback synchronization processing sequence on the audio playback means side 103 in FIG. In S302 of FIG. 3, the decoded audio data is written into the DMA transfer buffer. Audio data is written by blocking write. In other words, if the data size to be written does not fit in the free space of the buffer, data that can be written is written first, and the remaining data is written as soon as free space is created in the buffer by DMA transfer. When the audio data has been written in S302, the process proceeds to S303.

  In S303, the time required to output the audio data written in the audio DMA buffer and waiting to be transferred is calculated. In the calculation of the required time, the calculation is performed using the size of the audio data in the transfer buffer and the hardware characteristics from 112 DMA audio data transfer to 105 audio data output.

  In S304, an audio reproduction is obtained by subtracting the reproduction time for the audio data waiting for DMA transfer calculated in S303 from the reproduction time for the total size of the encoded audio data written in the DMA transfer buffer by the audio reproduction means. Find the elapsed time.

  Next, the operation elapsed time of the synchronous timer is checked in S305.

  In S306, the operation elapsed time of the synchronization timer obtained in S305 is compared with the audio reproduction elapsed time obtained in S304. If the elapsed operation time of the synchronization timer is later than the audio reproduction elapsed time by a predetermined time range or more for reproduction synchronization determination, the process proceeds to S307, and if not, the process proceeds to S308. In S307, the operation elapsed time of the synchronization timer is changed to the audio playback elapsed time.

  In S308, if the operation elapsed time of the synchronization timer has advanced beyond the audio reproduction elapsed time beyond the specified time range for synchronization determination, the process proceeds to S309. In S309, the audio playback means is temporarily stopped by the time difference.

  By the above method, when audio data is transferred to the audio output means by DMA, synchronous reproduction can be performed in consideration of audio output delay due to buffering in the DMA buffer.

  Audio playback is written to the audio DMA transfer buffer by blocking write. At this time, writing of audio data is blocked as much as possible. That is, the audio data transfer buffer is always kept full during reproduction. For example, this is realized by the method of claim 3 or giving priority to the process (or thread) of the audio playback means over other processes (or threads).

  At this time, the audio reproduction is continuous without interruption, and the audio reproduction time can be used as a reference in the synchronous reproduction. When the transfer buffer is full, the data size waiting for transfer is constant, the delay time for data transfer is constant, and the calculation of the elapsed audio playback time can be simplified. Also, when writing is blocked, there is an advantage that the right to execute processing can be transferred to another process.

  In video playback and audio playback, the determination of synchronous playback is that each elapsed playback time is within a specified synchronization time range based on the elapsed operation time of the synchronization timer. However, in audio playback, the amount of data actually decoded is equal to the amount of data corresponding to the delay time due to buffering of DMA transfer, rather than the amount of audio data corresponding to the elapsed playback time, that is, the actual playback output time. large.

  Therefore, the larger the transfer buffer size than the audio data size corresponding to the specified synchronization time, the larger the amount of decoded data than the amount of audio data for the elapsed audio playback time, and the audio decoding process even during playback synchronization. (103 in FIG. 1) is overworked. Therefore, the size of the DMA transfer buffer for audio data can be changed to be smaller than the data size corresponding to the specified synchronization time for synchronous reproduction.

  In addition, by reducing the DMA transfer buffer size, the number of times audio data writing is blocked can be increased.

It is a figure which shows an example of a reproduction | regeneration system. It is a schematic diagram about coincidence of audio playback elapsed time and video playback elapsed time. It is a flowchart of a synchronous reproduction process.

Claims (4)

A reproduction step of reproducing the encoded video data and the encoded audio data according to the output of the system timer;
An audio decoding step of decoding the reproduced encoded audio data and writing it into a DMA transfer buffer;
A video decoding step of decoding the reproduced encoded video data;
An audio output step of reading out the decoded audio data from the DMA transfer buffer and outputting the decoded audio data to an audio output device independently of the playback processing of the audio data in the playback step;
A video output step of outputting the video data decoded by the video decoding step to a video output device;
Based on an output value of a synchronous timer that counts a clock having a frequency higher than that of the system timer and an output delay time obtained from the state of the audio data stored in the DMA buffer, the audio data output by the audio output device A playback method comprising: a synchronization control step for controlling the audio output step and the video output step so that a playback elapsed time and a playback elapsed time of video data output from the video output device are synchronized.   2. The reproduction method according to claim 1, wherein a DMA transfer buffer for the audio data is always full during reproduction of the video data.   2. The reproduction method according to claim 1, wherein the buffer size can be changed so that a delay time generated for buffering the audio data by the DMA buffer becomes smaller than a specified time range.   A storage medium storing a program for executing the method according to claim 1 by a computer.

Images