JP2500579B2 - Video and audio synchronization system - Google Patents

Description

Detailed Description of the Invention

[0001]

[Industrial field of application] With recent advances in natural image and voice coding technology, large capacity recording media, and high-speed data communication lines, high-speed conversion of natural image data digitally encoded at high compression rates has been achieved. It has become possible to decode them, to continuously reproduce and display them, and to handle them as moving images. This is not limited to simple continuous display of images, but it is expected that it will be used as "telephone with images" or "e-mail with images" accompanied by sound, and will also be widely used in database display technology in the future.

The present invention relates to a video / audio synchronization system,
In particular, the present invention relates to a video / audio synchronization system that realizes video-audio synchronization, that is, simultaneity at the time of reproducing a moving image that handles digitally encoded video and audio data, regardless of the operating speed.

[0003]

2. Description of the Related Art In a conventional video / audio synchronization system, video encoding and audio encoding are performed by independent devices, and the video and audio are associated with each other according to a certain format, transmitted or stored, and then reproduced again. After separating
It was played back with individual video and audio decoders. For this reason, as a method for realizing simultaneity between digitally encoded video and audio data, a reproduction format created by predicting a delay amount between video and audio in each reproduction system in advance is used. A method of preparing and combining video and audio data so that they are simultaneously reproduced, or a method of providing an “output delay circuit” on the reproduction system side and adjusting the delay amount for each moving image of interest has been used.
For example, in the "video / audio editing system" described in Japanese Patent Laid-Open No. 3-150552, as an example of the video / audio editing system, a synchronization method is mainly used for digitizing and editing audio on a magnetic tape for a video film such as a movie. However, after cutting the audio data according to the address information of the magnetic film,
It was designed to be reproduced by using an image film and an editing magnetic film capable of running in synchronization with perforation. That is, the system synchronism is realized by using hardware, and the function is exhibited by performing sufficient adjustment in advance. In addition, the editing means described in the above publication corresponds to "when the dialogue is finished earlier than the screen" that occurs during after recording of the dialogue, "only the length without changing the tone of the voice" There is also a method to check the output video and audio with human eyes and ears and make corrections by adding one or more operations. Was.

[0004]

In the conventional video / audio synchronization system described above, the amount of data generated due to the nature of the target video, that is, the complexity of the screen, the large amount of movement, etc., is not constant and is encoded. Since it varies from unit to unit, the time required for video decoding and playback cannot be uniquely determined, so give up synchronizing from the beginning and predict the delay amount as an average value to make all data the same. After checking the amount of deviation between the video and audio that was actually played, such as when editing a movie or editing a movie, re-estimate the delay amount between the video and audio again, and combine them. Since the simultaneity was obtained by such a method, the amount of delay between video and audio was corrected in real time by a method according to the situation, and it was not possible to strictly maintain synchronization between video and audio during playback, and continuous audio Play breaks and tones Failure to sound reproduction there is a problem that arise, such as changes.

An object of the present invention is to provide a video / audio synchronization system capable of correcting the amount of deviation occurring during decoding and reproduction of digitally encoded video and audio data in real time by a method according to the situation. Especially.

[0006]

A video / audio synchronizing system of the present invention is a video / audio synchronous reproducing system for synchronously decoding and reproducing coded data of a moving image including digitally coded video and audio. A coded data storage device for storing coded data of digitally coded video and audio, a coded video decoding / playback device for decoding and playing back the coded data of the video for each arbitrarily defined data amount, and the audio data A coded audio decoding / reproducing apparatus for decoding and reproducing a data amount arbitrarily given from the coded data; a coded video decoding / reproducing apparatus for reading coded data of video and audio from the coded data storage device; The transfer control means for transferring to the encoded audio decoding / reproducing device and the time measuring device for measuring time are referred to, and the transfer control means and the encoded video decoding / reproducing device are referred to. An apparatus and a decoding / reproducing apparatus activation control means for controlling activation of the encoding / decoding / decoding / reproducing apparatus, wherein the decoding / reproducing apparatus activation control means terminates one of the video and audio decoding / reproducing processings. By monitoring the time until the end of both decoding and reproduction processing measured by the time measuring device which starts measurement as an opportunity, and detects the difference in speed between the decoding and reproduction processing of video and the decoding and reproduction processing of audio, Notifying the data transfer means of a change in the data transfer timing of the encoded data of the video, and changing the start timing of the encoded video decoding / reproducing device,
The configuration is such that the display time of the video after the decoding / reproduction processing is changed.

In the video / audio synchronization system of the present invention, the time measuring device starts measurement upon the end of one of the video and audio decoding / reproducing processes, and when the end of both decoding / reproducing processes is detected, decoding is performed. The decoding / reproducing apparatus starting means has a function of interrupting the reproducing apparatus starting means, and the decoding / reproducing apparatus starting means changes the start timing of the coded video decoding / reproducing apparatus in response to the interruption of the time measuring apparatus, and It may have a function of changing the display time.

In the video / audio synchronization system of the present invention, the coded video decoding / reproducing apparatus and the coded audio decoding / reproducing apparatus may be realized as software.

[0009]

[Operation] Assuming that the reproduction time axis of the audio data is the conceptual operation time of the entire system, the actual image frame displayed at the audio reproduction processing time for the unit data amount and the display at this time by calculation The continuous video and audio are synchronized by measuring the display time difference from the predicted ideal frame and correcting the display timing of the actual frame from the result. (The image frame refers to one frame in handling a moving image, and in this case, it is called an encoded image frame on the premise that encoding processing for replacing image data with numerical data has been performed. The unit is called a frame, and the actual frame refers to an actual display frame at an arbitrary specified time of the moving image that is reproduced and displayed while depending on the operation speed of the video decoding / reproducing apparatus and the property of the image. An ideal frame is an image frame that should be displayed at any specified time when the playback processing speed of audio data is constant and the speed does not change within a unit time and the time required for video composite playback is sufficiently small. Point)

Embodiments of the present invention will now be described with reference to the drawings.

FIG. 1 is a block diagram of a first embodiment of the present invention.

The video / audio synchronization system according to the first embodiment of the present invention is a coded data storage device 1 for storing coded data of video and audio that have been digitally coded in advance, and a video and audio data from the coded data storage device 1. Receiving the encoded data of the audio and referring to the time measuring device 2 for measuring the time, the mutual output timing of the image and the audio is controlled, and the encoded data of the image is set to the data amount of one frame, and the encoding of the audio is performed. The synchronization control device 3 outputs the encoded data of video as a data amount corresponding to one frame, and the synchronization control device 3.
A coded video decoding / reproducing device 4 which decodes and reproduces the encoded data of the video output for each frame, and a code which decodes and reproduces the encoded data of the audio of a data amount arbitrarily given from the synchronization control device 3. Encoded audio decoding / reproducing apparatus 5, encoded video decoding / reproducing apparatus 4 and encoded audio decoding / reproducing apparatus 5
The output device 6 which receives the reproduction output of the above and converts it into a video and audio and visualizes it.

Further, the synchronization control device 3 includes transfer control means 7 for reading out the encoded data of video and audio from the encoded data storage device 1 and transferring them to the encoded video decoding / reproducing device 4 and the encoded audio decoding / reproducing device 5. , A decoding / reproducing device activation control means 8 for controlling the coded video decoding / reproducing device 4, the encoded audio decoding / reproducing device 5, and the transfer control means 7 with reference to the time measuring device 2, and an interrupt time interval of the time measuring device 2. The time measuring device 2 includes an interrupting device 10 for interrupting the synchronous control device 3 and a timer 1 for measuring an arbitrary time.
1 and software including.

In the present invention, there is no limitation on the digital encoding system of video and audio signals handled by the encoded video decoding / reproducing apparatus 4 and the encoded audio decoding / reproducing apparatus 5, and the function is satisfied. If a sufficient buffer memory is provided (for example, a hard disk is supported), and the operation start command is received with the encoded data for decoding and playback stored in this buffer memory, the data is controlled by the internal control of the device. Decrypt and play back. In this processing, one frame for video and one audio data for audio are reproduced at one time.

As in the previous section, the recording format and the combining method of the video and audio data in the encoded data storage device 1 are not specified, and the respective data can be read out sufficiently quickly, for example, time code and It is sufficient that the simultaneity can be determined by a format in which audio data corresponding to before and after one frame of video data is arranged.

Next, the operation of the first embodiment will be described.

FIG. 2 is a flow chart for explaining the operation of the first embodiment. The names and reference numerals of the respective parts used in the description of the operation are the same as those in FIG.

When the operation is started, the decoding / reproduction device activation control means 8 in the synchronous reproduction device 3 makes a step (hereinafter referred to as S) in the data buffer in the encoded / voice decoding / reproduction device 5 via the transfer control means 7. The encoded voice data read from the encoded data storage device 1 in 1 is input for an arbitrary unit time t seconds. At this time, the reproduction of the video is associated in advance so that n frames are reproduced within t seconds. Next, in S2, the decoding / reproduction device activation control means 8 instructs the start of sound generation for a unit time of t seconds and the start of continuous video display, and in S3, through the transfer control means 7, the encoded video for one frame. The data is input to the coded video decoding / reproducing device 4 to activate the decoding / reproducing function. The input coded data is decoded by each decoding / reproducing device under internal control, and the video and audio converted into analog signals are sent to the output device 6 for visualization.
By the operation up to this point, the display of the video of the first frame and the utterance of voice are started.

Next, the decoding / reproducing device activation control means 8 inputs the encoded video data of the second frame into the encoded video decoding / reproducing device 4 via the transfer control means 7, and activates the decoding / reproducing function. Immediately thereafter, the coded video decoding / reproducing device 4 starts the reproducing process, and the output device 6 displays the second frame in accordance with its own operation clock. If this operation is repeated n times, moving images of n frames are reproduced and displayed, and n + 1
If it is repeated more than once, more moving images than n frames will be reproduced. By the way, at present, the video data to be reproduced is for n frames, and if the processing speed of the video and audio reproduction functions is uniform and is performed regularly, the relationship between the video frames and audio in FIG. As shown in the explanatory diagram (No. 1) to be described, the display of the video of the nth frame ends exactly t seconds after the reproduction starts,
It can be said that it is the moment when the image of the (n + 1) th frame is displayed. In addition, the n + 1 frame of the image is named “measurement target image frame”. However, it is rare that all video and audio ends at the same time without any error. Normally, one of them ends earlier and the other ends later. Therefore, this is the difference between the end time of the video and the audio, that is, the audio reproduction end time and n + of the video.
In order to measure the time difference from the display time of the first frame, first, in S4, it is determined whether or not the unit time t seconds has elapsed and the reproduction of the audio data has ended, and if it has not ended, the decoding for each frame is performed. Each time the reproduction operation is performed, it is determined in S5 whether or not this video is a video frame to be measured, and if it is not the measurement target, the process returns to S3.

When it is determined in S4 that the unit time t seconds has elapsed and the reproduction of the audio data is completed, the timer 11 is set from the setting means 9 in S6 and the time measurement is started. Subsequently, in S7, the encoded video data is input to the encoded video decoding / reproducing device 4,
The decoding / reproducing function is activated, and in S8, it is discriminated whether or not this video is a video frame to be measured every time the decoding / reproducing operation is performed for each frame, and the process is repeated until it is detected that the video is a measurement target. When it is detected that it is a video frame to be measured,
The time measurement ends in S9.

If it is determined in S5 that the video is the video frame to be measured, even though the reproduction of the audio data is not completed, the timer 11 is set by the setting means 9 in S10. Start time measurement. Then S
When it is determined in 11 that the unit time t seconds has elapsed and the reproduction of the audio data has ended, the time measurement is ended in S9, similarly to S8.

For measuring the elapsed time in the time measuring device 2, the interrupt means 10 connected to the timer 11 is used. An interrupt is issued to the synchronous control device 3 at every interrupt time interval i set by the synchronous control device 3,
Counts the number of interrupts to measure elapsed time. However, the time measuring method is not limited to this method.

When the time measurement is completed in S9, next S12
Then, the elapsed time at this time is set as the delay amount d of the video and audio, and the following process is selected according to the magnitude of the delay amount d.

As described above, ideally, the frame display after t seconds is the (n + 1) th frame, or the delay amount d is within the time required to display one frame (within the vertical blanking period of the output device 10). That is, if the delay is d≈0), it is considered that the simultaneity between the video and the audio is maintained, and the synchronization processing is ended in S13. If the delay amount d is within one frame display period (d ≦ t / n seconds), the synchronization correction operation is performed by extending or shortening the display time of the (n + 1) th frame in S14, and the synchronization processing is performed in S13. finish. More specifically, if the audio is delayed, the display timing of the (n + 2) th frame is delayed to extend the display time of the (n + 1) th frame and align the ends. Further, when the video is delayed, the display timing of the (n + 2) th frame is advanced to shorten the display time of the (n + 1) th frame so that the ends are aligned.

If it is determined in S12 that the delay amount d is larger than the one-frame display period (d> t / n seconds), in S15, frame thinning for canceling frame display is performed, or the same frame is held and synchronization correction operation is performed. , S13, the synchronization process ends. More specifically, if the voice is delayed, by waiting for (t / n seconds) or more until the timing at which the display of n + 2 frames can be synchronized becomes n + 1.
If the image display is delayed by extending the frame display time of the nth frame, the display of the (n + 2) th frame is stopped, and the adjustment for exceeding the one frame display period (t / n seconds) is performed. That is,
In this case, the display order is [... n-1, n, n + 1, n + 3 ...]. Also, if this is still insufficient, n + as needed
Cancel the frame display of 3, 4, 5 ...

Next, a description will be given by showing concrete numerical values.

Here, the video / audio synchronization system according to the present invention is a general-purpose personal computer (hereinafter referred to as PC).
In order to realize the above, the encoded video decoding / reproducing apparatus 4 of FIG.
And the encoded audio decoding / reproducing device 5 are realized by an expansion board compatible with a PC, and an output device 6 (the time required for displaying one frame is about 33 milliseconds) is prepared.
Further, in order to implement the coded data storage device 1 and the synchronization control device 3 as programs, a RAM (random access memory) is sufficiently provided. However, as described above, the buffer memories required for the respective decoding / reproducing devices 4 and 5 are assumed to be included in the respective decoding / reproducing devices. The time required for data transfer is sufficiently fast.

As a precondition, it is assumed that a moving image [video + audio] of 2 seconds or more is reproduced. Also, for convenience of explanation,
The encoded voice data is converted into [4-bit stereo 8.27 kHz.
z ADPCM (adaptive differential pulse code modulation) data], and this audio data for 1 second (8270 bytes)
Is sent as a unit. When the reproduction of the audio data for one second is completed, the audio data for the second second is continuously sent and the reproduction is continued so that the reproduction of the audio is not interrupted even when the moving image is reproduced for two seconds or more.

As the operation, first, the decoding / reproducing of the audio is started, and at the same time, the data transfer of the first frame and the decoding / reproducing operation are started, the decoding / reproducing operation of the video is repeated, and the decoding / reproducing frames are counted. If the number of frames becomes the number of frames to be measured (pattern A when the video playback speed is fast and the audio playback speed is slow), or when the audio playback is completed (when the audio playback speed is fast and the video playback speed is slow, Pattern B), and the operation of the timer 11 (the interrupt interval of the timer 11 is about 10 milliseconds, and a value that is easy to calculate is used, so 10 milliseconds is used here). In the case of pattern A, when the voice reproduction for one second is completed, or in the case of pattern B, when the reproduction order of the measurement target frame is reached, the timer 11 is stopped and the number of timer interrupts up to that point is calculated to determine the delay amount. . Now, the number of video frames scheduled to be displayed per second is 15 (hereinafter t / n
≈66 msec, so 66 msec). Therefore, description will be made by classifying by the value of the delay amount d. Delay amount d
But. . . (1) The video is 10 ms slower than the audio (shown in the explanatory diagram (part 2) explaining the relationship between the video frame and the audio in FIG. 4) (2) The video is 50 ms slower than the audio (the video in FIG. 5 An explanatory diagram (part 3) for explaining the relationship between frames and audio is shown. (3) An image is 70 milliseconds slower than the audio (see the explanatory diagram (part 4) for explaining the relationship between video frames and audio in FIG. 6). (4) The audio is slower than the video by 10 milliseconds (shown in FIG. 7 which is an explanatory view (No. 5) explaining the relationship between the video frame and the audio). (5) The audio is slower than the video by 50 milliseconds (FIG. 8). (6) An explanatory diagram (part 6) for explaining the relationship between the video frame and the sound) (6) An audio delay of 70 milliseconds from the image (an explanatory view (part 7) for explaining the relationship between the video frame and the sound in FIG. 9) )) The synchronization correction operation will be described separately for the above cases. However, for the purpose of clarifying the state of FIGS. 5, 6, 8 and 9, the chart format is used, the chart (a) is a diagram showing the delay state between video and audio, and the chart (b) is The figure shows the synchronization correction method. (1) When the image is slower than the audio by 10 milliseconds (FIG. 4) This corresponds to the case where ┃d┃≈0 is determined in S12 of FIG. 2, and the synchronization correction work is not performed. (2) When the video is slower than the audio by 50 milliseconds (FIG. 5) This is the case where the video is delayed when it is determined that 0 <┃d┃ ≦ t / n in S12 of FIG. The display time of (n + 1) is shortened. In this case, the delay amount d
Is longer than the time (33 milliseconds) for displaying one frame of the output device and shorter than the one-frame display period (66 milliseconds). Therefore, the display time of n + 1 frames is reduced by 33 milliseconds. That is, the synchronization correction operation is performed by advancing the display timing of the (n + 2) th frame by 33 milliseconds. (3) When the image is slower than the audio by 70 milliseconds (FIG. 6) This is the case where the image is delayed when it is determined that t / n <┃d┃ in S12 of FIG. 2, and the 17th frame (n +
2) Cancel the display. Since the delay amount d at this time is longer than the one frame display period (66 milliseconds), the 17th frame (n + 2) is skipped and the 18th frame (n +
The data of 3) is transferred to the video decoding / reproducing device 4. By this operation, the actual playback frame is [1, ..., 15, 16, 18,
..], and the synchronization correction operation is performed. (4) When the audio is slower than the video by 10 milliseconds (FIG. 7) This corresponds to the case where ┃d┃≈0 is determined in S12 of FIG. 2, and the synchronization correction is not performed. (5) When the audio is slower than the video by 50 milliseconds (FIG. 8) This is the case where the audio is delayed when it is determined that 0 <┃d┃ ≦ t / n in S12 of FIG. 2, and the 16th frame The time (33 milliseconds) for displaying (n + 1) is extended, the timing of the decoding / reproducing process of the 17th frame (n + 2) is delayed, and the synchronization correction operation is performed. The difference from (2) is that the decoding and reproduction of the video is delayed with respect to the audio in the previous example, while the audio is delayed with respect to the video. (6) When the sound is slower than the video by 70 milliseconds (FIG. 9) This corresponds to the case where the sound is delayed by determining t / n <┃d┃ in S12 of FIG.
The display period of the display of the frame is extended as in (5). The actual process is similar to (5), but holds the display for a longer time. Since the display period of one frame is 66 milliseconds, the correction of 70 milliseconds is about one frame.

As described above, according to the measured delay amount,
The synchronization correction operation is performed by classifying into 6 states. 2 here
Since the video [video + audio] was played back for more than a second, the number of video playback frames is continuously counted, and the synchronization correction operation as described above is repeated every arbitrary unit time t (here, 1 second). .

FIG. 10 is a block diagram of the second embodiment of the present invention.

The video / audio synchronization system of the second embodiment of the present invention has the same configuration as the video / audio synchronization system of the first embodiment, and stores video and audio encoded data that has been digitally encoded in advance. An encoded data storage device 1;
The encoded data of the video and audio is received from the encoded data storage device 1, the mutual output timing of the video and the audio is controlled by referring to the time measuring device 2 for measuring the time, and the encoded data of the video corresponds to one frame. And the encoded data of audio is output as encoded data of video as a data amount corresponding to one frame, and the encoded data of video output by the synchronization control device 3 for one frame. A video / audio decoding / reproducing device 12 which decodes and reproduces each of them and simultaneously decodes and reproduces encoded data of audio of a data amount arbitrarily given from the synchronization control device 3, and a reproduction output of the video / audio decoding / reproducing device 12. And an output device 6 for converting into voice and visualizing it.

Further, the video / audio decoding / reproducing device 12 decodes the coded data of the video output from the synchronization control device 3 for each frame and reproduces it.
And a coded voice decoding / reproducing means 14 for decoding and reproducing coded data of voice having a data amount arbitrarily given from the synchronization control device 3, and the synchronization control device 3 is the first embodiment. In the same manner as the above, refer to the time measuring device 2 and the transfer control means 7 for reading the encoded data of the video and audio from the encoded data storage device 1 and transferring them to the encoded video decoding / reproducing device 4 and the encoded audio decoding / reproducing device 5. A decoding / reproducing device activation control means 8 for controlling the encoded video decoding / reproducing device 4, the encoded audio decoding / reproducing device 5, and the transfer control means 7;
The time measuring device 2 is constituted by software including a setting means 9 for setting an interrupt time interval i of the time measuring device 2.
Also, like the first embodiment, it is composed of software including interrupt means 10 for interrupting the synchronization control device 3 and a timer 11 for measuring an arbitrary time. It should be noted that devices and means having the same functions and configurations as those in the first embodiment are designated by the same reference numerals.

Here, the encoding method and system of the video and audio signals handled by the encoded video decoding / reproducing means 13 and the encoded audio decoding / reproducing means 14 may be any method and system that can be processed by software. It may be anything. In addition, it has a sufficient buffer memory to satisfy these functions. For example, when the operation start command is received when the encoded data for decoding and reproduction is stored in the buffer memory realized by the hard disk. ,
Decode and play video and audio according to the procedure of the program. In this process, one frame is reproduced for video, and a given amount of data is reproduced for audio at once.

Regarding the video and audio data stored in the encoded data storage device 1, the recording format and combining method thereof are not limited, but the respective data can be read out sufficiently quickly, for example, time code or 1
It is assumed that the simultaneity can be determined by a format in which audio data corresponding to before and after the video data of the frame is arranged.

By setting the conditions as described above,
The difference between the first embodiment and the second embodiment is that in terms of operation, the device for decoding and reproducing video and audio is either hardware or software, and is otherwise equivalent. Therefore, the description of the operation is omitted.

[0036]

As described above, according to the present invention, a coded data storage device for storing coded data of digitally coded video and audio, and decoding of coded data of video for each arbitrarily defined data amount. A coded video decoding / playback device for playing back and playing, a coded audio decoding / playback device for decoding and playing back an arbitrary amount of data out of coded audio data, and a video and audio code from the coded data storage device. The transfer control means for reading the encoded data and transferring it to the encoded video decoding / reproducing apparatus and the encoded audio decoding / reproducing apparatus, and the time measuring apparatus for measuring the time, the transfer control means, the encoded video decoding / reproducing apparatus and the encoded audio decoding. A decoding / reproducing device activation control means for controlling activation with a reproducing device is provided, and the decoding / reproducing device activation control means terminates either one of the video / audio decoding / reproducing processing. The time until the end of both decoding / playback processing measured by the time measuring device that starts measurement is monitored by detecting the difference in speed between the video decoding / playback processing and the audio decoding / playback processing, and By notifying the transfer means of the change in the data transfer timing of the video and audio encoded data, and adjusting the video decoding and reproduction display timing based on the audio decoding and reproduction operation speed to realize the synchronization correction. Decoding of coded video and audio data The amount of deviation that occurs during playback can be corrected in real time by a method according to the situation, and the synchronization of video and audio can be strictly maintained during playback, and continuous audio This has the effect of preventing interruptions in audio reproduction such as interruptions in reproduction and changes in tone.

[Brief description of drawings]

FIG. 1 is a block diagram of a first embodiment of the present invention.

FIG. 2 is a flowchart illustrating the operation of the first embodiment.

FIG. 3 is an explanatory diagram (part 1) explaining the relationship between a video frame and audio.

FIG. 4 is an explanatory diagram (part 2) explaining the relationship between video frames and audio.

FIG. 5 is an explanatory diagram (part 3) explaining the relationship between video frames and audio.

FIG. 6 is an explanatory view (No. 4) for explaining the relationship between video frames and audio.

FIG. 7 is an explanatory diagram (Part 5) for explaining the relationship between video frames and audio.

FIG. 8 is an explanatory view (No. 6) for explaining the relationship between video frames and audio.

FIG. 9 is an explanatory diagram (No. 7) for explaining the relationship between video frames and audio.

FIG. 10 is a block diagram of a second embodiment of the present invention.

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 1 coded data storage device 2 time measuring device 3 synchronization control device 4 coded video decoding / playback device 5 coded audio decoding / playback device 6 output device 7 transfer control means 8 decoding / playback device activation control means 9 setting means 10 interrupt means 11 timer 12 video / audio decoding / reproducing apparatus 13 coded video decoding / reproducing means 14 coded audio decoding / reproducing means

Claims (3)

(57) [Claims] 1. A video / audio synchronous reproduction system for synchronously decoding and reproducing coded data of a moving image including digitally coded video and audio, wherein the digitally coded coded data of video and audio are stored in advance. A coded data storage device, a coded video decoding / playback device that decodes and plays back the coded data of the video for each data amount that is arbitrarily determined, and a data amount that is arbitrarily given from the coded data of the audio. A coded audio decoding / reproducing apparatus for decoding and reproducing data, and a transfer control means for reading coded data of video and audio from the coded data storage apparatus and transferring to the coded video decoding / reproducing apparatus and the coded audio decoding / reproducing apparatus. Controlling the activation of the transfer control means, the coded video decoding / reproducing apparatus and the coded audio decoding / reproducing apparatus with reference to a time measuring apparatus for measuring time Decoding / reproducing apparatus activation control means for performing both of the measurement by the time measuring apparatus, the decoding / reproducing apparatus activation control means starting measurement upon termination of either one of the video / audio decoding / reproducing processing. The time until the end of the decoding / reproducing process is monitored, the difference in speed between the decoding / reproducing process of the video and the decoding / reproducing process of the audio is detected, and the data transfer timing of the encoded data of the video is transmitted to the data transfer means. Change the start timing of the coded video decoding and playback device,
A video / audio synchronization system characterized in that a display time of a video after decoding and reproducing processing is changed. 2. The time measuring device starts measurement upon the end of one of the video and audio decoding / reproducing processes, and when the end of both decoding / reproducing processes is detected, interrupts the decoding / reproducing device activation means. The decoding / reproducing apparatus starting means has a function of changing the start timing of the coded video decoding / reproducing apparatus in response to the interruption of the time measuring apparatus, and changing the display time of the video after the decoding / reproducing process. The video / audio synchronization system according to claim 1, further comprising: 3. The video / audio synchronization system according to claim 1, wherein the coded video decoding / reproducing apparatus and the coded audio decoding / reproducing apparatus are realized as software.