JP2012105236A - Synchronous control apparatus and synchronous control method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To realize quick video switching in a synchronous control apparatus with a simple structure.SOLUTION: A synchronous control apparatus 10 synchronously outputs a video signal and a sound signal from a superimposing signal obtained by superimposing first video data, second video data and audio data. When a display switching command from the video signal on the first video data to the video signal on the second video data is given, a demultiplexer 130 extracts the second video data corresponding to a time stamp of the displayed first video data. A synchronous control portion 200 decides output timing of the video signal and the sound signal based on the time stamp of the extracted second video data and the time stamp of the audio data.

Description

  The present invention relates to a synchronization control device and a synchronization control method, and more particularly to a synchronization control device and a synchronization control method for multimedia data.

  The MPEG (Moving Picture Expert Group) standard is widely known as an international standard for encoding video data and audio data and multiplexing the data. Information for time synchronization called PTS (Presentation Time Stamp) is added to video data and audio data recorded based on the MPEG standard. For example, in the case of a stream having a plurality of audio data corresponding to video data, it is possible to switch only audio while reproducing the video by this PTS. Similarly, in the case of a stream having a plurality of types of video data corresponding to audio data, it is possible to switch only the screen angle while reproducing the same sound by this PTS. When handling such stream data, it is necessary to synchronize video and audio.

  Patent Document 1 discloses a technique for synchronizing video and audio in stream data. FIG. 5 is a block diagram showing the configuration of the synchronized playback device 10 described in Patent Document 1. As shown in FIG. The ATAPI control unit 110 reads stream data from the medium 300 and stores the read stream data in the playback buffer 120. The demultiplexer 130 reads the stream data from the playback buffer 120 and separates it into video data, audio data, and reproduction information. The demultiplexer 130 stores the video data in the video buffer 140. The demultiplexer 130 stores the audio data in the audio buffer 150. The demultiplexer 130 supplies the reproduction information to the STC generation unit 160.

  The STC generator 160 generates an STC (System Time Clock) for controlling timing based on the reproduction information, and supplies the STC (System Time Clock) to the video decoder 170 and the audio decoder 190. The video decoder 170 reads and decodes the video data stored in the video buffer 140 and outputs it as a video signal. The audio decoder 190 reads and decodes the audio data stored in the audio buffer 150 and outputs it as an audio signal. The PTS detector 180 reads the video data stored in the video buffer 140 via the video decoder 170 and detects the PTS. Similarly, the PTS detection unit 180 reads audio data stored in the audio buffer 150 via the audio decoder 190 and detects PTS. The PTS detection unit 180 supplies the PTS detection result to the synchronization control unit 200. The synchronization control unit 200 determines the output timing of the video decoder 170 and the audio decoder 190 based on the detection state of the PTS.

  FIG. 6 is a conceptual diagram showing the relationship between the video buffer 140 during normal playback of a stream and an output video image signal. The video image is output so that the values of STC and PTS are the same according to an instruction from the synchronization control unit 200. Here, a certain amount of video data is stored in the video buffer 140 in advance so that the data supply to the video decoder 170 is not exhausted.

  Next, an operation when performing angle switching (switching only the angle of the screen while reproducing the same sound) using the synchronized playback device 10 described in Patent Document 1 will be described with reference to FIGS. 7 and 8. . FIG. 7 is a diagram illustrating an operation concept at the time of angle switching. FIG. 8 is a flowchart showing the operation of the synchronization control device 10 at the time of angle switching.

  The user instructs angle switching at an arbitrary timing using a remote controller or the like. In the example of FIG. 7, a video switching instruction is given during display of PTS # 5 of video data relating to the first angle (hereinafter referred to as Video # 1). Here, the video buffer 140 of this example stores data of PTS 6 to 10 of Video # 1. The video decoder 170 stops processing and discards the video data in the video buffer 140 (S11).

  The synchronization control device 10 continues to display Video # 1 until the angle switching is completed (S12). In the example of FIG. 7, the synchronization control device 10 continues to display the PTS # 5 of Video # 1 until the angle switching is completed. Next, the demultiplexer 130 is set so as to extract video data (hereinafter referred to as “Video # 2”) relating to the designated angle (S13). As a result, Video # 2 data is stored in the video buffer 140. At this time, since the video data of Video # 1 is sequentially discarded from the video buffer 140, the data of Video # 2 starts to be stored from the data of PTS11. Thereafter, the video decoder 170 resumes the processing (S14).

  The PTS detector 180 detects the PTS of Video # 2 (S15). Further, the PTS detector 180 extracts the PTS of the audio data (hereinafter referred to as “Audio # 1”) (S16). The synchronization control unit 200 subtracts the difference between the PTS of the Video # 2 and the PTS of the Audio # 1 from the difference between the PTS of the Video # 1 and the PTS of the Audio # 1, and determines whether or not the absolute value of the subtraction result is within a threshold value. Determine (S17).

  If the absolute value of the subtraction result is within the threshold (S17: Yes), the synchronization control unit 200 determines that the video and audio are synchronized, and instructs the video decoder 170 to switch the video. Then, the video decoder 170 outputs a video signal obtained by decoding Video # 2 (S18). On the other hand, if the absolute value of the subtraction result is not within the threshold (S17: No), the PTS of Video # 2 and the PTS of Audio # 1 are extracted again (S15, S16).

  For example, in the angle switching example of FIG. 7, the PTS of Video # 1 is 5, and the PTS of Audio # 1 is 6 (not shown). In this case, the PTS of Video # 2 is 11, and the threshold is 2. In this case, after the re-extraction of PTS (Video # 2: 11 → 12 →..., Audio # 1: 6 → 7 →...) Occurs, it is determined that the audio and video are synchronized. . After the determination, the display image is switched to Video # 2. In this way, a certain waiting time occurs in switching between images.

  Next, a configuration in which the synchronization control device 10 has a plurality of video buffers will be considered. FIG. 9 is a block diagram showing a configuration when the synchronization control apparatus 10 has a plurality of video buffers. Compared with the configuration of FIG. 5, the synchronization control device 10 includes a video buffer 1 (140), a video buffer 2 (141), a video buffer 3 (142), a video decoder 1 (170), a video decoder 2 (171), and video. A decoder 3 (172) is provided. Each video buffer stores video data of each angle. Each video decoder reads video data from a corresponding video buffer and performs a decoding process. Here, only the designated video decoder performs output processing, thereby realizing video switching. The demultiplexer 130 reads three types of video data, and stores the read video data in the video buffers 1 to 3.

  Note that, for example, in the Blu-ray ROM (Read Only Memory) standard, nine types of video data and 32 types of audio data may be included in the same medium, and the synchronization control device may include an arbitrary number of video buffers or the like corresponding thereto. It is possible to provide only.

  Processing at the time of angle switching by the synchronization control device 10 shown in FIG. 9 will be described. FIG. 10 is a diagram showing an operation concept at the time of angle switching (switching from Video # 1 to Video # 2) by the synchronization control device 10 shown in FIG. FIG. 11 is a flowchart showing an operation at the time of angle switching (switching from Video # 1 to Video # 2) of the synchronization control device 10 shown in FIG. As described above, the synchronization control device includes three video buffers and three video decoders, and the demultiplexer 130 supplies video data of each angle to each video buffer.

  When receiving an instruction to switch the angle from Video # 1 to Video # 2 from the user, the PTS controller 180 reads the video data from the video buffer 1 (170) and extracts the PTS. Then, the PTS controller 180 reads the video data from the video buffer 2 (171) and extracts the PTS (S15). Further, the PTS controller 180 extracts the PTS of Audio # 1 (S16). The synchronization control unit 200 subtracts the difference between the PTS of the Video # 2 and the PTS of the Audio # 1 from the difference between the PTS of the Video # 1 and the PTS of the Audio # 1, and determines whether or not the absolute value of the subtraction result is within a threshold value. Determine (S17).

  If the absolute value of the subtraction result is within the threshold (S17: Yes), the synchronization control unit 200 determines that the video and audio are synchronized, and instructs the video decoder 170 to switch the video. Then, the video decoder 170 outputs a video signal obtained by decoding Video # 2 (S18). On the other hand, if the absolute value of the subtraction result is not within the threshold (S17: No), the PTS of Video # 2 and the PTS of Audio # 1 are extracted again (S15, S16).

  Here, in the synchronous control device 10 shown in FIG. 9, video data is read in parallel to a plurality of video buffers. Therefore, as shown in FIG. 10, it is possible to synchronize at an early stage from the angle switching instruction and to switch the video quickly.

JP 2006-340102 A

  In the configuration of the synchronous control device 10 shown in FIG. 5 described above, there is a problem that a certain waiting time occurs when the angle is switched (switching of the output video signal). On the other hand, the configuration of the synchronization control device 10 shown in FIG. 9 requires a plurality of video buffers and video decoders, and thus has a problem that the hardware configuration becomes complicated and the device is expensive.

  That is, there is a problem that quick video switching cannot be realized by a synchronous control device with a simple configuration.

  One aspect of a synchronous control device according to the present invention is a synchronous playback device that outputs a video signal and an audio signal in synchronization from a superimposed signal on which first video data, second video data, and audio data are superimposed, A first video buffer for storing the first video data; a second video buffer for storing the second video data; a first audio buffer for storing the first audio data; A video decoder that decodes the first or second video data stored in the first or second video buffer and outputs the video signal; and the first video data stored in the first audio buffer. An audio decoder that decodes audio data and outputs the audio signal; and the superimposition signal as the first video data and the second video. Data and the audio data, and at the time of the display switching command from the video signal applied to the first video data to the video signal applied to the second video data, the first video data being displayed Based on the time stamp, the second video data is extracted from the superimposed signal and stored in the second video buffer; and the video data read from the first or second video buffer A synchronization processing unit for determining the output timing of the video signal and the audio signal based on the time stamp and the time stamp of the audio data read from the audio buffer.

  One aspect of the synchronization control method according to the present invention is a synchronization control method for synchronously outputting a video signal and an audio signal from a superimposed signal on which first video data, second video data, and audio data are superimposed, The time stamp of the displayed first video data is extracted in response to a display switching command from the video signal related to the first video data to the video signal related to the second video data, and the time stamp The second video data corresponding to the second video data is extracted, and the video signal and the audio signal are synchronized based on the time stamp of the extracted second video data and the time stamp of the audio data. .

  In the present invention, when the display is switched, the time stamp of the first video data currently being displayed is extracted, and the second video data to be switched corresponding to the time stamp is extracted. The extracted second video data is stored in a second video buffer that is a buffer for storing switching target data. By using video data with a time stamp being displayed, video and audio can be quickly synchronized. Furthermore, since there are only two video buffers, synchronization control can be realized with a simple configuration.

  ADVANTAGE OF THE INVENTION According to this invention, the synchronous control apparatus which can implement | achieve quick video switching with the synchronous control apparatus of simple structure, and a synchronous control method can be provided.

1 is a block diagram illustrating a configuration of a synchronization control device according to a first exemplary embodiment; 3 is a flowchart showing an operation of the synchronous control device according to the first exemplary embodiment; 3 is a flowchart showing an operation of the synchronous control device according to the first exemplary embodiment; It is a figure which shows the operation | movement concept at the time of angle switching by the synchronous control apparatus concerning Embodiment 1. FIG. It is a block diagram which shows the structure of the synchronous control apparatus concerning patent document 1. FIG. In the synchronous control device according to Patent Document 1, it is a conceptual diagram showing a relationship between a video buffer during normal playback of a stream and an output video image signal. It is a figure which shows the operation | movement concept at the time of angle switching concerning the synchronous control apparatus concerning patent document 1. FIG. 10 is a flowchart showing an operation at the time of angle switching of the synchronous control device according to Patent Document 1; It is a block diagram which shows the structure of the synchronous control apparatus which has a some video buffer. It is a figure which shows the operation | movement concept at the time of angle switching by the synchronous control apparatus which has a some video buffer. It is a block diagram which shows the structure of the synchronous control apparatus which has a some video buffer.

<Embodiment 1>
Embodiments of the present invention will be described below with reference to the drawings. FIG. 1 is a block diagram showing the configuration of the synchronization control device according to the present embodiment. In addition, it means that the processing part and the memory | storage part which attached | subjected the same code | symbol as the synchronous control apparatus 10 shown in FIG. 5, FIG. 9 have the same role in principle.

  The synchronization control apparatus 10 includes an ATAPI control unit 110, a playback buffer 120, a demultiplexer 130, a video buffer 1 (140), a video buffer 2 (141), an audio buffer 150, an STC generation unit 160, a video decoder 170, and a PTS detection unit. 180, an audio decoder 190, and a synchronization control unit 200. As described above, the synchronization control apparatus includes two video buffers and one video decoder.

  The ATAPI control unit 110 reads stream data from the medium 300 and stores the read stream data in the playback buffer 120. The demultiplexer 130 reads the stream data from the playback buffer 120 and separates it into video data, audio data, and reproduction information. The demultiplexer 130 stores the video data in the video buffer 140. The demultiplexer 130 stores the audio data in the audio buffer 150. The demultiplexer 130 supplies the reproduction information to the STC generation unit 160.

  The demultiplexer 130 receives the PTS information of the currently displayed Video # 1 from the PTS control unit 180 when the angle is switched (in the following example, the video # 1 is switched to the Video # 2). The demultiplexer 130 then supplies the video buffer 1 (140) or the video buffer 2 (141) in order from the video data of Video # 2 having the same PTS as the received PTS.

  The video buffer 1 (140) and the video buffer 2 (141) are buffers for temporarily storing the video data extracted by the demultiplexer 130. Here, in the video buffer 1 (140) and the video buffer 2 (141), one side stores the video data related to the video (currently displayed video) displayed before the angle switching, and the other is displayed after the angle switching. Video data relating to the video to be recorded is stored. Each time the angle is switched, the video buffer 1 (140) and the video buffer 2 (141) are switched by a toggle. During normal display, the video buffer on the side other than the video buffer storing the video data for the current display does not hold the data. That is, one of the video buffers is empty.

  The audio buffer 150 is a buffer that temporarily stores the audio data supplied from the demultiplexer 130. The STC generation unit 160 generates an STC (System Time Clock) for controlling timing based on the reproduction information, and supplies the STC (System Time Clock) to the video decoder 170 and the audio decoder 190.

  The video decoder 170 reads video data from the video buffer 1 (140) or the video buffer 2 (141), decodes the read video data, and outputs it as a video signal. The video decoder 170 outputs a video signal in response to an output timing instruction from the synchronization control unit 200 described later.

  The audio decoder 190 reads and decodes the audio data stored in the audio buffer 150 and outputs it as an audio signal. The audio decoder 190 outputs an audio signal in response to an output timing instruction from a synchronization control unit 200 described later.

  The PTS detector 180 reads the video data stored in the video buffer 1 (140) or the video buffer 2 (141) via the video decoder 170, and detects the PTS. Similarly, the PTS detection unit 180 reads audio data from the audio buffer 150 via the audio decoder 190 and detects PTS. Furthermore, when an angle switching instruction is input from the outside, the PTS detection unit 180 extracts the PTS of the video data currently being displayed and notifies the demultiplexer 130 of the extracted PTS.

  The synchronization control unit 200 determines the output timing of the video decoder 170 and the audio decoder 190 based on the PTS detection state of the video data and audio data. At the time of angle switching, the synchronization control unit 200 subtracts the difference between the PTS of the video data and the audio data to be switched from the difference between the PTS of the currently displayed video data and the PTS of the audio data. The video data output timing is determined according to whether or not the absolute value of the video data exceeds a predetermined threshold value. The synchronization control unit 200 notifies the video decoder 170 and the audio decoder 190 of the determined output timing.

  Next, the operation when the synchronization control apparatus 10 according to the present embodiment performs angle switching (switching only the angle of the screen while reproducing the same sound) will be described with reference to FIG. 2, FIG. 3, and FIG. . 2 and 3 are flowcharts showing the operation of the synchronization control apparatus 10 according to the present embodiment. FIG. 4 is a diagram illustrating an operation concept at the time of angle switching by the synchronization control device 10 according to the present embodiment. 2, 3 and 4, an example of switching from Video # 1 to Video # 2 will be described. In this example, Video # 1 is stored in video buffer 1 (140).

  In the synchronous control device 10 according to the present embodiment, a new process (S2) is performed between S13 and S14 in the flowchart shown in FIG. The new process is shown in FIG. After the setting of the demultiplexer 130 is completed (S13), the PTS detection unit 180 reads the data of the video data Video # 1 currently being displayed from the video buffer 1 (140), and extracts the PTS (S21). The PTS detector 180 supplies the detected PTS to the demultiplexer 130 (S21). In the example of FIG. 4, the PTS detection unit 180 extracts PTS # 40.

  The demultiplexer 130 extracts the stream data related to PTS # 40 from the playback buffer 120 (S22), and extracts only the video data of Video # 2 (S23). Thereafter, the demultiplexer 130 stores the extracted video data of Video # 2 in the video buffer 2 (141) (S14). Then, the video decoder 170 is activated (S14). The subsequent processing is substantially the same as the processing shown in FIG.

  In the example of FIG. 4, the video data of PTS # 40 of Video # 2 is immediately stored in the video buffer 2 (141) after the angle switching instruction. Here, PTS # 40 is the same as the PTS value of Video # 1 when the angle switching instruction is accepted. Such control shortens the time until video and audio can be synchronized as compared with the processing shown in FIG.

  As described above, in the synchronous control device according to the present embodiment, at the time of display switching, the video data to be switched is stored in the empty video buffer according to the PTS of the currently displayed video data. As a result, the video data from the video buffer is not discarded. Furthermore, since synchronization determination is performed from video data having the same value as the time stamp of the video data being displayed, the time spent for synchronization control can be reduced.

  Furthermore, the synchronization control device according to the present embodiment can achieve the above-described effects with a simple configuration including two video buffers and one video decoder.

  Note that the present invention is not limited to the above-described embodiment, and can be changed as appropriate without departing from the spirit of the present invention.

DESCRIPTION OF SYMBOLS 10 Synchronization control apparatus 110 ATAPI control part 120 Playback buffer 130 Demultiplexer 140, 141 Video buffer 150 Audio buffer 160 STC generation part 170 Video decoder 180 PTS detection part 190 Audio decoder 200 Synchronization control part 300 Media

Claims (8)

A synchronous playback device that outputs a video signal and an audio signal synchronously from a superimposed signal obtained by superimposing first video data, second video data, and audio data,
A first video buffer for storing the first video data;
A second video buffer for storing the second video data;
A first audio buffer for storing the first audio data;
A video decoder that decodes the first or second video data stored in the first or second video buffer and outputs the video signal;
An audio decoder that decodes the first audio data stored in the first audio buffer and outputs the audio signal;
The superimposed signal is separated into the first video data, the second video data, and the audio data, and the display from the video signal related to the first video data to the video signal related to the second video data A data separation unit for extracting the second video data from the superimposed signal and storing it in the second video buffer based on a time stamp of the first video data being displayed at the time of the switching command;
Synchronization processing for determining the output timing of the video signal and the audio signal based on the time stamp of the video data read from the first or second video buffer and the time stamp of the audio data read from the audio buffer A synchronized playback device.   The data separation unit sequentially extracts the second video data having a time stamp equivalent to the time stamp of the first video data being displayed from the superimposed signal when the display switching command is issued. The synchronous control device according to claim 1.   The synchronization processing unit, based on the difference between the time stamp of the first video data and the time stamp of the audio data, at the time of the display switching command, the time stamp of the second video data and the time stamp of the audio data. The synchronous control device according to claim 1 or 2, wherein the video signal and the audio signal are synchronously output at a time point when a subtraction result obtained by subtracting the difference is within a predetermined threshold value. .   4. The synchronization control apparatus according to claim 1, wherein the superimposed signal is MPEG format multimedia data. A synchronization control method for synchronously outputting a video signal and an audio signal from a superimposed signal obtained by superimposing first video data, second video data, and audio data,
The time stamp of the displayed first video data is extracted in response to a display switching command from the video signal related to the first video data to the video signal related to the second video data, and the time stamp The second video data corresponding to the second video data is extracted, and the video signal and the audio signal are synchronized based on the time stamp of the extracted second video data and the time stamp of the audio data. .   6. The second video data having a time stamp equivalent to a time stamp of the first video data being displayed is sequentially extracted from the superimposed signal at the time of the display switching command. The synchronous control method described.   At the time of the display switching command, the difference between the time stamp of the second video data and the time stamp of the audio data is subtracted from the difference between the time stamp of the first video data and the time stamp of the audio data. 7. The synchronization control method according to claim 5, wherein the synchronization control is performed so that the video signal and the audio signal are output when a subtraction result falls within a predetermined threshold.   8. The synchronization control method according to claim 5, wherein the superimposed signal is MPEG format multimedia data.

Images