WO2013162335A1 - Image playback apparatus for 3dtv and method performed by the apparatus - Google Patents

Abstract

Disclosed are an image playback apparatus for 3DTV and a method performed by the apparatus. The image playback apparatus can determine the output timing of a buffer for storing a left image stream and a right image stream for a 3D image. Furthermore, the image playback apparatus can determine a buffer size or a buffer delay time by a receipt time difference between the left image stream and the right image stream. Furthermore, the image playback apparatus can correct a reference clock or a time stamp by a receipt time difference between the left image stream and the right image stream.

Description

Image playback device for 3DTV and method performed by the device

The present invention relates to an image reproducing apparatus for a 3DTV and a method performed by the image reproducing apparatus, and more particularly, to efficiently buffer a buffer for synchronizing a stream of a reference video and an additional video transmitted through different transmission paths. It's about processing.

In order to represent a 3D image in a conventional converged 3DTV, a reference image and an additional image are required. In this case, in general, the reference video is for high definition 2D HD fixed broadcasting, and is encoded by MPEG-2 and transmitted through a fixed broadcasting network. The additional video is for 2D mobile broadcasting and is encoded by AVC (Advanced Video Coding) and transmitted through an IP network. That is, the base view video and the additional view video are transmitted based on different types of transmission protocols. Then, the 3DTV generates and reproduces the 3D image by using the received reference image and the additional image. Here, the reference image is set as the left image for generating the 3D image, and the additional image is set as the right image for generating the 3D image.

Since the base view video and the additional view video for 3D video are transmitted through different transport protocols, they have different timing models. In detail, the PTS (Presentation) based timing model based on the MPEG-2 system is used as the reference image corresponding to the left image, and the RTP (Realtime Transport Protocol) based timing model corresponding to the right image is used. That is, since the timing models of the reference video and the additional video are different from each other, a video reproducing apparatus for expressing 3DTV may require a time stamp (PTS) of the reference video and a time stamp (RTP) of the additional video to be simultaneously output at any particular point in time. The offset between the two is set as the synchronization information, and the output time of the reference video and the additional video is determined by using the synchronization information.

In this case, since the image reproducing apparatus does not provide a separate buffer management and buffer control model, there is a problem in minimizing the efficiency of buffer use and the synchronization delay time. Meanwhile, the renderer buffer included in the image reproducing apparatus has a problem that an overwhelmingly large buffer capacity is required to store the stream for a predetermined time period compared to the ES (Elementary Stream) buffer.

Accordingly, a method for efficiently operating a buffer included in an image reproducing apparatus when a left image and a right image are not simultaneously received and there is a reception delay between each other is required.

The present invention provides an apparatus and method for determining a buffer size for storing a left video stream and a right video stream by using a reception time difference between a left video stream and a right video stream.

The present invention provides an apparatus and method for determining a buffer delay time for outputting a left video stream and a right video stream by using a difference in reception time between a left video stream and a right video stream.

The present invention provides an apparatus and method for correcting a reference clock or correcting a time stamp using a difference in reception time between a left video stream and a right video stream.

The present invention provides an apparatus and method for synchronizing a left video stream and a right video stream by converting a timestamp according to a reference clock.

A buffer control method of an image reproducing apparatus according to an embodiment of the present invention includes: obtaining synchronization information between a left image stream and a right image stream; Extracting a first timestamp of the left video stream and a second timestamp of the right video stream; Calculating a reception time difference between the left video stream and the right video stream using the first timestamp, the second timestamp, and the synchronization information; And determining a buffer size by using the reception time difference.

The acquiring of the synchronization information may include obtaining synchronization information including timestamp pair information including a first timestamp of a left video stream and a second timestamp of a right video stream, which should be simultaneously output at any particular time. Can be.

The acquiring of the synchronization information may include synchronization information including offset information between a first timestamp of the left video stream and a second timestamp of the right video stream.

The extracting of the first timestamp of the left video stream and the second timestamp of the right video stream may include extracting the first timestamp of the left video stream and the access unit of the left video stream and the access unit of the right video stream received at the same time. The first time stamp and the second time stamp may be extracted based on the result.

The extracting of the first timestamp of the left video stream and the second timestamp of the right video stream may include: extracting a first timestamp of the left video stream and a second timestamp of the right video stream extracted at a close time point through demultiplexing; Can be extracted.

The calculating of the reception time difference may include calculating a reception time difference by using a timestamp difference between a first timestamp and a second timestamp when the left video stream and the right video stream use the same reference clock. If the left video stream and the right video stream use different reference clocks or random offsets are applied, one of the first time stamp and the second time stamp is converted by using the synchronization information, and the first time stamp and the first time stamp are used. The difference in the reception time may be calculated using the time stamp difference between the two timestamps.

The determining of the buffer size may include calculating the number of delayed frames between the left video stream and the right video stream by using the number of access units stored in the buffer, the difference in reception time, and the frame rate of the video stream. As a result, the buffer size that is dynamically set can be determined.

In the determining of the buffer size, the buffer size may be determined by using a fixed bit rate and a reception time difference of the left video stream or the right video stream.

In the determining of the buffer size, the buffer size may be determined by using a data rate and a reception time difference of the left video stream or the right video stream.

According to another aspect of the present invention, there is provided a method of controlling a buffer of an image reproducing apparatus, the method including: obtaining synchronization information between a left image stream and a right image stream; Extracting a first timestamp of the left video stream and a second timestamp of the right video stream; Calculating a reception time difference between the left video stream and the right video stream using the first timestamp, the second timestamp, and the synchronization information; And determining a buffer delay time by using the reception time difference.

The determining of the buffer delay time may include: when the right video stream is received before the left video stream, increasing the buffer delay time of the right video stream by a difference in reception time, and when the left video stream is received before the right video stream. In addition, the buffer delay time of the left video stream may be increased by the difference of the reception time.

A clock correction method of an image reproducing apparatus according to an embodiment of the present invention includes the steps of: calculating a difference in reception time between an access unit of a left image stream and an access unit of a right image stream; And a reference clock signaled to the first received video stream among the left video stream and the right video stream by using the reception time difference.

In the correcting of the reference clock, the reference clock may be corrected by using a result of applying a reception time difference to the clock rate of the reference clock.

According to an aspect of the present invention, there is provided a method of correcting a time stamp of a video reproducing apparatus, the method comprising: calculating a difference in reception time between an access unit of a left video stream and an access unit of a right video stream; And correcting the first timestamp of the left video stream and the second timestamp of the right video stream by using the reception time difference.

The correcting of the first timestamp of the left video stream and the second timestamp of the right video stream may include performing a first timestamp and a right video of the left video stream by using a result of applying a reception time difference to a clock rate of a reference clock. The second timestamp of the stream may be corrected.

A synchronization method of an image reproducing apparatus according to an embodiment of the present invention includes the steps of obtaining synchronization information between a left image stream and a right image stream; Extracting a first timestamp of the left video stream and a second timestamp of the right video stream; Converting at least one of the first timestamp and the second timestamp to correspond to a reference clock using synchronization information; And matching and synchronizing a left video stream and a right video stream according to the reference clock.

An image reproducing apparatus according to an embodiment of the present invention includes a first buffer unit for receiving and storing a left image stream; A second buffer unit for receiving and storing a right image stream; And a buffer controller configured to determine a buffer size of the first buffer unit and the second buffer unit based on a difference in reception time between the left video stream and the right video stream, wherein the difference in the reception time is determined by the first of the left video stream. The time stamp, the second time stamp of the right video stream, and the synchronization information between the left video stream and the right video stream may be determined.

According to another embodiment of the present invention, an image reproducing apparatus includes: a first buffer unit configured to receive and store a left image stream; A second buffer unit for receiving and storing a right image stream; And a buffer controller configured to determine a buffer delay time of a first buffer unit or a second buffer unit by using a difference in reception time between the left video stream and the right video stream, wherein the difference in the reception time is determined by the first of the left video stream. The time stamp, the second time stamp of the right video stream, and the synchronization information between the left video stream and the right video stream may be determined.

According to another embodiment of the present invention, an image reproducing apparatus includes: a first buffer unit configured to receive and store a left image stream; A second buffer unit for receiving and storing a right image stream; And (i) correcting the reference clock signaled to the first received video stream of the left video stream and the right video stream by using the difference in the reception time between the access unit of the left video stream and the access unit of the right video stream, or (ii) And a buffer controller configured to correct the first timestamp of the video stream and the second timestamp of the right video stream.

According to another embodiment of the present invention, an image reproducing apparatus includes: a first buffer unit configured to receive and store a left image stream; And a second buffer unit for receiving and storing the right image stream. At least one of a first timestamp of the left video stream and a second timestamp of the right video stream is converted according to a reference clock using the synchronization information between the left video stream and the right video stream, and the left video stream and the right video stream are converted according to the reference clock. It may include a video synchronization unit for matching and synchronizing the video stream.

According to an embodiment of the present invention, the buffer may be efficiently managed by determining a buffer size for storing the left video stream and the right video stream by using a difference in reception time between the left video stream and the right video stream.

According to an embodiment of the present invention, the synchronization delay time may be minimized by determining a buffer delay time for outputting the left video stream and the right video stream using the difference in the reception time between the left video stream and the right video stream.

According to an embodiment of the present invention, even if a reception time difference occurs by correcting a reference clock or a timestamp by using a reception time difference between a left video stream and a right video stream, decoding of the video stream may operate normally. .

According to an embodiment of the present invention, by converting the time stamp according to the reference clock to synchronize the left video stream and the right video stream, it is possible to provide a high quality 3D video.

1 is a diagram illustrating an image transmission device and an image reproduction device according to an embodiment of the present invention.

2 is a diagram illustrating components of an image reproducing apparatus according to an embodiment of the present invention.

3 is a diagram illustrating an example of an image reproducing apparatus according to an embodiment of the present invention.

4 is a diagram illustrating a process of extracting synchronization information, a first time stamp, and a second time stamp according to an embodiment of the present invention.

5 is a diagram illustrating a process of controlling a buffer size according to an embodiment of the present invention.

6 is a diagram illustrating a process of correcting a reference clock or correcting a time stamp according to one embodiment of the present invention.

7 is a flowchart illustrating a process of controlling a buffer size according to an embodiment of the present invention.

8 is a flowchart illustrating a process of determining a buffer delay time according to an embodiment of the present invention.

9 is a flowchart illustrating a process of calibrating a reference clock according to an embodiment of the present invention.

10 is a diagram illustrating a process of correcting a time stamp according to one embodiment of the present invention.

11 is a diagram illustrating a process of synchronizing a left video stream and a right video stream according to an embodiment of the present invention.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1 is a diagram illustrating an image transmission device and an image reproduction device according to an embodiment of the present invention.

Referring to FIG. 1, the image transmitting apparatus 101 may transmit a main image and an additional image for the 3D image to the image reproducing apparatus 102. For example, the reference video is for stationary broadcast for high definition 2D High Definition (HD) and may be transmitted to the image reproducing apparatus 102 according to the MPEG-2 scheme. The additional video is for 2D mobile broadcast, and may be transmitted to the video reproducing apparatus 102 according to the AVC scheme. The image reproducing apparatus 102 may receive the reference image and the additional image in the form of a stream, and provide a 3D image to the viewer through 3D rendering.

Since the reference video and the additional video are transmitted through one channel of 6MHz, the 3DTV can broadcast both high definition 2DTV, mobile 2DTV, and high definition 3DTV, thereby maximizing frequency efficiency. Here, the reference image may correspond to the left image for the 3D image, and the additional image may correspond to the left image for the 3D image. Alternatively, the reference image may correspond to the right image, and the additional image may correspond to the left image. Hereinafter, it will be described on the premise that the reference video is transmitted to the left video stream and the additional video is transmitted to the right video stream.

For the 3D image, it is necessary to simultaneously provide the viewer with the reference image which is the left image and the additional image which is the right image. However, for 3DTV broadcasting services, processing time may vary due to different encoding, multiplexing, modulation, transmission, demodulation, demultiplexing, and decoding of the reference video and the additional video. Thus, the image transmitting apparatus 101 broadcasts synchronization information for identifying the left image and the right image that the image reproducing apparatus 102 should simultaneously output for synchronization between the reference image that is the left image and the additional image that is the right image. Signaling can be done through the stream.

In this case, the image transmitting apparatus 101 may transmit the synchronization information to the image reproducing apparatus 102 so that the image reproducing apparatus 102 may synchronize the reference image and the additional image having different timing models. Here, the synchronization information is an offset (an absolute value and a sign of the difference between the reference video and the additional video) which is a time stamp difference between each of the reference video and the additional video, which should be simultaneously output at a specific time point in the 3D video. Alternatively, the synchronization information may be pair information of time stamps of each of the reference video and the additional video. Then, the image reproducing apparatus 102 may determine an output time point of the base view video and the additional view video based on the synchronization information.

In the present invention, it is assumed that a base view video and an additional view video having different timing models required for 3DTV video are not simultaneously received by the 3DTV device. That is, since the reference video and the additional video are transmitted through different broadcasting networks, they may not be simultaneously transmitted to the video reproducing apparatus 102. Therefore, the video reproducing apparatus 102 needs to store the received video stream in a buffer before waiting to synchronize the reference video with the additional video, and wait until a video stream not yet received arrives.

Thus, the present invention proposes how to manage buffers for the base view video and the additional view video. Specifically, the present invention proposes how to determine the size of the buffer based on the difference in the reception time of the base view video and the additional view video. Meanwhile, the present invention may determine when to output to the decoder in the buffer for the reference video and the additional video, respectively. In addition, the present invention proposes a matching process between the base image and the additional image. In particular, in order to reduce the inefficiency of the buffer operation according to the reception delay of the video stream, while storing the stream of the left video and the stream of the right video in the buffer before decoding, to minimize the synchronization delay time between the video stream Can provide.

For example, in the present invention, the base view video may have a timing model based on the MPEG-2 system, and the additional view video may have a timing model based on the RTP. However, the present invention is not limited to the timing model, and may be applied to the case where the reference image and the additional image have different timing models.

2 is a diagram illustrating components of an image reproducing apparatus according to an embodiment of the present invention.

Referring to FIG. 2, the image reproducing apparatus 102 includes a first buffer unit 201, a second buffer unit 202, a buffer controller 203, a first decoder 204, and a second decoder 205. It may include a third buffer unit 206 and the image synchronization unit 207.

The first buffer unit 201 may store a left image stream which is a reference image transmitted through the first broadcasting network. The second buffer unit 202 may store a right image stream which is an additional image transmitted through the second broadcasting network. The first buffer unit 201 may temporarily store the left image stream before decoding the left image stream, and the second buffer unit 202 may temporarily store the right image stream before decoding the right image stream.

<Buffer size decision>

The buffer controller 203 may determine the buffer sizes of the first buffer unit 201 and the second buffer unit 202. In this case, the buffer controller 203 may determine the buffer sizes of the first buffer unit 201 and the second buffer unit 202 using the difference in the reception time between the left video stream and the right video stream. In this case, the reception time difference may be generated because the left video stream and the right video stream transmitted from the video transmission device 101 are not simultaneously received by the video playback device 102.

The buffer controller 203 may obtain synchronization information between the left video stream and the right video stream. At this time, the synchronization information may be extracted by demultiplexing the bitstream. For example, the synchronization information may include timestamp pair information including a first timestamp of a left video stream and a second timestamp of a right video stream, which should be simultaneously output at any particular point in time. Alternatively, the synchronization information may include offset information between the first timestamp of the left video stream and the second timestamp of the right video stream.

For example, when the left video stream is transmitted through the ATSC main broadcast standard, which is a fixed broadcast standard, and the right video stream is transmitted through the MDTV broadcast standard, which is a mobile broadcast standard, the first timestamp is a presentation timestamp (PTS) and a second. The time stamp may be a realtime protocol (RTP) time stamp. Here, the offset information may include an absolute value and a sign of a difference between the first timestamp and the second timestamp. The synchronization information may be signaled through at least one of a left video stream and a right video stream.

The buffer controller 203 may extract the first timestamp of the left video stream and the second timestamp of the right video stream. For example, the first timestamp may be a decoding timestamp (DTS) of the left video stream, and the second timestamp may be a decoding timestamp of the right video stream. If the RTP stream does not have a decoding timestamp, an output timestamp (PTS, RTP Timestamp) may be extracted. In addition, when no time stamp exists, a frame identification number or a Society of Motion Picture and Television Engineers (SMTP) Timecode may be used.

The object from which the time stamp is extracted may be an access unit that is received at the same time and is the access unit most recently stored in the first buffer unit 201 and the second buffer unit 202. Alternatively, the first timestamp and the second timestamp may be extracted from the left video stream and the right video stream, respectively, which are extracted at a close time through the demultiplexing process of the bitstream.

For example, the buffer controller 203 may extract the first timestamp of the left image frame most recently stored in the first buffer unit 201. The buffer controller 203 may extract the second timestamp of the right image frame most recently stored in the second buffer unit 202.

In this case, the buffer controller 203 may extract the PTS of the first frame of the group of picture (GOP) of the left video stream most recently stored in the first buffer unit 201. Alternatively, the buffer controller 203 may extract the PTS of the last frame or the reference frame among the group of pictures (GOPs) of the left image stream most recently stored in the first buffer unit 201.

The buffer controller 203 may extract the RTP timestamp of the first frame of the GOP of the right image stream most recently stored in the second buffer unit 202. Alternatively, the buffer controller 203 may extract the RTP timestamp of the last frame or the reference frame of the group of picture (GOP) of the right image stream most recently stored in the second buffer unit 202. Meanwhile, the buffer controller 203 may extract the time stamp of the most recently stored access unit instead of the GOP in order to calculate the buffer size more quickly.

A process of extracting the synchronization information, the first time stamp, and the second time stamp will be described in detail with reference to FIG. 5.

The buffer controller 203 may calculate a reception time difference between the left video stream and the right video stream. For example, the buffer controller 203 may calculate a reception time difference using the synchronization information, the time stamp of the left video stream, and the time stamp of the right video stream.

If the left video stream and the right video stream use the same reference clock, the buffer controller 203 may calculate the reception time difference using the time stamp difference between the first time stamp and the second time stamp. If the left video stream and the right video stream use different reference clocks, the buffer controller 203 converts any one of the first timestamp and the second timestamp by using the synchronization information. The difference in reception time between the stamp and the second timestamp may be calculated. Meanwhile, when a random offset is applied to the first time stamp or the second time stamp, such as an RTP time stamp, the first time stamp and the second time stamp are converted by using either of the synchronization information and the first time stamp or the second time stamp. The difference in reception time between time stamps can be calculated. In detail, the buffer controller 203 may convert the first timestamp into the same form as the second timestamp or convert the second timestamp into the same form as the first timestamp using the synchronization information.

Further expansion, the buffer control unit 203 converts the output time information (time stamp, frame identification number, SMTPE timecode, etc.) of one video stream into output time information of the other video stream by using the synchronization information. Receive time difference can be calculated.

When calculating the reception time difference between the left video stream and the right video stream, when the output time stamps (PTS, RTP Timestamp) other than the decoding time stamp (DTS) are used, the decoding order and the buffer output order may not match. In this case, the buffer controller 203 may control the buffer size and the output time in consideration of the number of consecutive B frames in the video stream. For example, when the number of consecutive B frames in the video stream is 3, the first buffer unit 201 and the second video frame may include the received time difference calculated according to the output timestamp and the frames of the actual left and right video streams. A time difference input to each of the buffer units 202 may vary up to four. In this case, the buffer controller 203 may correspond to an error by adding a 4 frame transmission time to the previously calculated reception time difference. Alternatively, the buffer controller 203 may control the buffer size and the buffer output time in consideration of the GOP size. For example, when the output unit of the buffer is a GOP unit, the buffer controller 203 may correspond to an error by adding a frame transmission time equal to the number of frames constituting the GOP to the previously calculated reception time difference.

The buffer controller 203 may determine the buffer size by using a reception time difference between the left video stream and the right video stream. For example, the buffer controller 203 calculates the number of delayed frames between the left video stream and the right video stream by using the number of access units stored in the buffer, the difference in reception time, and the frame rate of the video stream. As a result, the buffer size that is dynamically set can be determined.

The buffer controller 203 may determine the buffer size by using a fixed bit rate and a reception time difference of the left video stream or the right video stream. Alternatively, the buffer controller 203 may determine the buffer size using a difference between the data rate and the reception time of the left video stream or the right video stream.

The above-described process of controlling the buffer size will be described in more detail with reference to FIG. 5.

<Determine Buffer Delay Time>

The buffer controller 203 may calculate a reception time difference between the left video stream and the right video stream by using the first timestamp of the left video stream, the second timestamp of the right video stream, and the synchronization information. The process of calculating the reception time difference is the same as described above.

The buffer controller 203 may determine the buffer delay time using the difference in the reception time. In this case, when the right video stream is received before the left video stream, the buffer controller 203 may increase the buffer delay time of the right video stream by the difference in the reception time. That is, since the right video stream is received before the left video stream, the buffer controller 203 may delay the output time of the second buffer unit 202 by the difference in the reception time from the output time of the first buffer unit 201. .

On the contrary, when the left video stream is received before the right video stream, the buffer controller 203 may increase the buffer delay time of the left video stream by the difference in the reception time. That is, since the left video stream is received before the right video stream, the buffer controller 203 may delay the output time of the first buffer unit 201 by the difference in receiving time from the output time of the second buffer unit 202. .

<Reference Clock Correction and Timestamp Correction>

As described above, in the case of the 3DTV service, the reception time of the left video stream and the right video stream in the video reproducing apparatus 102 may be different. However, it may be difficult to predict the reception delay time between the left video stream and the right video stream when the left video stream and the right video stream are transmitted to the video reproducing apparatus 102. In particular, when the left video stream and the right video stream provide independent services (eg, terrestrial 2D broadcasting service, mobile 2D broadcasting service, IPTV broadcasting service, and VOD broadcasting service) in an independent form, a reference clock of the video stream ( For example, PCR) and time stamps (eg, PTS and DTS) of the access unit may not consider the reception delay time of the 3D service to ensure compatibility with the existing 2D service.

In particular, when time information such as a reference clock and a timestamp is used not only for the 2D service but also for the 3D service, the reception delay time until the other video stream operates when the 3D service is operated should be considered.

The present invention corrects a reference clock or a timestamp even when there is a difference in reception time between a left video stream and a right video stream, so that the first decoder 204 and the right video process the left video stream even when there is a difference in reception time. The second decoder 205 for processing the stream can operate normally.

To correct the reference clock, the buffer controller 203 may calculate a difference in reception time between the access unit of the left video stream and the access unit of the right video stream. In addition, the buffer controller 203 may correct the reference clock signaled to the first received video stream among the left video stream and the right video stream by using the reception time difference. In this case, the buffer controller 203 may correct the reference clock by using the result of applying the reception time difference to the clock rate of the reference clock.

To correct the time stamp, the buffer controller 203 may calculate a difference in reception time between the access unit of the left video stream and the access unit of the right video stream. The buffer controller 203 may correct the first timestamp of the left video stream and the second timestamp of the right video stream by using the reception time difference. In this case, the buffer controller 203 may correct the first timestamp of the left video stream and the second timestamp of the right video stream by using the result of applying the reception time difference to the clock rate of the reference clock. A process of correcting the reference clock and time stamp will be described in more detail with reference to FIG. 6.

The first decoder 204 may decode the left video stream transmitted from the first buffer unit 201 and store the decoded video stream in the third buffer unit 206. In addition, the second decoder 205 may decode the right video stream transmitted from the second buffer unit 202 and store the same in the third buffer unit 206.

The image synchronizer 207 may synchronize the left image stream and the right image stream stored in the third buffer unit 206 and transfer the same to the 3D renderer. For example, the image synchronizer 207 may obtain synchronization information derived by demultiplexing the bitstream. The image synchronizer 207 may obtain a first timestamp of the left video stream and a second timestamp of the right video stream stored in the third buffer unit 206.

The image synchronizer 207 may convert at least one of the first time stamp and the second time stamp to correspond to the reference clock by using the synchronization information. For example, the image synchronizer 207 may convert the RTP timestamp of the right video stream into the same form as the timestamp of the left video stream so as to correspond to a PCR (program clock reference) that is a reference clock. The left image stream has a PTS which is a timestamp corresponding to PCR. The image synchronizer 207 may transfer the left image stream and the right image stream extracted from the third buffer unit 206 based on the PCR to the 3D renderer that generates the 3D image by matching the time stamp with the left image stream.

That is, the left video stream and the right video stream have different timing models, but the image synchronizer 207 converts the first timestamp of the right video stream or the second timestamp of the left video stream according to a reference clock to obtain the same timing model. Can be converted to a timestamp of type. The image synchronizer 207 may match the left video stream and the right video stream by matching the first time stamp and the second time stamp.

3 is a diagram illustrating an example of an image reproducing apparatus according to an embodiment of the present invention.

Referring to FIG. 3, the image reproducing apparatus includes a demultiplexer 301, a PES decapsulator 302, an RTP decapsulator 303, an ES buffer 304, an ES buffer 305, an ES buffer controller 306, and an MPEG-2 Decodoer. 307, an AVC decoder 308, a renderer buffer 309, and an L / R synchronizer 310. Here, the ES buffer 304 corresponds to the first buffer unit 201, the ES buffer 305 corresponds to the second buffer unit 202, and the ES buffer controller 306 corresponds to the buffer controller 203. do. The MPEG-2 Decodoer 307 corresponds to the first decoder 204, and the AVC decoder 308 corresponds to the second decoder 205. In addition, the renderer buffer 309 corresponds to the third buffer unit 206, and the L / R synchronizer 310 corresponds to the image synchronizer 207.

The demultiplexer 301 may demultiplex from a bitstream to extract a left video stream, a right video stream, and synchronization information. Here, the left video stream may be used for high definition 2D HD fixed broadcasting corresponding to the reference video of the 3D video. The left video stream may be processed according to the MPEG-2 scheme and transmitted through the ATSC main stream. Meanwhile, the right image stream corresponds to the additional image of the 3D image and may be used for 2D mobile broadcasting. The right video stream may be processed according to AVC (Advanced Video Coding) and transmitted through the ATSC M / H stream. In this case, the extracted synchronization information may be transmitted to the ES buffer controller 306.

The PES decapsulator 302 may decapsulate a packetized elementary stream (PES) of a left image stream. The presentation timestamp (PTS), which is the first timestamp of the left video stream extracted in this process, may be transmitted to the ES buffer controller 306 and the L / R Synchronizer 310. Meanwhile, the RTP decapsulator 303 may decapsulate the real-time transportation protocol (RTP) packet of the encapsulated right video stream. The RTP timestamp, which is the second timestamp of the right video stream extracted in this process, may be transmitted to the ES buffer controller 306 and the L / R Synchronizer 310.

The left image stream derived from the PES decapsulator 302 may be stored in the ES buffer 304, and the right image stream derived from the RTP decapsulator 303 may be stored in the ES buffer 305. Here, the ES buffer 304 and the ES buffer 305 may temporarily store the left video stream and the right video stream before decoding in a situation where a reception delay occurs between the left video stream and the right video stream. The ES buffer 304 and the ES buffer 305 may be set as the TS buffer 304 and the TS buffer 305.

The ES buffer controller 306 may control and manage the ES buffer 304 and the ES buffer 305.

(1) As an example, the ES buffer controller 306 may determine the buffer sizes of the ES buffer 304 and the ES buffer 305. At this time, the ES buffer controller 306 may determine the buffer sizes of the ES buffer 304 and the ES buffer 305 by using the difference in the reception time of the left video stream and the right video stream. In this case, the reception time difference may be generated because the left video stream and the right video stream transmitted from the video transmission device 101 are not simultaneously received by the video playback device 102.

The ES buffer controller 306 may acquire synchronization information between the left video stream and the right video stream. The ES buffer controller 306 may extract the first timestamp of the left video stream and the second timestamp of the right video stream. The ES buffer controller 306 may calculate a reception time difference between the left video stream and the right video stream using the first time stamp, the second time stamp, and the synchronization information, and determine the buffer size using the reception time difference.

(2) The ES buffer controller 306 may determine the buffer delay time by using the reception time difference. In this case, when the right video stream is received before the left video stream, the ES buffer controller 306 may increase the buffer delay time of the right video stream by a difference in the reception time. That is, since the right video stream is received before the left video stream, the ES buffer controller 306 may delay the output time of the ES buffer 305 by the difference of the reception time from the output time of the ES buffer 304.

On the contrary, when the left video stream is received before the right video stream, the ES buffer controller 306 may increase the buffer delay time of the left video stream by a difference in the reception time. That is, since the left video stream is received before the right video stream, the ES buffer controller 306 may delay the output time of the ES buffer 304 by the difference in the reception time from the output time of the ES buffer 305.

As a result, the ES buffer controller 306 outputs a time point at which the ES buffer 304 delivers the left video stream to the MPEG-2 decoder 307 and an ES buffer 305 delivers the right video stream to the AVC decoder 308. The output time point can be determined.

(3) The ES buffer controller 306 corrects the reference clock or timestamp even when there is a difference in reception time between the left video stream and the right video stream, thereby processing the MPEG-2 video stream even when there is a difference in the reception time. The decoder 307 and the AVC decoder 308 processing the right video stream can operate normally.

In order to correct the reference clock, the ES buffer controller 306 may calculate a difference in reception time between the access unit of the left video stream and the access unit of the right video stream. In addition, the ES buffer controller 306 may correct the reference clock signaled to the first received video stream among the left video stream and the right video stream by using the reception time difference. In this case, the ES buffer controller 306 may correct the reference clock by using a result of applying a reception time difference to the clock rate of the reference clock.

In order to correct the timestamp, the ES buffer controller 306 may calculate a reception time difference between the access unit of the left video stream and the access unit of the right video stream. The ES buffer controller 306 may correct the first timestamp of the left video stream and the second timestamp of the right video stream by using the reception time difference. In this case, the ES buffer controller 306 may correct the first timestamp of the left video stream and the second timestamp of the right video stream by using the result of applying the reception time difference to the clock rate of the reference clock.

The MPEG-2 decoder 307 may decode the left image stream transmitted from the ES buffer 304 and then store the image stream in the renderer buffer 309. The AVC decoder 308 may decode the right image stream transferred from the ES buffer 305 and store the same in the renderer buffer 309.

The L / R synchronizer 310 may synchronize the left video stream and the right video stream stored in the renderer buffer 309. In one example, the L / R synchronizer 310 may obtain synchronization information from the bitstream. The L / R synchronizer 310 may obtain timestamps of the left video stream and the right video stream stored in the renderer buffer 309. The L / R synchronizer 310 may convert the time stamp of the right video stream based on the time stamp of the left video stream using the synchronization information. For example, the L / R synchronizer 310 may convert the RTP timestamp of the right image stream into a PCR-based timestamp. The left image stream has a PTS corresponding to the PCR. The L / R synchronizer 310 may match the left image stream and the right image stream extracted from the renderer buffer 309 based on the PCR to the 3D renderer that generates the 3D image.

That is, although the left video stream and the right video stream have different timing models, the L / R synchronizer 310 converts the time stamp of the right video stream according to the reference clock of the time stamp of the left video stream, and then the reference clock. Accordingly, the left video stream and the right video stream may be matched by synchronizing the time stamp of the left video stream with the time stamp of the converted right video stream. Although the above-described embodiment describes a process of converting the timestamp of the right video stream according to the reference clock associated with the timestamp of the left video stream, the timestamp of the left video stream is converted to the reference clock associated with the timestamp of the right video stream. The transformation according to the present invention may be included in the embodiment of the present invention.

As a result, according to the present invention, the output time of transferring the left video stream corresponding to the left video of the 3D video from the ES buffer 304 to the MPEG-2 Decoder 307 and the right video stream corresponding to the right video of the 3D video The output time from the ES buffer 305 to the AVC Decoder 308 can be determined.

In addition, the present invention may determine the buffer sizes of the ES buffer 304 and the ES buffer 305 by using the reception time difference between the left video stream and the right video stream.

In addition, the present invention may correct the reference clock and the time stamp by using the difference in the reception time of the left video stream and the right video stream.

In addition, the present invention can match the left image stream and the right image stream delivered to the 3D renderer for generating a 3D image.

4 is a diagram illustrating a process of extracting synchronization information, a first time stamp, and a second time stamp according to an embodiment of the present invention.

Referring to FIG. 4, synchronization information may be signaled and transmitted in a bitstream through a first broadcast network. In some cases, the synchronization information may be transmitted through the second broadcast network. In order to perform synchronization between the left video stream and the right video stream, the video transmitting apparatus 101 may time stamps (PTS, RTP timestamp, etc.) of the left video stream and the right video stream, which should be simultaneously output at any particular time in the program. Pair information can be generated as synchronization information. Alternatively, the image transmission device 101 may generate offset information (absolute value and sign of the difference between time stamps) between the time stamps of the left video stream and the right video stream as synchronization information. Then, the image reproducing apparatus 102 may extract synchronization information from the signaling channel of the bitstream transmitted through the broadcasting network.

The buffer controller 203 may obtain synchronization information between the left video stream and the right video stream. At this time, the synchronization information may be extracted by demultiplexing the bitstream. For example, the synchronization information may include timestamp pair information including a first timestamp of a left video stream and a second timestamp of a right video stream, which should be simultaneously output at any particular point in time. Alternatively, the synchronization information may include offset information between the first timestamp of the left video stream and the second timestamp of the right video stream.

For example, when the left video stream is transmitted through the ATSC main broadcast standard, which is a fixed broadcast standard, and the right video stream is transmitted through the MDTV broadcast standard, which is a mobile broadcast standard, the first timestamp is a presentation timestamp (PTS) and a second. The time stamp may be a realtime protocol (RTP) time stamp. Here, the offset information may include an absolute value and a sign of a difference between the first timestamp and the second timestamp. The synchronization information may be signaled through at least one of a left video stream and a right video stream.

The buffer controller 203 may extract the first timestamp of the left video stream and the second timestamp of the right video stream. For example, the first timestamp may be a decoding timestamp (DTS) of the left video stream, and the second timestamp may be a decoding timestamp of the right video stream. If the RTP stream does not have a decoding timestamp, an output timestamp (PTS, RTP Timestamp) may be extracted. In addition, when no time stamp exists, a frame identification number or a Society of Motion Picture and Television Engineers (SMTP) Timecode may be used.

The object from which the time stamp is extracted may be an access unit that is received at the same time and is the access unit most recently stored in the first buffer unit 201 and the second buffer unit 202. Alternatively, the first timestamp and the second timestamp may be extracted from the left image and right image streams extracted at close points through the demultiplexing process of the bitstream.

For example, the buffer controller 203 may extract the first timestamp of the left image frame most recently stored in the first buffer unit 201. The buffer controller 203 may extract the second time stamp of the right image frame most recently stored in the second buffer unit 202. In this case, the buffer controller 203 may extract the PTS of the first frame of the group of picture (GOP) of the left video stream most recently stored in the first buffer unit 201. Alternatively, the buffer controller 203 may extract the PTS of the last frame or the reference frame among the group of pictures (GOPs) of the left image stream most recently stored in the first buffer unit 201.

The buffer controller 203 may extract the RTP timestamp of the first frame of the GOP of the right image stream most recently stored in the second buffer unit 202. Alternatively, the buffer controller 203 may extract the RTP timestamp of the last frame or the reference frame of the group of picture (GOP) of the right image stream most recently stored in the second buffer unit 202. Meanwhile, the buffer controller 203 may extract the time stamp of the most recently stored access unit instead of the GOP in order to calculate the buffer size more quickly.

Meanwhile, the buffer controller 203 may extract the first time stamp from the access unit of the left video stream recently stored in the decoder buffer. The video reproducing apparatus 102 may extract the second timestamp from the access unit of the right video stream recently stored in the decoder buffer. In FIG. 4, the access unit is located on the left side and recently stored, and the access unit is located on the right side, which means previously stored. In this case, the access unit of the left video stream from which the first timestamp as a reference is extracted and the access unit of the right video stream from which the second timestamp is extracted are received at the same time. Here, the decoder buffer may be expressed as ES buffer or Ts buffer.

The first time stamp and the second time stamp may be any one of a decoding timestamp (DTS), a presentation timestamp (PTS), an RTP timestamp, an SMTPE timecode, and a frame number. In this case, the time stamp extracted by the buffer controller 203 may be a decoding time stamp (DTS). However, if the decoding time stamp does not exist, the output time stamps PTS and RTP Timestamp may be extracted. In addition, when there is no time stamp, a frame identification number, an SMTPE timecode, or the like may be extracted.

5 is a diagram illustrating a process of controlling a buffer size according to an embodiment of the present invention.

The buffer controller 203 controls the buffer sizes of the first buffer unit 201 and the second buffer unit 202 by using the synchronization information, the first timestamp of the left video stream, and the second timestamp of the right video stream. can do.

The buffer controller 203 may determine the buffer sizes of the first buffer unit 201 and the second buffer unit 202. In this case, the buffer controller 203 may determine the buffer sizes of the first buffer unit 201 and the second buffer unit 202 using the difference in the reception time between the left video stream and the right video stream. In this case, the reception time difference may be generated because the left video stream and the right video stream transmitted from the video transmission device 101 are not simultaneously received by the video playback device 102.

The buffer controller 203 uses the synchronization information, the first timestamp of the left video stream, and the second timestamp of the right video stream to determine buffer delay times of the first buffer unit 201 and the second buffer unit 202. You can decide.

The buffer controller 203 may determine buffer delay times of the first buffer unit 201 and the second buffer unit 202. In this case, the buffer controller 203 may determine the buffer delay time between the first buffer unit 201 and the second buffer unit 202 using the difference in the reception time between the left video stream and the right video stream. In this case, the reception time difference may be generated because the left video stream and the right video stream transmitted from the video transmission device 101 are not simultaneously received by the video playback device 102.

The buffer controller 203 may calculate a reception time difference between the left video stream and the right video stream. For example, the buffer controller 203 may calculate a reception time difference using the synchronization information, the first timestamp of the left video stream, and the second timestamp of the right video stream. Extraction of the synchronization information, the first timestamp, and the second timestamp have been described above with reference to FIG. 4.

If the left video stream and the right video stream use the same reference clock, the buffer controller 203 may calculate the reception time difference using the time stamp difference between the first time stamp and the second time stamp.

If the left video stream and the right video stream use different reference clocks, the buffer controller 203 converts any one of the first timestamp and the second timestamp by using the synchronization information. The difference in reception time between the stamp and the second timestamp may be calculated. Meanwhile, when a random offset is applied to the first time stamp or the second time stamp, such as an RTP time stamp, the first time stamp and the second time stamp are converted by using either of the synchronization information and the first time stamp or the second time stamp. The difference in reception time between time stamps can be calculated. In detail, the buffer controller 203 may convert the first timestamp into the same form as the second timestamp or convert the second timestamp into the same form as the first timestamp using the synchronization information.

If the synchronization information is pair information of a timestamp, it may be converted as follows.

In one example, the buffer control section 203 calculates a time stamp difference (RTD_V) between the additional access is included in the stream of video unit (M-AU _n) of RTP_Timestamp (RTP_V _n) and synchronization information of the access unit RTP_Timestamp (RTP_Sync_V) of . Then, the buffer controller 203 may add the calculated time stamp difference RTD_V to the PTS (PTS_Sync) of the access unit as the synchronization reference, and perform a modulo operation at 2 ^ 32 as a result of the addition. Through this process, the buffer controller 203 may convert the RTP time stamp of each access unit included in the stream of the additional video into a time stamp (PTS of M-AU _n ) capable of calculating an output time point based on PCR. .

If RTP_V _n is smaller than RTP_Sync_V, the buffer controller 203 uses (RTP_V _n RTP_Sync_V) + 2 ^ 32 when calculating RTD_V to prevent the rollover of the time stamp. If RTP_Vn is greater than or equal to RTP_Sync_V, the buffer control unit 203 uses (RTP_V _n RTP_Sync_V)% 2 ^ 32 when calculating RTD_V.

If the synchronization information is offset information of a timestamp, it may be converted as follows.

When the frame rate of the left video stream and the right video stream for the 3DTV broadcasting service and the clock rates of the time stamps are the same, time stamp offsets of the left video stream and the right video stream may be used as synchronization information. For example, in fixed and mobile converged 3DTV systems, the frame rates of the left video stream and the right video stream are the same, and the clock rates of the MPEG-2 system PTS and the MDTV system RTP time stamp are the same. Accordingly, a time stamp offset method may be used, and the time stamp offset method uses, as synchronization information, a time stamp offset corresponding to an access unit of a left video stream and a right video stream that should be output at any same point in time for a 3DTV service. .

When the offset sign bit is 0, the buffer control unit 203 adds 2 ^ 32 to the result of adding the RTP_Timestamp (RTP_V _n ) of the access unit (M-AU _n ) of the right video stream to the time stamp offset of the access unit to be synchronized. Modulo operation can be performed. Then, the buffer controller 203 may calculate a time stamp (PTS of M-AU _n ) in which the output time point of the PCR reference of each access unit included in the right image stream can be calculated. Here, the offset sign bit of 0 means that the time stamp offset is positive, and 1 means that the time stamp offset is negative.

If the offset sign bit is 1 and RTP_V _n is greater than or equal to the time stamp offset, the buffer control unit 203 uses RTP_V _n timestamp_offset when calculating the PTS of the access unit included in the right video stream. If the offset sine bit is 1 and RTP_V _n is smaller than the time stamp offset, the buffer controller 203 uses RTP_V _n timestamp_offset + 2 ^ 32 when calculating the PTS of the access unit included in the right picture stream.

Then, the buffer control unit 203 converts the output time information (time stamp, frame identification number, SMTPE timecode, etc.) of one video stream into output time information of the other video stream by using the synchronization information. The difference can be calculated.

The following shows the process of calculating the reception time difference. Method 1 shows the case where the synchronization information is a timestamp pair, and method 2 shows the case where the synchronization information is offset information.

(Method 1)

(i) Sync information: Timestamp pair of left video stream and timestamp of right video stream (Left: 100, Right: 200)

(ii) Time stamps of the access unit of the left video stream and the access unit of the right video stream, which are recently stored in the first buffer unit 201 and the second buffer unit 202, respectively (left: 500, right: 800).

(iii) The result of converting the timestamp of the right video stream into the timestamp of the left video stream: 800-(200-100) = 700

 (iv) Difference in reception time between left video stream timestamp and converted right video stream timestamp: 500 (left)-700 (right) = -200

(v) Assuming that the clock rate of the timestamp of the left video stream is 100 Hz, since -200/100 = -2, the left video stream arrives 2 seconds before the right video stream.

(Method 2)

(i) Sync information: Timestamp offset between left video stream and right video stream: -100

(ii) Time stamps of the access unit of the left video stream and the access unit of the right video stream, which are recently stored in the first buffer unit 201 and the second buffer unit 202, respectively (left: 500, right: 800).

(iii) The result of converting the timestamp of the right video stream into the timestamp of the left video stream: 800 + (-100) = 700

(iv) Difference in reception time between left video stream timestamp and converted right video stream timestamp: 500 (left)-700 (right) = -200

(v) Assuming that the clock rate of the timestamp of the left video stream is 100 Hz, since -200/100 = -2, the left video stream arrives 2 seconds before the right video stream.

When calculating the reception time difference between the left video stream and the right video stream, when the output time stamps (PTS, RTP Timestamp) other than the decoding time stamp (DTS) are used, the decoding order and the buffer output order may not match. In this case, the buffer controller 203 may control the buffer size and the output time in consideration of the number of consecutive B frames in the video stream. For example, when the number of consecutive B frames in the video stream is 3, the first buffer unit 201 and the second video frame may include the received time difference calculated according to the output timestamp and the frames of the actual left and right video streams. A time difference input to each of the buffer units 202 may vary up to four. In this case, the buffer controller 203 may correspond to an error by adding a 4 frame transmission time to the previously calculated reception time difference.

Alternatively, the buffer controller 203 may control the buffer size and the output time in consideration of the GOP size. For example, when the output unit of the buffer is a GOP unit, the buffer controller 203 may correspond to an error by adding a frame transmission time equal to the number of frames constituting the GOP to the previously calculated reception time difference.

The buffer controller 203 may determine the buffer size by using a reception time difference between the left video stream and the right video stream. For example, the buffer controller 203 calculates the number of delayed frames between the left video stream and the right video stream by using the number of access units stored in the buffer, the difference in reception time, and the frame rate of the video stream. As a result, the buffer size that is dynamically set can be determined.

The buffer controller 203 may determine the buffer size by using a fixed bit rate and a reception time difference of the left video stream or the right video stream. Alternatively, the buffer controller 203 may determine the buffer size using a difference between the data rate and the reception time of the left video stream or the right video stream.

At this time, the reception time difference

Assume that Hereinafter, a process of calculating the buffer size using the reception time difference will be described.

(Method 1) When the number of access units of the left video stream stored in the first buffer unit 201 and the number of access units of the right video stream stored in the second buffer unit 202 are known

In this case, when the frame rate of the video stream is fr, the number of frames delayed between the left video stream and the right video stream is

to be. That is, the buffer control unit 203 is the first of the received video stream

You can store frames in a buffer and keep them. In other words, the size of the buffer that stores the received video stream is larger than the size of the buffer that stores the received video stream.

Can be as large as The buffer size can change dynamically.

(Method 2) When the first buffer section 201 and the second buffer section 202 use a fixed bit rate as the ES buffer

At this time, if the fixed bit rate is br, the buffer controller 203 first determines the maximum size of the buffer of the received video stream.

Can be maintained.

(Method 3): When the first buffer section 201 and the second buffer section 202 use the variable bit rate as the TS buffer

At this time, if the variable bit rate is tbr, the buffer control unit 203 first determines the maximum size of the buffer of the received video stream.

Can be maintained.

For example, the buffer controller 203 may convert the RTP timestamp, which is the second timestamp, into a PCR based program timestamp. The buffer controller 203 may extract a difference between the RTP timestamp and PTS% 2 ^ 32 converted according to the PCR, and determine the buffer size using the extracted difference. For example, the buffer controller 203 may calculate a buffer size according to Equation 1 below.

Here, timestamp_difference means a reception time difference based on a time stamp difference of a left video stream or a right video stream. If the RTP timestamp converted according to PCR is larger than PTS% 2 ^ 32, the buffer size of the second buffer unit 202 is determined by Equation 1, and the buffer size of the first buffer unit 201 is determined. Becomes the VBV buffer size. For example, when the RTP timestamp converted by PCR is 180,000 larger than PTS% 2 ^ 32, the buffer size of the second buffer unit 202 is Buffer_size = (180,000 / 90,000) * 19.39 Mbps + VBV buffer size = 38.78Mb + VBV buffer size can be determined.

In a similar manner, the buffer controller 203 may determine an output time point of the first buffer unit 201 and the second buffer unit 202 by using the reception time difference. Here, the output time point may correspond to the buffer delay time. In this case, the method of determining the output time point may be utilized even when using a fixed size buffer, unlike the method of dynamically determining the buffer size.

At this time, the buffer control unit 203 outputs a point in time at which the first buffer unit 201 transfers the reference image to the first decoder unit 204 and the second buffer unit 202 is added to the second decoder unit 205. An output time point for delivering an image may be determined. For example, the buffer controller 203 may extract a time stamp of a frame of a reference image most recently stored in the first buffer unit 201. The buffer controller 203 may extract a time stamp of a frame of the additional image most recently stored in the second buffer unit 202.

In this case, the buffer controller 203 may extract the PTS of the first frame of the group of picture (GOP) of the ATSC main stream of the reference video most recently stored in the first buffer unit 201. The buffer controller 203 may extract the RTP timestamp of the first frame of the GOPs of the M / H stream of the additional video most recently stored in the second buffer unit 202.

Here, the PTS (Presentation Timestamp) is a 33-bit time stamp transmitted on a PES header to synchronize the presentation time of the audio / video access unit (AU) in the ATSC main broadcasting network. Means. In this case, PES (Packetized Elementary Stream) refers to a packet obtained by dividing each ES into a specific length for transmission of an audio / video elementary stream (ES) in an ATSC main broadcasting network. At this time, the header of the PES includes the PTS. Meanwhile, RTP (Realtime Transport Protocol) refers to a standard for transmitting audio / video data through an IP network.

The buffer control unit 203 converts the RTP converted according to PCR based on the timestamp (RTP Timestamp and PTS) information extracted from the access unit most recently input to the first buffer unit 201 and the second buffer unit 202. By comparing the time stamp and PTS% 2 ^ 32, buffer delay times of the first buffer unit 201 and the second buffer unit 202 may be determined. For example, when the RTP timestamp converted by PCR is 180,000 smaller than PTS% 2 ^ 32, the RTP timestamp-based right video stream is 180,000 (difference) / 90,000 (clock rate) time than the PTS-based left video stream. It means that it is being received as soon as possible. Therefore, the buffer control unit 203 stores the right video stream in the second buffer unit 202 for 2 seconds and then outputs the ES to the left video stream and the right video stream simultaneously by outputting it to the second decoder 205. The data may be transmitted to the decoder 204 and the second decoder 205.

6 is a diagram illustrating a process of correcting a reference clock or correcting a time stamp according to one embodiment of the present invention.

Referring to FIG. 6, T1 represents a time for receiving the access unit AU1 of the left video stream, and T2 represents a time for receiving an access unit AU1 of the right video stream. And,

Represents T2-T1. In FIG. 6, T2 is larger than T1, which indicates that the left video stream is received by the video reproducing apparatus 102 before the right video stream. On the other hand, it is assumed that the reference clock is signaled through the transmission network of the left video stream.

Then, the reference clock may be corrected according to Equation 2 below.

Here, the reference clock Y means the corrected reference clock, and the reference clock X means the original reference clock. According to Equation 2, the image reproducing apparatus 102 that provides the 3DTV service may use the reference clock Y obtained by subtracting the result of combining the reception time difference and the clock rate with the reference clock X. For example, suppose that the reference clock X is 127,000,000, the reception time difference is 1 second, and the clock rate of the reference clock is 27,000,000 Hz. The reference clock Y = 127,000,000-27,000,000 = corrected PCR of the reference clock X before 1 second. 100,000,000 is available.

According to Equation 2, the reference clock corrected for the reference clock representing T2 is T1. That is, when the reference clock is corrected, an effect of delaying the reference clock by the time from when one video stream is received until the other video stream is received.

The PTS, DTS, and RTP timestamp indicating the output time of the right video stream may be changed to a form in which a reference clock of the left video stream may be used using the signaled synchronization information. The time information of the changed right video stream and the corrected reference clock may be used for decoding and outputting a buffer of the right video stream. For example, since the result of converting the time stamp of the access unit 1 (AU1) of the right video stream is T1, the access unit 1 (AU1) of the right video stream is the same as the access unit 1 (AU1) of the left video stream. Can be output. On the other hand, when T1 is larger than T2, since the right video stream is already stored in the buffer at the time of receiving the left video stream, correction of the reference clock may be unnecessary.

On the other hand, when the reception delay is corrected by correcting the time stamp instead of the correction method of the reference clock, the time stamp of the left and right images may be corrected according to Equation 3 below.

Herein, the frame time information Y means a corrected time stamp, and the frame time information X means an original time stamp. Instead of calibrating the reference clock, you can correct the timestamp to use the transmitted reference clock. Specifically, all time stamps (eg, PTS, DTS, RTP Timestamp) related to the access unit of the left video stream and the right video stream need to be corrected by adding the received time difference. For example, when the reception time difference is 1 second and the clock rates of the PTS, DTS, and RTP time stamps of the video stream are all 90,000 Hz, the video reproducing apparatus 102 corrects the derived result by adding 90,000 to the original time stamp. Time stamps are available.

In the case of correcting the time stamp in FIG. 6, the result of correcting the frame time information indicating T1 represents T2. That is, the correction of the frame time information has the effect that the frame time information is delayed by the time from when one video stream is received until the other video stream is received. In the above-described example, the frame time information of the right video stream, PTS, DTS, RTP Timestamp, etc., is changed into a form in which the reference clock of the left video stream can be used by using the signaled synchronization information, and the changed time stamp and the original reference clock are used. (Left video reference clock) can be utilized for decoding the right video stream and outputting the buffer. Here, the original reference clock refers to the reference clock of the left video stream. When T1 is larger than T2, since the right video stream is already stored in the buffer at the time of receiving the left video stream, correction of the frame time information may be unnecessary.

7 is a flowchart illustrating a process of controlling a buffer size according to an embodiment of the present invention.

In operation 701, the image reproducing apparatus 102 may acquire synchronization information between the left image stream and the right image stream. For example, the image reproducing apparatus 102 obtains synchronization information including timestamp pair information including a first timestamp of a left video stream and a second timestamp of a right video stream, which should be simultaneously output at any particular point in time. can do. In addition, the image reproducing apparatus 102 may obtain synchronization information including offset information between the first timestamp of the left video stream and the second timestamp of the right video stream.

In operation 702, the video reproducing apparatus 102 may extract a first timestamp of the left video stream and a second timestamp of the right video stream. For example, the image reproducing apparatus 102 may extract the first time stamp and the second time stamp based on accesses of the left video stream and the right video stream received at the same time. Alternatively, the image reproducing apparatus 102 may extract the first timestamp of the left video stream and the second timestamp of the right video stream extracted at a close time point through demultiplexing.

In operation 703, the video reproducing apparatus 102 may calculate a difference in reception time between the left video stream and the right video stream using a first time stamp, a second time stamp, and synchronization information. For example, if the left video stream and the right video stream use the same reference clock, the image reproducing apparatus 102 may calculate the reception time difference using the time stamp difference between the first time stamp and the second time stamp. . If the left video stream and the right video stream use different reference clocks or random offsets are applied, the video reproducing apparatus 102 uses either the first time stamp or the second time stamp using the synchronization information. And convert the received time difference using the time stamp difference between the first time stamp and the second time stamp.

In operation 704, the image reproducing apparatus 102 may determine the buffer size by using the reception time difference. For example, the image reproducing apparatus 102 calculates the number of delayed frames between the left image stream and the right image stream by using the number of access units stored in the buffer, the difference in reception time, and the frame rate of the video stream, and the calculated number of frames. You can determine the dynamically set buffer size. As another example, the image reproducing apparatus 102 may determine the buffer size by using a fixed bit rate and a reception time difference of the left image stream or the right image stream. As another example, the image reproducing apparatus 102 may determine the buffer size by using a difference between the data rate and the reception time of the left video stream or the right video stream.

8 is a flowchart illustrating a process of determining a buffer delay time according to an embodiment of the present invention.

In operation 801, the image reproducing apparatus 102 may acquire synchronization information between the left image stream and the right image stream. For example, the image reproducing apparatus 102 obtains synchronization information including timestamp pair information including a first timestamp of a left video stream and a second timestamp of a right video stream, which should be simultaneously output at any particular point in time. can do. In addition, the image reproducing apparatus 102 may obtain synchronization information including offset information between the first timestamp of the left video stream and the second timestamp of the right video stream.

In operation 802, the image reproducing apparatus 102 may extract a first timestamp of the left video stream and a second timestamp of the right video stream. For example, the image reproducing apparatus 102 may extract the first time stamp and the second time stamp based on the access unit of the left video stream and the access unit of the right video stream received at the same time. Alternatively, the image reproducing apparatus 102 may extract the first timestamp of the left video stream and the second timestamp of the right video stream extracted at a close time point through demultiplexing.

In operation 803, the image reproducing apparatus 102 may calculate a difference in reception time between the left video stream and the right video stream using a first time stamp, a second time stamp, and synchronization information. For example, if the left video stream and the right video stream use the same reference clock, the image reproducing apparatus 102 may calculate the reception time difference using the time stamp difference between the first time stamp and the second time stamp. . If the left video stream and the right video stream use different reference clocks or random offsets are applied, the video reproducing apparatus 102 uses either the first time stamp or the second time stamp using the synchronization information. And convert the received time difference using the time stamp difference between the first time stamp and the second time stamp.

In operation 804, the image reproducing apparatus 102 may determine the buffer delay time by using the reception time difference. For example, when the left image stream is received before the right image stream, the image reproducing apparatus 102 may increase the buffer delay time of the right image stream by a difference in the reception time. In addition, when the right video stream is received before the left video stream, the video reproducing apparatus 102 may increase the buffer delay time of the left video stream by a difference in the reception time.

9 is a flowchart illustrating a process of calibrating a reference clock according to an embodiment of the present invention.

In operation 901, the image reproducing apparatus 102 may calculate a reception time difference between the access unit of the left image stream and the access unit of the right image stream.

In operation 902, the image reproducing apparatus 102 may correct the reference clock signaled to the first received image stream among the left image stream and the right image stream by using the reception time difference. For example, the image reproducing apparatus 102 may correct the reference clock by using a result of applying a reception time difference to the clock rate of the reference clock.

10 is a diagram illustrating a process of correcting a time stamp according to one embodiment of the present invention.

In operation 1001, the image reproducing apparatus 102 may calculate a reception time difference between the access unit of the left video stream and the access unit of the right video stream.

In operation 1002, the image reproducing apparatus 102 may correct the first timestamp of the left video stream and the second timestamp of the right video stream by using the reception time difference. For example, the image reproducing apparatus 102 may correct the first timestamp of the left video stream and the second timestamp of the right video stream by using a result of applying a reception time difference to the clock rate of the reference clock.

11 is a diagram illustrating a process of synchronizing a left video stream and a right video stream according to an embodiment of the present invention.

In operation 1101, the image reproducing apparatus 102 may acquire synchronization information between the left image stream and the right image stream.

In operation 1102, the image reproducing apparatus 102 may extract a first timestamp of the left video stream and a second timestamp of the right video stream.

In operation 1103, the image reproducing apparatus 102 may convert at least one of the first time stamp and the second time stamp to correspond to the reference clock by using the synchronization information.

In operation 1104, the image reproducing apparatus 102 may match and synchronize the left image stream and the right image stream according to the reference clock.

Methods according to an embodiment of the present invention can be implemented in the form of program instructions that can be executed by various computer means and recorded in a computer readable medium. The computer readable medium may include program instructions, data files, data structures, etc. alone or in combination. Program instructions recorded on the media may be those specially designed and constructed for the purposes of the present invention, or they may be of the kind well-known and available to those having skill in the computer software arts.

As described above, the present invention has been described by way of limited embodiments and drawings, but the present invention is not limited to the above embodiments, and those skilled in the art to which the present invention pertains various modifications and variations from such descriptions. This is possible.

Therefore, the scope of the present invention should not be limited to the described embodiments, but should be determined not only by the claims below but also by the equivalents of the claims.

Claims (20)

1. Obtaining synchronization information between the left video stream and the right video stream;

   Extracting a first timestamp of the left video stream and a second timestamp of the right video stream;

   Calculating a reception time difference between the left video stream and the right video stream using the first timestamp, the second timestamp, and the synchronization information; And

   Determining a buffer size using the reception time difference

   A buffer control method of an image reproducing apparatus comprising a.
2. The method of claim 1,

   Acquiring the synchronization information,

   A method of controlling a buffer of a video reproducing apparatus, comprising: synchronizing information including timestamp pair information including a first timestamp of a left video stream and a second timestamp of a right video stream to be output simultaneously at any particular time;
3. The method of claim 1,

   Acquiring the synchronization information,

   A buffer control method of a video reproducing apparatus for obtaining synchronization information including offset information between a first timestamp of a left video stream and a second timestamp of a right video stream.
4. The method of claim 1,

   Extracting the first time stamp of the left video stream and the second time stamp of the right video stream,

   A buffer control method of a video reproducing apparatus for extracting a first time stamp and a second time stamp based on a most recently stored access unit of an access unit of a left video stream and an access unit of a right video stream received at the same time.
5. The method of claim 1,

   Extracting the first time stamp of the left video stream and the second time stamp of the right video stream,

   A buffer control method of a video reproducing apparatus, which extracts a first timestamp of a left video stream and a second timestamp of a right video stream extracted at a close time point through demultiplexing.
6. The method of claim 1,

   The step of calculating the difference in reception time,

   If the left video stream and the right video stream use the same reference clock, the difference in reception time is calculated using the time stamp difference between the first time stamp and the second time stamp.

   If the left video stream and the right video stream use different reference clocks or random offsets are applied, one of the first time stamp and the second time stamp is converted by using the synchronization information, and the first time stamp and the first time stamp are used. 2. A buffer control method of a video reproducing apparatus for calculating a difference in reception time using a time stamp difference between two timestamps.
7. The method of claim 1,

   Determining the buffer size,

   The number of delayed frames between the left video stream and the right video stream is calculated using the number of access units stored in the buffer, the difference in reception time, and the frame rate of the video stream, and the buffer size dynamically set according to the calculated number of frames is determined. A method of controlling a buffer of a video reproducing apparatus.
8. The method of claim 1,

   Determining the buffer size,

   The buffer control method of the image reproducing apparatus for determining the buffer size by using the fixed bit rate and the reception time difference of the left or right video stream.
9. The method of claim 1,

   Determining the buffer size,

   The buffer control method of the image reproducing apparatus for determining the buffer size by using the difference between the data rate and the reception time of the left video stream or right video stream.
10. Obtaining synchronization information between the left video stream and the right video stream;

    Extracting a first timestamp of the left video stream and a second timestamp of the right video stream;

    Calculating a reception time difference between the left video stream and the right video stream using the first timestamp, the second timestamp, and the synchronization information; And

    Determining a buffer delay time by using the reception time difference

    A buffer control method of an image reproducing apparatus comprising a.
11. The method of claim 10,

    Determining the buffer delay time,

    If the right video is received before the left video, the buffer delay time of the right video stream is increased by the difference in the reception time.

    If the left video stream is received before the right video stream, the buffer control method of the video playback apparatus to increase the buffer delay time of the left video stream by the difference in the reception time.
12. Calculating a reception time difference between the access unit of the left video stream and the access unit of the right video stream; And

    Correcting a reference clock signaled to a first received video stream among a left video stream and a right video stream by using the reception time difference

    Clock correction method of the image reproducing apparatus comprising a.
13. The method of claim 12,

    Correcting the reference clock,

    And correcting the reference clock by using a result of applying a reception time difference to the clock rate of the reference clock.
14. Calculating a reception time difference between the access unit of the left video stream and the access unit of the right video stream; And

    Correcting a first timestamp of a left video stream and a second timestamp of a right video stream by using the reception time difference.

    Time stamp correction method of the video playback device comprising a.
15. The method of claim 14,

    Correcting the first timestamp of the left video stream and the second timestamp of the right video stream,

    A method of correcting a time stamp of a video reproducing apparatus for correcting a first timestamp of a left video stream and a second timestamp of a right video stream using a result of applying a reception time difference to a clock rate of a reference clock.
16. Obtaining synchronization information between the left video stream and the right video stream;

    Extracting a first timestamp of the left video stream and a second timestamp of the right video stream;

    Converting at least one of the first timestamp and the second timestamp to correspond to a reference clock using synchronization information; And

    Synchronizing by matching a left video stream and a right video stream according to the reference clock;

    Synchronization method of the image playback device comprising a.
17. A first buffer unit for receiving and storing a left image stream;

    A second buffer unit for receiving and storing a right image stream; And

    A buffer controller for determining a buffer size of the first buffer unit and the second buffer unit by using a difference in reception time between the left image stream and the right image stream

    Including,

    The reception time difference is,

    And a first time stamp of the left video stream, a second time stamp of a right video stream, and synchronization information between a left video stream and a right video stream.
18. A first buffer unit for receiving and storing a left image stream;

    A second buffer unit for receiving and storing a right image stream; And

    A buffer controller for determining a buffer delay time of a first buffer unit or a second buffer unit by using a difference in reception time between the left image stream and the right image stream

    Including,

    The reception time difference is,

    And a first time stamp of the left video stream, a second time stamp of a right video stream, and synchronization information between a left video stream and a right video stream.
19. A first buffer unit for receiving and storing a left image stream;

    A second buffer unit for receiving and storing a right image stream; And

    (I) correcting a reference clock signaled to the first received video stream of the left video stream and the right video stream by using the difference in the reception time between the access unit of the left video stream and the access unit of the right video stream, or (ii) the left video A buffer controller for correcting the first timestamp of the stream and the second timestamp of the right video stream

    Image playback device comprising a.
20. A first buffer unit for receiving and storing a left image stream; And

    A second buffer unit for receiving and storing a right image stream;

    At least one of a first timestamp of the left video stream and a second timestamp of the right video stream is converted according to a reference clock using the synchronization information between the left video stream and the right video stream, and the left video stream and the right video stream according to the reference clock. Image synchronizer for matching and synchronizing video streams

    Image playback device comprising a.

Images