KR20110062914A - Method for assinging timestamps to the video frames with non-increasing order - Google Patents

Abstract

PURPOSE: A time stamp allocation method for a video service having non-increasing frame order is provided to automatically calculate a time stamp by using a frame indicator when a video of a hierarchical frame rate structure is encoded. CONSTITUTION: A plurality of frames that are decoded in a non-increasing frame order based video encoding method is received. Time stamps are calculated by using frame rate indicators corresponding to each frame(S420). The time stamp of a current frame is added on a connection list(S440). The time stamp of the calculated current frame is stored according to the frame rate indicators. All time stamps are deleted if the current frame rate indicator is equivalent to the maximum frame rate(S460).

Description

METHODE FOR ASSINGING TIMESTAMPS TO THE VIDEO FRAMES WITH NON-INCREASING ORDER}

The present invention relates to a time stamp allocation method for a video service having a non-sequential frame order, and more particularly, to display video content having a non-sequential frame order among video contents encoded in a frame unit on a screen in a complete order. It is about a method.

The present invention is derived from a study conducted as part of the IT growth engine technology development project of the Ministry of Knowledge Economy and the Ministry of Information and Telecommunication Research and Development. [Task management number: 2008-S-006-02, Title: Open IPTV in wired and wireless environment] 2.0) Technology Development].

In general, a video serviced through various media has a form of a moving image showing several still frames in sequence within a short time. In addition, in the case of digital video technology, redundant information between frames displayed for a short time is appropriately removed. To this end, video data is compressed into small data that can be transmitted in real time. This compression process is called digital video encoding.

Recently, various digital video encoding techniques have been studied to improve the compression ratio. In particular, a non-sequential frame order-based encoding technique is a technique for arranging frames in a different order from time.

In the video coding based on the non-sequential frame order, the order of the frames that are actually encoded and output is arranged differently from the time order of the originally input frames. In this case, when decoding the encoded video data, the order of the plurality of decoded frames is different from the time order to be displayed on the actual screen. That is, the time order of the frame originally input before the encoding is different from the order of the frames decoded and output.

Accordingly, a method of adjusting and displaying the order of the frames decoded and output according to the time order to be displayed on the actual screen is required.

In a method of assigning a time stamp according to an embodiment of the present invention, receiving a plurality of frames decoded by a non-sequential frame order based video encoding technique, and using a frame rate indicator corresponding to each of the plurality of frames, the plurality of frames Computing a time stamp corresponding to each, and adding the calculated time stamp of the current frame to the connection list.

The method may further include storing the calculated time stamp of the current frame for each layer of the frame rate indicator.

In this case, the calculating may include calculating a time stamp of the current frame by using a time stamp increment corresponding to the frame rate indicator of the current frame and a time stamp corresponding to the stored frame rate indicator of the current frame.

The method may further include updating a time stamp corresponding to the frame rate indicator of the current frame among time stamps stored for each layer to the calculated time stamp of the current frame.

In addition, when the calculated time stamp of the current frame exists in the linked list, the calculating step may be performed by adding the calculated time stamp of the current frame and the time stamp increment corresponding to the frame rate indicator of the current frame to obtain a time stamp of the current frame. Can be recalculated.

Also, when the calculated frame rate indicator of the current frame corresponds to the highest frame rate, the method may further include deleting all time stamps less than the calculated time stamp of the current frame among the time stamps added to the linked list. .

The apparatus for assigning a time stamp may further include a calculator configured to calculate a time stamp corresponding to each of the plurality of frames by using a frame rate indicator corresponding to each of the plurality of frames decoded by the non-sequential frame order based video encoding technique, and the calculated unit. It may include a list control unit for adding the time stamp of the current frame to the connection list.

The video encoding method may further include encoding a plurality of frames using an out-of-order frame order based video encoding technique, and transmitting a frame rate indicator indicating an order of each of the frames output through the encoding together with the plurality of encoded frames. It may include a step.

The present invention can automatically calculate a time stamp using a frame indicator in video coding based on out of order frame order and video coding having a hierarchical frame rate structure.

In addition, the frames decoded and output through the time stamp may be displayed in an order that should be displayed on the actual screen.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings.

However, in describing the present invention, when it is determined that the detailed description of the related air technology or configuration may unnecessarily obscure the subject matter of the present invention, the detailed description thereof will be abbreviated or omitted.

1 is a diagram provided to explain video encoding having a hierarchical frame rate structure.

First, referring to FIG. 1, FIG. 1 is a diagram illustrating an order of frames input over time for video encoding. Here, the order of frames input for video encoding may indicate the order of frames to be displayed on the screen.

For example, in the case of encoding 13 frames as shown in FIG. 1, the time stamp may be allocated to each frame while increasing from 0 to 12 in order of the frames. Accordingly, time stamp 12 may be allocated to the last 13th frame. Here, the time stamp may be information indicating a time order in which each of the plurality of frames is displayed on the actual screen.

In this case, when encoding 13 frames, the time stamp may be allocated to each of the 13 frames as a real number or an integer multiple of an integer value such as 0 to 12.

For example, when each of the plurality of frames has 30 output intervals per second, and 1/30 second is set to an integer interval of 3000, the time stamp assigned to each of the plurality of frames may be “integer value × 3000”. . That is, in the case of the last frame, a time stamp may be assigned to "12 x 3000".

In this case, the plurality of frames may be encoded by a non-sequential frame order based video encoding technique having a hierarchical frame rate structure. Then, according to FIG. 1, each of 13 frames encoded and output may include a corresponding frame rate indicator. Here, the frame rate indicator may be information indicating which frame rate the current frame corresponds to.

For example, when having four hierarchical frame rate structures, the frame rate indicator may be preset as shown in Table 1 below. That is, as shown in Table 1, the frame rate indicators T0 to T3 may have a hierarchical structure according to the frame rate.

Frame rate indicator Actual frame rate T0 5 frames / sec T1 10 frames / sec T2 20 frames / sec T3 60 frames / sec

Referring to Table 1, when frames corresponding to all frame rate indicators from T0 to T3 are transmitted, frames output through encoding may have a frame rate of 60 frames per second.

In addition, referring to FIG. 1, when frames corresponding to frame rate indicators T0 to T2 except T3 are transmitted, the frame rate may have a frame rate of 20 frames per second. Similarly, when transmitting frames corresponding to the frame rate indicators T0 and T1 except for T2, the frame rate may have a frame rate of 10 frames per second. Finally, when transmitting a frame corresponding to the frame rate indicator T0, it may have a frame rate of 5 frames per second.

In this case, an actual frame rate corresponding to each frame rate indicator may be set based on the number of frames corresponding to each layer of the frame rate indicator.

For example, as shown in Table 1, when T0 is set to 5 frames / sec, two frames corresponding to time stamps 0 and 12 may exist between the frame rate indicator T0 and the frame rate indicator T1 in FIG. 1. Accordingly, the actual frame rate corresponding to the frame rate indicator T1 may be set to 5 frames / sec × 2 = 10 frames / sec.

In the same manner, when three frames exist between the frame rate indicator T0 and the frame rate indicator T1, the actual frame rate corresponding to the frame rate indicator T1 may be set to 5 frames / sec × 3 = 15 frames / sec. .

Similarly, each of the actual frame rates corresponding to the frame rate indicators T1 to T3 may be set based on the number of frames included between the upper layer and itself.

FIG. 2 is a diagram provided to explain an encoding order using video encoding having a hierarchical frame rate structure.

13 frames in FIG. 1 may be encoded in the frame order as shown in FIG. 2 to have a hierarchical frame structure according to the frame rate. In this case, a scalable video encoding technique may be used to have a hierarchical frame structure according to the frame rate.

For example, referring to FIG. 1, a frame of time stamp 0 may be encoded, and a frame of time stamp 12 may be encoded by predicting a motion based on the encoded frame 0. FIG. Then, a frame corresponding to time stamp 6 may be encoded by predicting a motion based on the encoded frames 0 and 12. FIG. In the same manner, a frame corresponding to time stamp 3 may be encoded by predicting a motion based on the encoded frames 0 and 6.

In addition, a frame corresponding to time stamp 1 may be encoded by predicting motion based on the encoded frames 0 and 3, and a frame corresponding to time stamp 2 may be encoded based on 1 and 3 frames. . In the same manner, frames corresponding to times 7 to 11 may be encoded.

Then, the output order of the encoded frames is as shown in FIG. Referring to FIGS. 1 and 3, in FIG. 1, frames sequentially input from 0 to 12 in encoding may be encoded in a hierarchical structure and may be output out of order as shown in FIG. 3. That is, the order of frames input before encoding may be changed as shown in FIG. 3 through encoding.

In this case, each of the encoded and output frames may include a frame rate indicator to calculate a time stamp of each of the encoded and output frames. Here, referring to FIG. 3, time stamps of frames encoded and output may be 0 to 12. In addition, according to FIG. 3, the frame rate indicator included in each of the encoded and output frames may be T0 to T3.

Hereinafter, a method of automatically calculating a time stamp of each frame by using a frame rate indicator included in each of the encoded and output frames will be described with reference to FIGS. 4 and 5. That is, a method of calculating time stamps 0 to 12 corresponding to each of 13 frames using the frame rate indicators T0 to T3 of FIG. 3 will be described.

4 is a flowchart provided to explain a time stamp assignment method according to an embodiment of the present invention, and FIG. 5 is a diagram illustrating a configuration of an apparatus for assigning a time stamp according to an embodiment of the present invention. According to FIG. 5, the time stamp assignment apparatus 500 may include a receiver 510, a calculator 530, a list controller 550, and a storage 570.

First, the receiver 510 may receive decoded frames based on an out of order frame order having a hierarchical frame rate structure. In this case, the decoded frames may include a frame rate indicator corresponding to each. Here, the frame rate indicator may be information indicating which frame rate the current frame corresponds to, as described with reference to Table 1 below.

In more detail, the frame rate indicator may be included in each of the frames encoded at the time of encoding and transmitted from the encoding apparatus to the decoding apparatus. That is, the encoding apparatus may encode a plurality of frames by an out-of-order frame order based video encoding technique having a hierarchical frame rate structure. In this case, the encoding apparatus may transmit a frame rate indicator indicating the order of the encoded and output frames together with the encoded frames to the decoding apparatus.

Then, the decoding apparatus may extract a frame rate indicator corresponding to each of the plurality of frames by decoding the encoded plurality of frames (S410). Then, the time stamp assignment apparatus may receive the extracted frame rate indicator and the plurality of decoded frames from the decoding apparatus.

In operation S420, the calculator 530 may calculate a time stamp of the current frame using the time stamp increment TS _GTK corresponding to the received frame rate indicator of the current frame and the frame rate indicator of the current frame. Here, the time stamp increment may be preset as shown in Table 2 for each hierarchical frame rate indicator.

Frame rate indicator Timestamp increment T0 12 T1 6 T2 3 T3 One

In more detail, the calculator 530 adds the time stamp increment TS _GTK and the time stamp TSN _{TK _ Old} corresponding to the frame rate indicator of the current frame stored in the storage unit 570 to determine the time of the current frame. The stamp can be calculated. As an example, if the frame rate indicator is four (K = 3), using the formula shown in Figure 7, the calculation section 530 is the time stamp of the frame corresponding to the frame rate indicator T0 to T3 each TSN _T0 to TSN _T3 can be calculated.

In this case, as illustrated in FIG. 6, the storage unit 570 includes spaces TSN _{T0 _R} , for storing time stamps corresponding to each of the frame rate indicators having a hierarchical structure. TSN _{T1 _R} , TSN _{T2 _R} and TSN _{T3 _R} .

In addition, the time stamp corresponding to the frame rate indicator of the current frame among the time stamps stored in the storage unit 570 may be updated with the time stamp of the current frame calculated by the calculator 530. Through this, the storage unit 570 may store the latest time stamp for each layer of the frame rate indicator.

For example, as shown in FIG. 3, 13 frames having a frame rate indicator in the order of "T0, T0, T1, T2, T3, T3, T3, T3, T2, T3, T3, T3, T3" are received from the decoding apparatus. In this case, since each layer of the frame rate indicator stored in the storage unit 570 is initialized to zero, the time stamp of the first frame may be zero.

Then, the storage unit 570 may be initialized to [0] [0] [0] [0] for each layer of the frame rate indicator. That is, the storage unit 570 may store time stamps for each layer of the frame rate indicator. In this case, time stamp 0 of the first frame may be added to the connection list.

Subsequently, the calculation unit 530 calculates a time stamp increment TSG _TO 12 corresponding to the frame rate indicator T0 of the second frame and a time stamp TSN corresponding to the frame rate indicator T0 of the second frame stored in the storage 150. _{TO _R} ) 12, which is the sum of 0, may be calculated as the time stamp TSN _TO of the second frame.

Then, the list controller 550 may update the time stamp corresponding to the frame rate indicator T0 of the second frame stored in the storage 150 to the calculated time stamp of the second frame. That is, the storage unit 570 may update the time stamp 0 corresponding to the frame rate indicator T0 to 12 with [12] [0] [0] [0].

In this case, when the calculated time stamp of the current frame does not exist in the linked list (S430: NO), the list controller 550 may add the calculated time stamp of the current frame to the linked list (S440).

In addition, when the frame rate indicator of the current frame corresponds to the highest frame rate (S450: YES), the list controller 550 may delete all values smaller than the calculated time stamp of the current frame from the linked list (S460).

If the frame rate indicator of the current frame does not correspond to the highest frame rate (S450: NO), the list controller 550 may add the time stamp of the current frame to the linked list without deleting the time stamps from the linked list. .

For example, when only the time stamp 0 of the first frame is present in the linked list, since the time stamp 12 of the second frame does not exist in the linked list, the list controller 550 may add 12 to the linked list. At this time, the time stamp may be added to the connection list in ascending order. Then, the linked list may include 0 and 12 in ascending order (0-> 12).

In the same way, the calculator 530 may calculate the time stamps of the third and fourth frames as 6 and 3, respectively. Then, the storage unit 570 may update the time stamp for each layer of the frame rate indicator to [12] [6] [3] [0]. In this case, 6 and 3 may be added to the connection list to include time stamps in the order of 0-> 3-> 6-> 12.

In addition, the calculator 530 may calculate the time stamp of the fifth frame as one. Then, the storage unit 570 may be updated to [12] [6] [3] [1], and 1 may be added to the connection list.

In this case, since the frame rate indicator T3 of the fifth frame is the highest frame rate indicator, the list controller 550 may delete a value 0 smaller than the time stamp 1 of the fifth frame from the linked list. Then, the connection list may include time stamps in the order of 1-> 3-> 6-> 12.

On the other hand, in step S430, if the time stamp of the current frame exists in the linked list (S430: YES), the calculation unit 530 increments the time stamp corresponding to the calculated time stamp of the current frame and the frame rate indicator of the current frame. In operation S470, the time stamp of the current frame may be recalculated.

For example, referring to FIG. 3, through a time stamp calculation up to the sixth frame, the linked list includes the time stamps in the order of 2-> 3-> 6-> 12, and calculates the time stamp of the seventh frame. In this case, the calculator 530 may calculate the time stamp of the seventh frame as 3 according to the method described above.

At this time, since the calculated time stamp 3 of the seventh frame exists in the linked list, the calculator 530 calculates the time stamp increment 1 corresponding to the frame rate indicator T3 of the seventh frame from the calculated time stamp 3 of the seventh frame. In addition, the time stamp of the seventh frame may be recalculated to four.

Then, the storage unit 570 may be updated to [12] [6] [3] [4]. Also, the list controller 550 may add 4 to the linked list. In this case, as described in operation 450, since the frame rate indicator T3 of the seventh frame corresponds to the highest frame rate, the list controller 550 may delete all time stamps having a value smaller than 4 from the linked list. That is, 2 and 3 may be deleted from the linked list. Then, the linked list may include time stamps in the order of 4-> 6-> 12.

Referring to FIG. 3, through the same method as described in steps S420 to S470, the calculation unit 530 sets the time stamps of each of the 13 frames to “0, 12, 6, 3, 1, 2, 4, 5, 9, 7, 8, 10, 11 ". That is, time stamps of respective frames decoded using the frame rate indicators of the order "T0, T0, T1, T2, T3, T3, T3, T3, T2, T3, T3, T3, T3" are set to "0, 12". , 6, 3, 1, 2, 4, 5, 9, 7, 8, 10, 11 ".

As described above, the time stamp assignment apparatus 400 may calculate time stamps corresponding to each of the plurality of decoded frames in the order of frames encoded and output by the non-sequential frame order based video encoding scheme of FIG. 3.

In addition, in FIG. 4, after S440 is performed, steps S450 and S460 are described. However, S440 may be performed after S450 and S460 are performed. That is, when the frame rate indicator of the current frame corresponds to the highest frame rate, the time stamp of the current frame may be added to the linked list after deleting values smaller than the time stamp of the current frame from the linked list.

In the above, for convenience of description, a method of allocating time stamps using four frame rate indicators having a hierarchical structure from T0 to T3 has been described, but this corresponds to an embodiment and has a hierarchical structure. The frame rate indicators may be preset to less than four or more than four. In this way, the number of frame rate indicators is adjusted so that the hierarchical frame rate structure can be increased or decreased.

In the above description, the time stamp corresponding to each frame is calculated by using frames encoded to have a hierarchical frame rate structure. However, this is an embodiment, and the hierarchical structure is based on conditions other than the frame rate. The time stamp corresponding to each of the frames encoded to have can be automatically calculated.

As described above, the present invention has been described by way of limited embodiments and drawings, but the present invention is not limited to the above embodiments, and those skilled in the art to which the present invention pertains various modifications and variations from such descriptions. This is possible.

Therefore, the scope of the present invention should not be limited to the described embodiments, but should be determined not only by the claims below but also by the equivalents of the claims.

1 is a diagram provided to explain video encoding having a hierarchical frame rate structure.

FIG. 2 is a diagram provided to explain an encoding order using video encoding having a hierarchical frame rate structure.

3 is a diagram illustrating a frame order output during video encoding having a hierarchical frame rate structure.

4 is a flowchart provided to explain a time stamp allocation method according to an embodiment of the present invention.

5 is a diagram illustrating a configuration of an apparatus for assigning time stamps according to an embodiment of the present invention.

FIG. 6 is a diagram illustrating a structure in which time stamps corresponding to each of the frame rate indicators having a hierarchical structure are stored.

7 is a diagram illustrating an equation for calculating a time stamp using a frame rate indicator.

<Explanation of symbols for the main parts of the drawings>

410: receiver

430: calculation unit

450: list control unit

470: storage unit

Claims (18)

Receiving a plurality of frames decoded by an out of order frame order based video encoding technique; Calculating a time stamp corresponding to each of the plurality of frames by using a frame rate indicator corresponding to each of the plurality of frames; And Adding the calculated time stamp of the current frame to a linked list Time stamp assignment method comprising a. The method of claim 1, Storing the calculated time stamp of the current frame for each layer of the frame rate indicator Time stamp assignment method further comprising. The method of claim 2, The calculating step, And calculating a time stamp of the current frame using a time stamp increment corresponding to the frame rate indicator of the current frame and a time stamp corresponding to the frame rate indicator of the stored current frame. The method of claim 3, wherein Updating a time stamp corresponding to the frame rate indicator of the current frame among the time stamps stored for each layer to the calculated time stamp of the current frame Time stamp assignment method further comprising. The method of claim 3, wherein The calculating step, If the calculated time stamp of the current frame exists in the linked list, the time stamp of the current frame is recalculated by adding the calculated time stamp of the current frame and a time stamp increment corresponding to the frame rate indicator of the current frame. Time stamp assignment method. The method of claim 3, wherein Adding to the linked list, And adding the recalculated time stamp of the current frame to the linked list. The method of claim 6, If the frame rate indicator of the current frame corresponds to the highest frame rate, deleting all time stamps below the calculated time stamp of the current frame among the time stamps added to the linked list; Time stamp assignment method further comprising. The method of claim 1, The calculating step, And calculating a time stamp corresponding to each of the plurality of frames in the order of frames encoded and output by the non-sequential frame order based video encoding technique. The method of claim 1, And the encoded plurality of frames has a hierarchical frame rate structure. The method of claim 1, And the time stamp is information indicating a time sequence in which each of the plurality of frames is displayed on a screen. A calculator configured to calculate a time stamp corresponding to each of the plurality of frames by using a frame rate indicator corresponding to each of the plurality of frames decoded by an out-of-order frame order based video encoding technique; And A list controller for adding the calculated time stamp of the current frame to a linked list Time stamp assignment device comprising a. The method of claim 11, A storage unit for storing the calculated time stamp of the current frame for each layer of the frame rate indicator More, The calculation unit, And calculating a time stamp of the current frame by using a time stamp increment corresponding to the frame rate indicator of the current frame and a time stamp corresponding to the frame rate indicator of the current frame stored in the storage unit. The method of claim 11, The calculation unit, If the calculated time stamp of the current frame exists in the linked list, the time stamp of the current frame is recalculated by adding the calculated time stamp of the current frame and a time stamp increment corresponding to the frame rate indicator of the current frame. and, The list control unit, And adding the recalculated time stamp of the current frame to the connection list. The method of claim 11, The list control unit, And when the calculated time stamp of the current frame does not exist in the linked list, adding the calculated time stamp of the current frame to the linked list. The method of claim 14, The list control unit, And when the frame rate indicator of the current frame corresponds to the highest frame rate, deleting at least one time stamp smaller than the calculated time stamp of the current frame from the connection list. Encoding a plurality of frames using an out of order frame order based video encoding technique; And Transmitting a frame rate indicator indicating an order of each of the frames output through the encoding together with the encoded plurality of frames. Video encoding method comprising a. The method of claim 16, The step of encoding, And encoding the plurality of frames to have a hierarchical frame rate structure. The method of claim 16, The hierarchical frame rate structure is increased or decreased by adjusting the number of frame rates corresponding to the frame rate indicator.

Images