JP3174042B2 - Time decoding method for B-VOP - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention is useful for encoding digital audiovisual materials that require a plurality of independently encoded audiovisual objects to be synchronized for presentation. The invention is particularly useful when the temporal sampling of audiovisual material is not the same.

[0002]

2. Description of the Related Art In MPEG1 and MPEG2,
The input video consists of image frames sampled at regular time intervals. This represents the finest temporal resolution of the input video. FIG. 1 shows a video sequence at a constant frame rate in which image frames are sampled at regular intervals. In the encoded representation of a video sequence using the MPEG1 and MPEG2 standards, the display order of the decoded frames is represented by a temporal reference. This parameter is described in the picture header of the bit stream syntax. The value of this parameter is incremented by one for each decoded frame when checking the display order.

[0003] H. In the H.263 standard, frames may be skipped and, therefore, variable frame rate video sequences can be decoded. However, sampling of the frames remains unchanged. Thus, the time reference method used in MPEG1 and MPEG2 is still appropriate and only needs to be modified to increment by (1 + the number of non-transferred pictures at the input frame rate) instead of incrementing by one. is there.

[0004] At present, research and development have been made to encode video as an independent object in a multiplexed video image. This represents a new concept in the decoding and synchronization of each video object. These individual video objects may originate from multiple sources,
It is also expected that they may have completely different frame rates. Some of the objects may have a substantially continuous temporal sampling rate. When these video target images are combined and displayed, they become a composite image. Therefore, some kind of synchronization is required for this synthesis. The display frame rate may be different from the frame rate of any video object image. FIG. 2 shows an example of two video objects having different frame rates. Even if a common frame rate between the two video objects could be found, the frame rate would not necessarily be the same as the output frame rate of the synthesis processor.

[0005] The following describes the problem in the video domain, but the same principles of the invention can be extended to the audio domain and also to the combined domain of the two.

It is clear from the above that the current situation in the art does not meet the requirements for synchronization of video objects. Also, the current state of the art does not provide a common reference time when different video objects have different frame rates that are not multiples of each other.

The first problem is how to provide a common local time base mechanism for each video object. This time base is based on very fine temporal granularity
It must be able to provide (temporal granularity) and also be able to deal with the fact that there can be very long intervals between two successive instances of the video object picture.

[0008] A second problem is how to provide a mechanism for synchronizing video objects having different frame rates.

[0009]

The above-mentioned problem can be solved by introducing a common time resolution used for all local time bases. To provide a wide range of temporal granularity, the local time base is divided into two different parts. The first part includes a temporal resolution with a fine granularity that provides a short time reference. The second part includes a temporal resolution with a coarse grain that provides a long time reference. The short time reference is included in each video object picture and provides a reference time at the time of the video object picture. Then, this short time reference is synchronized with the long time reference common to all video object pictures. This long time reference is used to synchronize all the various video objects to a common time reference provided by the master clock.

[0010]

According to the present invention, there is provided a method of decoding the time of a B-VOP (VOP encoded by bidirectional prediction) included in compressed data, wherein
A modulo time base increment that represents an increment in seconds and a VOP time base increment that represents an increment that is less than 1 second, from the compressed data, the I immediately preceding the B-VOP in display order.
The time of 1 second unit of -VOP (intra-coded VOP) or P-VOP (predicted-coded VOP) is obtained,
Decoding the modulo time reference of the B-VOP, and decoding the VOP time reference increment of the B-VOP, at the obtained time in seconds, the decoded modulo time reference increment and the decoded VOP time Add the result of adding the reference increment to the B-VOP
This is a time decoding method of the VOP with the time of.

[0011]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS The present invention operates by using two time display formats. The first time display format is:
This is a short time reference added to the video target image. Less than,
This time reference is called a VOP (Video Object Plane, ie, a video target image) time increment. This VOP time increment serves as the timing used for the video object in conjunction with the long time reference added to the group of video objects that are decoded and combined together. This long time reference is called a modulo time reference. And these VOPs
The time increment and the modulo time base are used in conjunction to determine the actual time base to use for synthesizing the video object image into the final composite sequence for display.

To facilitate editing the bitstream and combining different video objects from different sources into a new group of video objects, the local time of each video object is determined from a common time reference. A third component is needed that provides a constant offset value to the reference. Hereinafter, this offset is referred to as a VOP time offset. This prevents different target pictures from having to be synchronized with a granularity equal to the modulo time base interval. This component should be constant for each video object in the group to which the video objects being multiplexed together belong.

First, the modulo time reference will be described.

The modulo time reference represents the coarse resolution of the local time reference. It does not have a value like the VOP time increment. In fact, it is a VOP
This is a more important synchronization mechanism for synchronizing the time increment with the local time reference of the video object image. It is placed as a marker in the encoded bitstream,
Indicates that the VOP time increment of the following video object must be reset, and that the reference to the local time reference must be incremented in units of one or more modulo time reference intervals. FIGS. 3, 4, 5 to 10 and 1
At 1, the modulo time base is represented in the bitstream header by a continuous "1" inserted before the VOP time increment followed by a "0". The number of consecutive "1" s is zero or more. The number of "1s" inserted into the bitstream depends on the number of units on a modulo time base that has elapsed since the last I-VOP or P-VOP. In the encoder and decoder, the modulo time base counter is incremented by one each time a "1" is detected. The modulo time base counter is finite in length, so in a real system the modulo time base is reset to zero if it exceeds its maximum value. In a typical video sequence, video objects form a group of VOPs. Therefore,
The modulo time base is typically reset at the start of this VOP group.

Next, the VOP time increment will be described.

The VOP time increment must be in units that can utilize the shortest temporal sampling of the video object picture. It may be a negative time reference used for the target image. It therefore represents the finest temporal resolution granularity required or the finest temporal resolution granularity available.

The VOP time increment may then be represented by a finite length number that is greater than or equal to the ratio of the global time reference interval / the local time reference resolution. FIG. 3 shows the V used for I and P-video object pictures.
An example of OP time increment and reference to a modulo time reference is shown. An absolute time reference is used. The VOP time increment is reset each time a modulo time reference is detected. FIG. 4 shows another example using I, P, and B-video object pictures. The operation is the same, except that the modulo time base is similarly repeated in the B-video object image. If the modulo time base is not repeated the same way in the B-video image, ambiguities will arise due to differences in decoding and presentation order. This is described in more detail below.

Since the VOP time increment corresponds to the presentation time reference, a potential problem arises when the encoding order is different from the presentation order. This is caused by the B-video object picture. MPEG-1
And MPEG-2 B-pictures,
-Video object pictures are coded after their reference video object pictures, even if their presentation order is before the reference I-video object picture and the reference P-video object picture. Since the VOP time increment is finite and relative to the modulo time reference, the VOP time increment is reset if a modulo time reference is detected. However, the order of encoding for B-video objects remains delayed. 5 to 8 show possible ambiguities. It is not possible to determine when the VOP time increment should be reset. In fact, FIG.
Given a sequence of events to be encoded as shown in FIG. 6, it is not possible to know which timing position in FIGS. 6, 7 and 8 is to be represented. This problem arises because of the use of a single modulo time base that is shared by all different types of video object pictures, where different coding orders and presentation orders are mixed. There is nothing that can be done about the encoding order. This is because this reference information is required by the B-video object image. Also, it is not preferable that each of the different prediction modes has its own modulo time reference.

Next, the VOP time offset will be described.

In addition to the above, the modulo time base is shared among all video objects. This means that synchronization between different video object pictures has a granularity equal to the modulo time base interval. This is especially unacceptable when video objects from different groups are combined to form a new group of video objects. FIG. 11 shows an example of two different video objects encoded with two different local time bases offset from each other. As described above, when these video target images are multiplexed, the synchronization of the video target images is also shifted. Finer granularity is achieved by having a VOP time offset in each of the individual video objects. This means that only this value changes when the video object image is manipulated and multiplexed. Not only does the VOP time increment need not be changed, but different video objects can be multiplexed without using coarse-grained timing differences.
FIG. 11 illustrates the use of this time reference offset.

The preferred embodiment of the present invention comprises a method for encoding a time reference used for each of the individual video object picture bit streams, a method for multiplexing different video object pictures into a common time reference, It includes a method of demultiplexing the multiplexed bit stream into components, and a method of reproducing a time reference from the component bit stream.

Next, time-based coding will be described.

A flowchart of an embodiment for encoding a time reference is shown in FIG. In the encoder, in step 1, a local time reference is first initialized to a local start time. Processing moves to step 2, where the encoder determines the current value of the local time reference. In step 3, the obtained local time reference is compared with a pre-coded modulo time reference to check if the interval exceeds the modulo time reference interval. If so, control transfers to step 4, where the required number of modulo time bases are inserted into the bitstream. If the interval is not exceeded, no special processing is required. The process then proceeds to step 5, where the VOP time increment is inserted into the bitstream. Next, in step 6, the target image is encoded and inserted into the bit stream. The encoder then performs a check in step 7 to determine if there are any more target pictures to be coded. If there is a target image to be coded, the process returns to step 2 where a local time reference is obtained. If there is no target image to be coded, the process ends.

To determine the absolute and relative VOP time increments for each of the I / P-video object picture and the B-video object picture, the following equations are used. t _GTBn = n × t _GTBI + t _GTB0 (n = 0,1,2,3, ...) (1) t _AVTI = t _{ETBI / P} -t _GTBn (2) t _RVTI = t _ETBB -t _{ETBI / P} ( 3) where t _GTBn is the encoder time reference represented by the nth coded modulo time reference.

T _GTBI is a predetermined modulo time reference interval.

T _GTB0 is a start time based on the encoder time.

T _AVTI is an absolute VOP time increment for an I or P-video object picture.

T _{ETBI / P} is an encoder time reference at the start of encoding an I or P-video object picture.

T _RVTI is a relative VOP time increment with respect to the B-video target picture.

T _ETBB is an encoder time reference at the start of encoding of the B-video object picture. Next, multiplexing of a plurality of video target images will be described.

When a plurality of video target pictures are multiplexed into one, the multiplexing device examines the bit stream of the multiplexed video target picture to determine not only the synchronization but also the multiplexing order. The operations involved are shown in FIG. In step 11, the VOP time offset for each video object to be multiplexed is inserted into the bitstream. Next, in step 12, all bit streams of the video objects to be multiplexed are examined to determine whether all the video objects are at their respective modulo time bases. If so, processing proceeds to step 13, where a common modulo time base is inserted into the multiplexed bit stream. If not, processing proceeds to step 14, where the next encoded video object is inserted into the multiplexed bit stream. In step 15, the bit stream of the video object to be multiplexed is again checked for more video objects to be multiplexed. If so, control proceeds to step 12 again. If not, the process ends.

Next, the demultiplexing of a bit stream including a plurality of video images will be described.

FIG. 14 shows the demultiplexing of the bit stream containing the multiplexed video image. This process is Step 2
Starting at 1, where the VOP time offset is decoded and sent to the decoder for use in synchronization. And
In step 22, the multiplexed bit stream is inspected to determine if a modulo time base has been detected. If a modulo time base is detected,
Processing proceeds to step 23, where a modulo time base is inserted into all video subject bitstreams. If a modulo time base is not detected, processing proceeds to step 24, where the next video object is examined and inserted into the appropriate video object image bitstream. Finally, the multiplexed bit stream is examined again to determine if there are any more video objects to demultiplex. If so, the process returns to step 22
Proceed to. If not, the process ends.

Next, time-based reproduction will be described.

An embodiment for reproducing the time reference is shown in FIG. When playing back the local time reference, the process starts at step 31, where the local time reference is initialized taking into account the VOP time offset decoded by the demultiplexer. The process then proceeds to step 32, where the bitstream is examined to determine whether the modulo time base has been decoded. If the modulo time reference has been decoded, processing proceeds to step 33, where the local time reference is incremented by an increment of the modulo time reference. Then, the process proceeds to step 37. If the modulo time base has not been decoded, the process proceeds to step 34, where the video object is examined to determine if it is a B-video object. If it is a B-video target picture, the process proceeds to step 35, where a decoding time reference of the B-video target picture is calculated based on equation (6). Then, the process proceeds to step 37. If the result of step 34 is B
If it is not a video object image, the process proceeds to step 36, where a decoding time reference is calculated based on equation (5). Then, the process proceeds to step 37. In step 37, the bitstream is examined to determine if there are any more video objects to decode. If so, the process proceeds to step 32 again. If not, the process ends.

The following formula is used to determine the presentation timestamp of the video object image. t _GTBn = n × t _GTBI + t _GTB0 (n = 0,1,2,3, ...) (4) t _{DTBI / P} = t _AVTI + t _GTBn (5) t _DTBB = t _RVTI + t _{DTBI / P} (6) Here, t _GTBn is the decoding time reference represented by the nth decoded modulo time reference.

T _GTBI is a predetermined modulo time reference interval.

T _GTB0 is a start time based on the decoding time.

T _{DTBI / P} is a decoding time reference at the starting point of decoding of an I or P-video target picture.

T _AVTI is the decoded absolute VOP time increment for the I or P-video object picture.

T _DTBB is a decoding time reference at the start point of decoding of the B-video object picture.

T _RVTI is the decoded relative VOP time increment for the B-video object picture.

Next, an embodiment of the bit stream encoder will be described.

FIG. 16 shows the modulo time base and VOP
FIG. 4 is a block diagram illustrating an embodiment of a bitstream encoder for encoding a time increment. For this explanation, the example shown in FIG. 4 is used. Since bi-directional prediction is used, the order of encoding is different from the order of presentation shown in FIG. The encoding order is B-
Before the VOP, an I-VOP followed by a P-VOP
Started from. This is explained in the following three paragraphs.

Step 41 is an initializer.
, Where the bitstream encoder starts by initializing the local time base register to the initial value of the time code. The same time code value is encoded in the bitstream. At the start of the encoding of the next I-VOP, step 42, which is a time code comparator,
Compares the presentation time of the I-VOP with the local time reference register. The result is sent to step 43, which is a modulo time base coder. The modulo time base encoder inserts the required number of "1s" into the bitstream equal to the number of elapsed modulo time base increments.
This is followed by the symbol "0" to indicate the end of the modulo time reference code. The local time base register is updated with the current modulo time base. Processing then proceeds to step 44, which is a VOP time-based incremental encoder, where the remainder of the I-VOP presentation time code is encoded.

This process is repeated for the next encoded video object picture which is a P-VOP. Step 42, which is a time code comparator, compares the presentation time of the P-VOP with the local time reference register. The result is sent to step 43, which is a modulo time base coder. The modulo time base encoder inserts the required number of "1" s equal to the number of elapsed modulo time base increments. This is followed by the symbol "0" to indicate the end of the modulo time reference code. The B-VOP time reference register is set to the value of the local time reference register, and the local time reference register is updated to the current modulo time reference. Processing then proceeds to step 44, a VOP time-based incremental encoder, where the remainder of the P-VOP presentation time code is encoded.

This process is repeated for the next video object picture to be encoded, which is a B-VOP. Step 42, which is a time code comparator, compares the presentation time of the B-VOP with the B-VOP time reference register.
Step 43 where the result is a modulo time base coder
Sent to The modulo time base encoder inserts the required number of "1" s equal to the number of elapsed modulo time base increments. This is followed by the symbol "0" to indicate the end of the modulo time reference code. Both the B-VOP time reference register and the local time reference register
-Not changed after VOP processing. The process then proceeds to step 44, which is a VOP time-based incremental encoder,
Therefore, the remaining part of the presentation time code of the B-VOP is encoded.

The local time reference register stores the next VOP
Reset at the next I-VOP representing the beginning of the group.

Next, an embodiment of the bit stream decoder will be described.

FIG. 17 is a block diagram illustrating an embodiment of a decoder used for modulo time reference and VOP time increment for reproducing a presentation time stamp. As in the encoder embodiment, the example shown in FIG. 4 is used. The order of decoding is the same as the order of encoding: before the B-VOP, the I-VOP
And the subsequent P-VOP are decoded. This is explained in the following paragraphs.

Step 51 is an initializer.
, Where the local time reference register is set to the value of the time code decoded from the bitstream. Processing then proceeds to step 52, which is a modulo time base decoder, where the modulo time base increment is decoded. The total number of modulo time base increments to be decoded is given by the number of “1” s that are decoded before the symbol “0”. Next, the VOP time reference increment is VO
It is decoded in step 53 which is a P time reference incremental decoder. In step 54, which is a time reference calculator, the presentation time of the I-VOP is reproduced. The sum of the decoded modulo time base increment is added to the local time base register. Then, the VOP time reference increment is added to the local time reference register,
The presentation time of the I-VOP is obtained. The process then proceeds to step 55, which is a video object picture decoder, where the video object picture is decoded.

For the P-VOP, the process is repeated at step 52, which is a modulo time base decoder, where the modulo time base increment is decoded. The total number of modulo time base increments to be decoded is given by the number of “1” s that are decoded before the symbol “0”. Next, the VOP time reference increment is decoded in step 53, which is a VOP time reference increment decoder. In step 54, which is a time reference calculator, the presentation time of the P-VOP is reproduced. The B-VOP modulo time reference register is set to the value of the local time reference register. The sum of the decoded modulo time base increment is added to the local time base register. And
The VOP time reference increment is added to the local time reference register to obtain a P-VOP presentation time. Processing proceeds to the video target picture decoder, where the video target picture is decoded.

For a B-VOP, the process is repeated in step 52, which is a modulo time base decoder, where the modulo time base increment is decoded. The total number of modulo time base increments to be decoded is given by the number of “1” s that are decoded before the symbol “0”. Next, the VOP time reference increment is decoded in step 53, which is a VOP time reference increment decoder. In step 54, which is a time reference calculator, the presentation time of the B-VOP is reproduced. The sum of the decoded modulo time base increment and the VOP time base increment is B-
This is added to the VOP time reference register to obtain the presentation time of the B-VOP. Both the B-VOP time reference register and the local time reference register remain unchanged. Processing then proceeds to a video target picture decoder, where the video target picture is decoded.

The local time reference register stores the next VOP
Reset at the next I-VOP representing the beginning of the group.

Next, a specific example will be described.

Referring to FIG. 18, there is shown an example of the step of encoding the compressed data into bit stream data. As shown in the upper row of FIG. 18, the compressed video data VOP is composed of I1, B1, B
2, P1, B3, P2 are arranged in a line, and GOP
A (group of picture) header is inserted at the start of the VOP group. The local time at which the display is performed and at which the display is performed is determined for each VOP using the local time clock. For example,
The first VOP (I1-VOP) is displayed at 1: 23: 45: 350 milliseconds (1: 23: 45: 350) counted from the very beginning of the video data, and the second VOP is displayed.
The OP (B1-VOP) is displayed at 1: 23: 45: 750, and the third VOP (B2-VOP) is displayed at 1: 23: 45: 750.
23: 46: 150, and so on.

In order to encode VOPs, it is necessary to insert display time data into each VOP.
If the time data is to be inserted completely, including hours, minutes, seconds, and milliseconds, a significant data area is required in the header portion of each VOP. The aim of the present invention is to reduce such data area,
Another object is to simplify time data to be inserted into each VOP.

Each of the VOPs shown in the top row of FIG. 18 stores display time data consisting of milliseconds in the VOP time increment area. Also, the VO in the top row
Each of P also temporarily stores display time data including hours, minutes, and seconds. The GOP header is the first VOP
Display data consisting of hours, minutes and seconds used for (I1-VOP) is stored.

As shown in the second row of FIG.
The VOP is delayed by a predetermined time using a buffer (not shown). According to the bidirectional prediction method, when a VOP is generated from a buffer, the order of the VOPs changes.
Should be located after the P-VOP referenced by that B-VOP. Therefore, the VOP is I1, P1, B
1, B2, P2, and B3 are arranged in a line.

As shown in the third row of FIG.
At time T1, ie, when the GOP header is just encoded, when stored in the GOP header,
The minute and second data are directly stored in the local time reference register. In the example shown in FIG. 18, the local time reference register stores 1:23:45. Then, before time T2, bit stream data corresponding to the GOP header is obtained, and hour, minute, and second data are inserted as shown in the lower part of FIG.

At time T2, the first VOP
(I1-VOP) is captured. The time code comparator is
The time stored in the local time reference register (hour, minute,
Second) is compared with the time (hour, minute, second) temporarily stored in the first VOP (I1-VOP). According to this example,
The comparison result is the same. Thus, the comparator generates a "0" indicating that the first VOP (I1-VOP) occurred at the same second as is held in the local time reference register. “0” generated by the comparator is directly added to the modulo time reference area of the first VOP (I1-VOP). At the same time, the first VOP
The hour, minute and second data temporarily stored in (I1-VOP) are removed. Therefore, before time T3, bit stream data corresponding to the first VOP (I1-VOP) is obtained, "0" is inserted in the modulo time reference area, and "350" is inserted in the VOP time increment area. Is done.

Next, at time T3, the second VOP
(P1-VOP) is captured. The time code comparator is
The time stored in the local time reference register (hour, minute,
Second) is compared with the time (hour, minute, second) temporarily stored in the second VOP (P1-VOP). According to this example,
The result of the comparison is that the time temporarily stored in the second VOP (P1-VOP) is one second greater than the time stored in the local time reference register. Thus, the comparator generates a "10" indicating that the second VOP (P1-VOP) occurred one second after the second held in the local time reference register. If the second
The comparator generates "110" if the VOP (P1-VOP) occurs in the second after the second held in the local time reference register.

After the time T3, a time equal to the time held in the local time reference register immediately before the time T3 is set in the B-VOP time reference register. In this example, the B-VOP time reference register contains:
1:23:45 is set. After time T3, the local time reference register stores the second VO
It is incremented at a time equal to the time temporarily stored in P (P1-VOP). Thus, in this example, the local time reference register is 1:23:
It is incremented to 46.

The resultant "10" generated by the comparator is directly used as the second VOP (P1-VOP).
Is given to the modulo time reference area of. At the same time,
The hour, minute, and second data temporarily stored in the second VOP (P1-VOP) are removed. Therefore, time T
Before 4, the bit stream data corresponding to the second VOP (P1-VOP) is obtained, "10" is inserted in the modulo time base area, and "550" is
Inserted in the time increment area.

Then, at time T4, the third VOP
(B1-VOP) is captured. The time code comparator is
The time stored in the B-VOP time reference register (hour,
The minutes (minutes, seconds) are compared with the times (hours, minutes, seconds) temporarily stored in the third VOP (B1-VOP). According to this example, the result of the comparison is the same. Therefore, the comparator
It generates "0" indicating that the third VOP (B1-VOP) occurred at the same second as the second held in the B-VOP time reference register. The resulting "0" generated by the comparator is directly used as the third VOP (B
1-VOP). At the same time, the hour, minute and second data temporarily stored in the first VOP (I1-VOP) are removed. Therefore, before the time T5, the third VOP (B1-
VOP) is obtained,
“0” is inserted into the modulo time base area, and “750” is inserted.
Is inserted into the VOP time increment area.

Then, at time T5, the fourth VOP
(B2-VOP) is captured. The time code comparator is
The time stored in the B-VOP time reference register (hour,
The minutes (minutes, seconds) are compared with the times (hours, minutes, seconds) temporarily stored in the fourth VOP (B2-VOP). According to this example, the result of the comparison is that the time temporarily stored in the fourth VOP (B2-VOP) is one second greater than the time stored in the B-VOP time reference register. Therefore, the comparator determines that the fourth VOP (B2-VOP) is B
-1 next to the second held in the VOP time reference register
Generate a "10" that represents what happened in seconds.

While processing a B-type VOP,
Regardless of what result the comparator produces, neither the local time reference register nor the B-VOP time reference register is incremented.

The resulting “10” generated by the comparator is the fourth VOP (B2-VOP)
Is given to the modulo time reference area of. At the same time,
The hour, minute, and second data temporarily stored in the fourth VOP (B2-VOP) are removed. Therefore, time T
Before 6, the bit stream data corresponding to the fourth VOP (B2-VOP) is obtained, "10" is inserted into the modulo time base area, and "150" is the VOP.
Inserted in the time increment area.

At time T6, the fifth VOP
(P2-VOP) is captured. The time code comparator is
The time stored in the local time reference register (hour, minute,
Second) is compared with the time (hour, minute, second) temporarily stored in the fifth VOP (P2-VOP). According to this example,
The result of the comparison is that the time temporarily stored in the fifth VOP (P2-VOP) is one second greater than the time stored in the local time reference register. Thus, the comparator generates a "10" indicating that the fifth VOP (P2-VOP) occurred one second after the second held in the local time reference register.

After time T6, the B-VOP time reference register is incremented to a time equal to the time held in the local time reference register immediately before time T6. In this example, the B-VOP time reference register is incremented at 1:23:46.
Further, after time T6, the local time reference register is incremented to a time equal to the time temporarily stored in the fifth VOP (P2-VOP).
Therefore, in this example, the local time reference register is incremented at 1:23:47.

The result "10" generated by the comparator is directly used as the fifth VOP (P2-VOP).
Is given to the modulo time reference area of. At the same time,
The hour, minute and second data temporarily stored in the fifth VOP (P2-VOP) are removed. Therefore, time T
Before bitstream No. 7, bit stream data corresponding to the fifth VOP (P2-VOP) is obtained, "10" is inserted in the modulo time base area, and "350" is set in the VOP.
Inserted in the time increment area.

Thereafter, the same processing is performed, and bit stream data for the subsequent VOPs is formed.

In order to decode the bit stream data, a process reverse to the above-described process is executed. First, the time (hour, minute, second) held in the GOP header
Is read. The read time is stored in the local time reference register.

When an I type or P type VOP, that is, a VOP other than the B type is received, the data stored in the modulo time reference area is read. If the read data is "0", that is, if there is no 1 before 0, the local time reference register is not changed. Also, the B-VOP time reference register is not changed. If the read data is "10", the time stored in the local time reference register is 1
Incremented by seconds. If the read data is "110", the time stored in the local time reference register is incremented by 2 seconds. Thus, the number of seconds to be incremented is determined by the number of ones inserted before zero. When the read data is "10" or "110", the B-VOP time reference register, which is a memory, copies the time held by the local time reference register immediately before the increment. Then, the time (hour, minute, second) held in the local time reference register is combined with the time (millisecond) held in the VOP time increment area, and the time at which an I-type or P-type VOP is to be generated is determined. Is determined.

When a B-type VOP is received, the data stored in the modulo time reference area is read. If the read data is "0", the time (hour, minute, second) held in the B-VOP time reference register is VO.
Combined with the time (milliseconds) held in the P time increment area, the time at which a B-type VOP should be generated is determined. If the read data is "10", B
The time (hour, minute,
Seconds) is added with one second, and the time obtained by the addition is combined with the time (milliseconds) held in the VOP time increment area to determine the time at which a B-type VOP is to be generated. If the read data is "110", the time (hour, minute, second) held in the B-VOP time reference register is added by 2 seconds, and the time obtained by the addition is added to the VOP time increment area. In combination with the time (milliseconds) held in the time period, the time at which a B-type VOP is to be generated is determined.

An advantage of the present invention is that video objects encoded by different encoders can be multiplexed. Further, the present invention facilitates manipulating compressed data obtained from different sources based on a target image to generate a new bit stream. The present invention
A method for synchronizing audiovisual objects is provided.

Although the present invention has been described above, what has been described above can be modified in various forms. Such modifications do not depart from the spirit and scope of the invention and, as will be apparent to those of ordinary skill in the art, all such modifications are intended to be covered by the appended claims. is there.

[Brief description of the drawings]

FIG. 1 is a diagram illustrating temporal sampling according to the prior art in which frames of a video sequence are sampled at regular intervals.

FIG. 2 is a diagram illustrating the concept of a video object image and the relationship between them. The sampling of the video object image may be irregular, and the sampling period may change rapidly.

FIG. 3 is a diagram illustrating the present invention in which a reference time of a video object image is represented by a modulo time reference and a VOP time increment. In this description, only I-VOPs and P-VOPs are used.

FIG. 4 is a diagram illustrating the present invention in which a reference time of a video object image is represented by a modulo time reference and a VOP time increment. In this description, I-VOP, P-VOP, and B-VOP are used.

FIG. 5 is a diagram illustrating an example of ambiguity that may occur when the presentation order and the encoding order are different for a B-video object image.

FIG. 6 is a diagram illustrating an example of ambiguity that may occur when the presentation order and the encoding order are different for a B-video object image.

FIG. 7 is a diagram illustrating an example of ambiguity that may occur when the presentation order and the encoding order are different for a B-video object image.

FIG. 8 is a diagram illustrating an example of ambiguity that may occur when the presentation order and the encoding order are different for a B-video object image.

FIG. 9 is a diagram illustrating how to resolve ambiguity by using an absolute time reference and a relative time reference.

FIG. 10 is a diagram illustrating how to resolve ambiguity by using an absolute time reference and a relative time reference.

FIG. 11 is a diagram illustrating a combination of two VOPs and synchronizing them with a common time reference by using a VOP time offset.

FIG. 12 is a flowchart illustrating time-based encoding.

FIG. 13 is a flowchart illustrating multiplexing of a plurality of video target images.

FIG. 14 is a flowchart illustrating demultiplexing of a plurality of video object images.

FIG. 15 is a flowchart illustrating reproduction of a presentation time stamp.

FIG. 16 is a block diagram illustrating the operation of a bit stream encoder for encoding a time reference.

FIG. 17 is a block diagram illustrating the operation of a bit stream decoder for decoding a time reference.

FIG. 18 is a time chart for explaining formation of bit stream data.

Of the front page Continued (72) inventor Chak Joo Lee, Singapore 449287 Singapore, lag Na Park, Marine Parade Road 15-13 number, block 5000 di (58) investigated the field (Int.Cl. ^7, DB name ) H04N 7/24-7/68

Claims (1)

(57) [Claims] 1. A method for decoding the time of a B-VOP (VOP encoded by bidirectional prediction) included in compressed data, wherein the compressed data is a modulo time base representing an increment of one second. An increment and a VOP time base increment that represents an increment of less than 1 second, and from the compressed data, an I-VOP (intra-coded VOP) or a P-VOP immediately before the B-VOP in display order.
(Predicted coded VOP) obtains the time in units of 1 second, decodes the modulo time reference of the B-VOP, decodes the VOP time reference increment of the B-VOP, and obtains the obtained 1 second A VOP time decoding method, wherein a result obtained by adding the decoded modulo time reference increment and the decoded VOP time reference increment to a unit time is the B-VOP time.