DESCRIPTION
METHOD AND APPARATUS FOR EDITING DATA STREAMS
Technical Field This invention relates to a method and apparatus for editing data streams, more particularly to a method and apparatus for merging a plurality of data streams.
Background Art Motion Picture Expert Group (MPEG) is one of the international standards for dynamic image compression, and was established by the International Standard Organization (ISO) for video and audio information compression. It is also a standard specifying the format of data streams. A data stream is a representation of video and audio data using a series of "0" and "1." In order to render such a series of "0" and "1" meaningful, the series of "0" and "1" has to be encoded. Generally, a data stream includes three different types of frames or pictures, namely, I frames (Intra-coded Picture), P frames (Predictive-coded Picture), and B frames (Bi- directionally Predictive-coded Picture). A group of pictures (GOP) is a combination of the aforesaid frames. I frames are encoded without reference to a preceding frame or a following frame, whereas P frames and B frames are encoded with motion compensation. In addition, a P frame is encoded with
reference to a preceding I frame or P frame, whereas a B frame is encoded with reference to immediately preceding and immediately following I or P frames. Therefore, the I frame has the largest amount of data, whereas the B-frame has the least amount of data. Thus, during decoding, a video buffer verifier (VBV) buffer in a decoder will allocate a relatively large buffer area to an I frame, and will allocate a relatively small buffer area to a B frame. Referring to Figure 1, when a decoder is decoding a group of pictures, a buffer disposed in the decoder will, with the input of the frames, allocate buffer areas to the frames, respectively. The vertical lines represent the sizes of the buffer areas allocated to the frames by the buffer. Thus, the vertical lines for the l-frames which are allocated with the largest buffer areas are longest. The oblique lines in the figure have the same slope. The slope of the oblique lines represents the data transmission rate at which the data of the frames is inputted into the buffer. If the sum of the buffer amount allocated to the frames and the original buffer size is greater than the largest buffer amount, overflow of the buffer will result. However, if the sum of the consumed (allocated) buffer amount and the original buffer size is less than the least buffer amount, underflow of the buffer will result. Buffer underflow in turn will result in a situation where the buffer cannot provide any data when it is desired to acquire frame data from the buffer during the course of data processing. Additionally, the display order of a group of pictures is different from the processing order thereof. The display order
is the arrangement sequence of the group of pictures when displayed on a screen, whereas the processing order is the sequence of processing of the group of pictures during decoding of the group of pictures. As shown in Figure 2, the display order of a group of pictures is IBBPBBP, and the frames to which the display order correspond are numbered 0, 1, 2...6, in sequence. However, the processing order of the group of pictures becomes IPBBPBB, and the numbers corresponding to the processing order are 0, 3, 1, 2, 6, 4, and 5. This is because the decoding of B frames requires reference to the data of the I and P frames in order to completely decode the B frames. Therefore, the processing order is to decode the I frames before decoding the P frames, and to decode the B frames by referring to the decoded I and P frames. When editing data streams, it is often necessary to delete frames in the front half, frames in the rear half, or a part of middle frames of a data stream, or to connect two different data streams after deleting a part of the frames. When there are changes in the frames in a data stream, buffer underflow will result upon buffering of the data stream through a buffer in the decoder due to discontinuity of the data stream. The causes for the underflow will be explained in detail hereinbelow. Since decoding of a P or B frame requires reference to a preceding frame, they cannot be the start frame of a data stream. During editing of a data stream, if the front part of the data is deleted, the first frame of the new data stream may be a P or B frame due to the site of the splicing point. Therefore, the data stream must be re-encoded to change the first frame
into an I frame. However, since an I frame will consume a relatively large number of bits in the buffer, the buffer must compensate the increased number of bits. However, during the process of compensation, if the buffer amount is less than the least buffer amount of the buffer, the buffer will have an underflow situation. When the rear part of the data of a data stream is deleted, it should be noted whether the last frame in the new data stream refers to a frame after the splicing point. If such is the case, it is necessary to re-encode the data stream so that the frames in the data stream will not refer to any frame after the splicing point. In the prior art, when an encoded data stream is being edited, in order to avoid the occurrence of underflow when an edited data stream passes through a buffer, a conventional method is to reduce the buffer amount allocated by the buffer to the I frames during decoding. However, this will seriously affect the picture quality of the decoded pictures. Since P and B frames have to refer to I frames, if the picture quality of the I frames is poor, the overall picture quality will be reduced. Another conventional method is described in U.S. Patent No. 6529555 B1. According to this method, data streams containing I, P, and B frames are all re-encoded to contain I frames only, and the number of the frames that need to be re- encoded is lengthened so that the buffer amount that the buffer intends to reduce can be distributed to a larger number of frames so as to resolve buffer underflow to thereby maintain the picture quality at an even level. However, because the
number of frames required to be re-encoded is large, this method takes a considerable amount of processing time. Besides, since this method requires augmentation of the number of re-encoded frames, it is not suitable for data streams with a relatively small number of frames.
Disclosure of Invention Therefore, an object of the present invention is to provide a method for editing data streams, in which a plurality of data streams can be edited into a data stream and in which occurrence of buffer underflow when an edited data stream passes through a buffer is prevented. Another object of this invention is to provide an apparatus for editing data streams, which can be used to edit data streams with a relatively small number of frames. Accordingly, a method for editing data streams of this invention is adapted to edit frames in a data stream A which are located before a first splicing point, and frames in a data stream B which are located after a second splicing point, into a data stream C such that underflow will not occur when the edited data stream C passes through a buffer. The method comprises the following steps: (1) based on positions of the first and second splicing points, determining the frames in the data streams A and B which require re-encoding and the frames which need to be retained therein, and recording an end value and a start value, the end value being size of a buffer amount of the buffer after a frame, which is at the same position as the last frame requiring
re-encoding in the data stream B prior to editing of the data stream B, passes through the buffer, the start value being size of the buffer amount of the buffer after a frame, which precedes the first frame in the data stream A that requires re-encoding, passes through the buffer; (2) determining whether the start value is smaller than the end value, and executing steps (3), (4), (5), (6), and (7) in sequence if the start value is smaller than or equal to the end value, and otherwise executing steps (8), (9), (10), and (11) if the start value is larger than the end value; (3) setting an intermediate buffer value; (4) decoding the frames in the data stream A and the data stream B which require re-encoding, respectively; (5) re-encoding the frames decoded from the data stream A, such that the buffer amount of the buffer can be maintained at the intermediate buffer value after the last re- encoded frame passes through the buffer; (6) re-encoding the frames decoded from the data stream B, such that the buffer amount after the last frame requiring re-encoding passes through the buffer is consistent with the end value; (7) adding the retained frames to the re-encoded frames obtained in steps (5) and (6) to yield the edited data stream C; (8) merging the frames requiring re-encoding in the data streams A and B into a group; (9) decoding the group; (10) re-encoding the group into a group of pictures such that the buffer amount after the group of pictures has
completely passed through the buffer is consistent with the end value; and (11 ) adding the frames retained in step (1 ) to the group of pictures that was re-encoded i n step ( 1 0) to form the edited data stream C . The apparatus for editing data streams which is constructed usi ng the aforesaid method includes a controller, a decoding unit, a re-encoding unit, and an integrator. The controller can determine the frames in the data streams A and B which require re-encoding and which need to be retained based on positions of the first and second splicing poi nts , set an intermediate buffer value, record an end value and a start val ue , and compare sizes of the start value and the end val ue so as to generate correspondi ng actions. The end value i s the size of a buffer a mount of the buffer after a frame , which is at the same position as the last frame requiring re- encoding in the data stream B prior to editing of the data stream B, passes through the buffer. The start value is the size of a buffer amount of the buffer after a frame, which precedes the first frame in the data stream A requi ring re- encoding , passes through the buffer. The control ler merges the frames i n the data streams A and B whi ch require re-encoding into a group when the start value is l arger than the end val ue. The decoding unit is connected el ectrically to the controller, and can decode the frames. The re-encoding unit is connected electrically to the controller and the decoding unit, and can re-encode the frames decoded by the decoding unit.
The integrator is connected electrically to the re-encoding unit and the controller, and can integrate the re-encoded frames and the retained frames into the edited data stream C. When the controller determines that the start value is smaller than or equal to the end value, the controller will control the re-encoding unit such that, when the re-encoding unit is performing re-encoding, the buffer amount of the buffer can be maintained at the intermediate buffer value after the last re-encoded frame in the data stream A passes through the buffer, and the buffer amount after the last frame requiring re- encoding in the data stream B passes through the buffer is consistent with the end value. When the controller determines that the start value is larger than the end value, the controller will control the re-encoding unit to cause the re-encoding unit to re-encode the group into a group of pictures such that the buffer amount after the group of pictures has completely passed through the buffer is consistent with the end value. The effect of this invention resides in that only a small number of frames need to be re-encoded when two data streams are edited into a data stream so that occurrence of underflow when the edited data stream passes through the buffer can be prevented. Besides, the computational- speed can be increased, and the invention can be applied to data streams with a relatively small number of frames.
Brief Description of Drawings Other features and advantages of the present invention will become apparent in the following detailed description of
the preferred embodiment with reference to the accompanying drawings, of which: Figure 1 is a schematic diagram illustrating the situation that, when a decoder is decoding a group of pictures, a buffer in the decoder allocates buffer areas to the frames, respectively, with the input of the frames; Figures 2 is a schematic diagram illustrating the display order, the display order numbers, the processing order, and the processing order numbers of the frames in the group of pictures; Figure 3 is a system block diagram of a preferred embodiment of a method and apparatus for editing data streams according to this invention; Figure 4 is a flowchart of the method of the preferred embodiment; Figure 5 is a schematic diagram of the preferred embodiment, illustrating frames in a data stream A and a data stream B; and Figure 6 is a schematic diagram of the preferred embodiment, illustrating size of an intermediate buffer value.
Best Mode for Carrying Out the Invention As shown in Figure 3, an apparatus for editing data streams according to this invention includes a controller 1, a decoding unit 2, a re-encoding unit 3, and an integrator 4. In this embodiment, frames in a data stream A which are located before a first splicing point, and frames in a data stream B which are located after a second splicing point, are
edited into a data stream C such that underflow will not occur when the edited data stream C passes through a buffer (not shown). The first and second splicing points can be set by the user according to actual requirements. Moreover, in this embodiment, the data streams A and B, are in MPEG-2 compression format, and have the same group of pictures. However, this invention can also be used for data streams in other compression format, and the data streams A and B can have groups of pictures different from that of this embodiment. Besides, the groups of pictures composing the two data streams are not required to be the same. As shown in Figures 3 and 4, the method for editing data streams according to this invention includes the following steps: In step 81, the controller 1 determines the frames in the data streams A and B which need to be re-encoded, retained, or deleted according to the first and second splicing points. With reference to Figure 5, Figure 5 shows the display order, the display order numbers, the processing order, and the processing order numbers of the data streams A and B, respectively, and indicates the positions of the splicing points. The processing order of the group of pictures of each of the data streams A and B is IPBBPBBPBBPBBPBBPBBPBBPBB.... Moreover, the first splicing point is located between the frames in the data stream A which have display order numbers 17 and 18, whereas the second splicing point is located between the frames in the data stream B which have display order numbers 7 and 8. The controller 1 will delete the frames in the data
stream A which have a display order number larger than or equal to 18, and will delete the frames in the data stream B which have a display order number smaller than or equal to 7. Herein, the controller 1 uses two rules to determine the frames that need to be re-encoded. The first rule is that a P or B frame cannot be the start frame of a data stream. The second rule is that the last frame of the data stream cannot refer to a frame after the splicing point. If a frame meets one of the aforesaid two conditions, the frame requires re-encoding. Therefore, based on these two rules, the controller 1 can determine that the frames in the data stream which require re- encoding are frames B, B which are shown in Figure 5 by hatching with oblique strokes, and the frames in the data stream B which require re-encoding are frames P and B which are shown in Figure 5 by hatching with oblique strokes. Apart from the frames that require re-encoding, the controller 1 will retain the frames that do not require re-encoding. Therefore, the processing order of the frames retained in the data stream A at this point is, in sequence, IPBBPBBPBBPBBPBBP, whereas the processing order of the frames retained in the data stream B is, in sequence, PBBPBBPBBPBBPBB.... Referring back to Figures 3 and 4, in step 82, the controller 1 determines the situation of the data streams passing respectively through the buffer prior to editing, and records a start value and an end value. The start value is the size of the buffer amount of the buffer after a frame preceding the first frame requiring re-encoding in the data stream A passes through the buffer, whereas the end value is the size of
the buffer amount of the buffer after a frame in the data stream B which is at the same position as the last frame requiring re- encoding as determined in step 81 passes through the buffer. In step 83, the controller 1 determines whether the start value is smaller than the end value. If the start value is smaller than or equal to the end value, the flow proceeds to step 84. Otherwise, the flow skips to step 94. In step 84, the controller 1 sets an intermediate buffer value based on the sizes of the start value and the end value. The time to which the intermediate buffer value corresponds is between the times to which the start value and the end value correspond. In step 85, the decoding unit 2 decodes the frames in the data stream A which have been determined to require re- encoding. In step 86, the re-encoding unit 3 re-encodes the frames decoded in step 85. Moreover, the controller 1 will supply the intermediate buffer value to the re-encoding unit 3 to enable the re-encoding unit 3 to encode all of the frames in the data stream A which require re-encoding based on the intermediate buffer value such that, under the condition that the compression quality is not degraded, the buffer amount of the buffer after the last frame passes through the buffer could be maintained at the intermediate buffer value. With reference to Figure 6, when step 86 is executed for the first time, the re-encoding unit 3 converts frames B, B to frames P, B by re-encoding. To facilitate description, the frames in the front half of the data stream A and the frames in
the rear half of the data stream B are omitted in Figure 6. When the intermediate buffer value is set to be relatively large, this represents that the buffer amount available for use by the frames in the data stream A which require re-encoding is small. Therefore, the picture quality of the frames requiring re- encoding will drop. However, a larger intermediate buffer value will cause the buffer amount to be closer to the largest buffer amount that can be accommodated by the buffer, and can therefore provide more buffer amount for use by the first I frame in the data stream B. This not only ensures that the buffer area will not have underflow, the start I frame in the data stream B can also have a sufficient buffer amount for use, thereby ensuring the picture quality of the data stream B. Thus, the controller 1 will set the size of the intermediate buffer value according to actual use conditions. In step 87, the controller 1 determines whether the buffer amount after the last frame requiring re-encoding passes through the buffer can be maintained at the intermediate buffer value. In the affirmative, the flow skips to step 89. Otherwise, the flow proceeds to step 88. In this example, supposing the buffer amount after the last frame requiring re-encoding passes through the buffer cannot be maintained at the intermediate buffer value without affecting the compression quality, the flow proceeds to step 88. In step 88, the controller 1 increases the frames in the data stream A which require re-encoding so that the frames requiring re-encoding in the data stream A do not solely include the two frames B, B determined in step 81. Thus, the situation
in which the compression quality is undesirably degraded to achieve the intermediate buffer value can be avoided. After completion of the execution of step 88, the flow skips back to step 85 to perform steps 85, 86, 87 and 88 in sequence until the buffer amount after the last frame requiring re-encoding passes through the buffer is consistent with the intermediate buffer value. In this example, the number of frames that the controller 1 finally needs to re-encode is 15. At this point, the buffer amount of the buffer can be maintained at the intermediate buffer value under the condition that the compression quality is not affected. Moreover, the processing order of the frames retained in the data stream A at this time is changed to IPB. In step 89, the decoding unit 2 decodes the frames in the data stream B which require re-encoding. In step 90, the re-encoding unit 3 re-encodes the decoded frames in the data stream B, and, under the condition that the compression quality is not degraded, the re-encoding unit 3 will attempt to bring the buffer amount after buffering of the last re- encoded frame into consistence with the end value. When step 90 is executed for the first time, the frames P, B in the data stream B which require re-encoding are re-encoded into frames
I, P. In step 91, the controller 1 determines whether the buffer amount after the last frame requiring re-encoding passes through the buffer is consistent with the end value. In the affirmative, the flow skips to step 93. Otherwise, the flow proceeds to step 92.
In this example, supposing the buffer amount after the last frame requiring re-encoding passes through the buffer cannot be maintained at the end value without affecting the compression quality, the flow proceeds to step 92. In step 92, the controller 1 increases the frames in the data stream B which require re-encoding so that the frames requiring re-encoding in the data stream B do not solely include the two P, B frames determined in step 81. In this way, the situation that the compression quality is degraded in order to achieve the end value can be avoided. After completion of the execution of step 92, the flow skips back to step 89, and steps 89, 90, 91 are carried out in sequence until the buffer amount after the last frame requiring re-encoding passes through the buffer is consistent with the end value. In this example, the buffer amount of the buffer can be maintained at the end value when the number of frames requiring re-encoding reaches 15. At this point, the processing order of the frames retained in the data stream B is, in sequence, PBB.... In step 93, the integrator 4 adds the retained frames to the re-encoded frames to transform the data stream A into a new data stream A', and the processing order of the data stream A' is IPBBPBBPBBPBBPBBPB. The data stream B is also transformed into a new data stream B', and the processing order of the data stream B' is, in sequence, IPPBBPBB.... The data stream B' is arranged after the data stream A' to obtain an edited data stream C. Thus, the processing order of the data stream C is IPBBPBBPBBPBBPBBPBIPPBBPBB....
In step 83, if the controller 1 determines that the start value is larger than the end value, the flow skips to step 94. In step 94, the controller 1 merges the frames in the data streams A and B which require re-encoding into a group. Therefore, when step 94 is executed for the first time, the frames B, B in the data stream A and the frames B, P in the data stream B are merged according to the processing order to form a group of frames BBPB. In step 95, the decoding unit 2 decodes the group obtained in step 94. In step 96, the re-encoding unit 3 re-encodes the frames decoded in step 95. Moreover, the re-encoding unit 3 will attempt to control the buffer amount after the group of frames has completely passed through the buffer to be the same as the end value under the condition that the compression quality is not degraded. When step 96 is executed for the first time, a group of frames IPBB will be obtained. In step 97, the controller 1 determines whether the buffer amount after the last frame requiring re-encoding passes through the buffer is consistent with the end value. In the affirmative, the flow skips to step 99. Otherwise, the flow proceeds to step 98. In step 98, the controller 1 increases the frames requiring re-encoding so that the frames which require re-encoding do not include the four frames BBPB determined in step 94 only. After completion of the execution of step 98, the flow skips back to step 94, and steps 94, 95, 96, and 97 are carried out in sequence until the buffer amount after the last frame requiring
re-encoding passes through the buffer is consistent with the end value. In this example, by repeating the aforesaid steps, the frames retained in the data stream A are found to be IPBB, and the frames retained in the data stream B are found to be PBB.... In step 99, the integrator 4 integrates the retained frames and the re-encoded frames into a new data stream D, in which the front half portion of the frames according to the processing order of the data stream D are the frames IPBB retained in the data stream A, whereas the frames in the rear half portion are the frames PBB... retained in the data stream B. The frames between the front half portion and the rear half portion are the re-encoded group of pictures IPBBPBBPBBPBB... PB. The processing order of the edited data stream D is IPBB IPBBPBBPBBPBB... PBPBB.... However, it is noted that, although two rules are used to determine the frames that require re-encoding in step 81 of this embodiment, and frames requiring re-encoding are increased dynamically in step 88 or step 92 when the number of frames requiring re-encoding is not sufficient and the compression quality is degraded, in utilizing this invention, frames requiring re-encoding may be increased in step 81 according to experience or actual use conditions so that the frames requiring re-encoding do not merely include the frames determined based on the two rules. For example, by setting the number of frames requiring re-encoding to 15, repetitions of step 88 or step 92 can be reduced or avoided, thereby increasing the overall computational speed. For the same
reason, the number of frames requiring re-encoding may also be increased in step 94 to reduce the number of times of repeating step 98. In addition, although this embodiment is used to edit two data streams A and B, it can also be used when there is only one data stream. That is, when there is only one data stream, and the data stream has two splicing points at different positions, the frames between the two splicing points are deleted, and the remaining data stream segments preceding and following the splicing points are edited so as to merge the segments into a new data stream. In sum, this invention can edit two data streams A and B into a data stream, and can eliminate the problem of underflow when the edited data stream passes through the buffer. The essential technical feature resides in that the controller 1 initially determines the situations when the data streams A and B pass respectively through the buffer prior to editing, and records a start value and an end value. The re-encoding unit 3 re-encodes some of the frames properly so that the buffer amount required for buffering the new data stream C or D, for which editing has been completed, through the buffer is consistent with the start value and the end value under the condition that the frames are identical so as to prevent buffer underflow. Since the number of frames requiring re-encoding is considerably reduced in this invention compared with the method adopted in the prior art, regardless of whether the flow from steps 81 to 93 or the flow from steps 81 to 83 and from steps 94 to 99 is followed, this invention not only provides a
faster computational speed compared with the prior art, it can also be used to edit data streams with a relatively small nu mber of frames. Furthermore, as this invention can dynamical ly i ncrease the frames requiri ng re-encoding , better compression quality can be ensu red . Thus , the objects of thi s invention can be positively achieved . While the present invention has been described i n connection with what is considered the most practical and preferred embodiment, it is understood that this invention is not limited to the disclosed embodiment but is intended to cover various arrangements included within the spirit and scope of the broadest interpretation so as to encompass all such modifi cations and equivalent arrangements.