JP2002300526A - Device and method for editing stream - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a stream editing device/method for decoding a picture, even if a stream is edited at an arbitrary point. SOLUTION: A stream SA from the head image of a first stream to an image E1, which is compressed and encoded without using bidirectional prediction which are nearest an editing point in a display order, a stream SC from the next image of an image E2 compressed and encoded without using prediction in the two directions, which are the nearest the editing point, in the display order of the second stream to the last image of the second stream and a stream SB, where the images from the next image of the image E1 of the first stream to the editing point and images from the editing point of the second stream to the image E2 are recompressed and encoded are connected, and an edit stream is generated.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

[0001] 1. Field of the Invention [0002] The present invention relates to a stream editing apparatus and a stream editing method for editing a compression-encoded image stream at an arbitrary time.

[0002]

2. Description of the Related Art As is well known, for example, MPEG (Moving)
In image compression encoding using inter-frame compression such as Picture Image Coding Experts Group), inter-frame prediction of image motion is used for image compression.

[0003] In this case, bidirectional prediction in which prediction is performed using past and future images for an image to be compressed,
The image is compressed by three methods, that is, unidirectional prediction in which prediction is performed using only past images, and a case in which prediction is not used.

An image to be compressed using bidirectional prediction is a B picture, an image to be compressed using unidirectional prediction is a P picture, an image to be compressed without prediction, that is, an intra-frame coded image is compressed. This is called an I picture.

Here, as shown in FIG. 4 (a), images are input in order of B0, B1, I2, B3, B4, P5, B
When the compression encoding is performed in the order of 6, B7, P8, B9, B10, P11, the I or P picture used for motion prediction is required for the compression encoding or decoding of the B picture.

For this reason, as shown in FIG.
2, B0, B1, P5, B3, B4, P8, B6, B
The images are exchanged in the order of 7, P11, B9, B10 and compression-encoded.

[0007] Since compression encoding is performed using such a method, it is not possible to decode a compression-encoded stream from an arbitrary picture. Picture). Then, the GOP is usually the minimum unit for stream editing.

[0008]

When an attempt is made to simply edit such an image stream at an arbitrary point in time, for example, as shown in FIG. When the editing is performed, the B6 image is subjected to bidirectional prediction using the P5 and P8 images, but cannot be decoded because the edited stream contains only the P5 image.

As shown in FIG. 5B, when editing is performed before the image of B7, the image of B7 performs bidirectional prediction using the images of P5 and P8, and the image of P8 is
Although unidirectional prediction is performed using the image No. 5, decoding cannot be performed because there is no P5 image in the edited stream. Furthermore, since the image of P8 cannot be decoded, the subsequent images cannot be decoded up to the image (I picture) on which the intra-frame processing has been performed.

For this reason, if the image before and after the edit point is a P picture or a B picture, a reference image for decoding the image does not exist in the edited stream.
There is a problem that the image cannot be correctly decoded.

Accordingly, the present invention has been made in view of the above circumstances, and is an extremely favorable stream editing apparatus and stream editing method capable of decoding an image even if a stream is edited at any point. The purpose is to provide.

[0012]

According to the present invention, there is provided a stream editing apparatus comprising: a first stream including an image compressed and encoded using bidirectional prediction and an image compressed and encoded without using bidirectional prediction; After the predetermined image, the second stream including the image that is compression-encoded using bidirectional prediction and the image that is compression-encoded without using bidirectional prediction is connected from the predetermined image. .

[0013] From among the images existing from the head of the first stream to the editing point, an image E1 which has been compression-encoded without using bidirectional prediction closest to the editing point in display order is searched, and the first image is searched for. A first clipping means for clipping from the first image of the stream to the image E1 as a stream A, and without using bidirectional prediction closest to the editing point in display order among images existing from the editing point to the end of the second stream. A second clipping means for retrieving an image E2 compressed and encoded as follows, and clipping from the next image of the image E2 to the last image of the second stream as a stream C; A re-encoding method for generating a stream B by performing a re-compression encoding process on an image from an image to an editing point and an image from an editing point of the second stream to an image E2. When, it is obtained as comprising stream A, B, and connecting means for generating an edit stream by connecting C.

Further, the stream editing method according to the present invention is directed to a method for editing a predetermined image of a first stream including an image compressed and encoded using bidirectional prediction and an image compressed and encoded without using bidirectional prediction. Later, a method of connecting a second stream including an image compressed and encoded using bidirectional prediction and an image compressed and encoded without bidirectional prediction from the predetermined image is targeted.

Then, among the images existing from the head of the first stream to the editing point, an image E1 which has been compression-encoded without using bidirectional prediction which is closest to the editing point in display order is searched, and the first image is searched for. A first clipping step of clipping from the first image of the stream to the image E1 as a stream A, and without using bidirectional prediction closest to the editing point in display order among images existing from the editing point to the end of the second stream. A second extraction step of searching for an image E2 compressed and encoded as follows, and extracting the next image from the image E2 to the last image of the second stream as a stream C; A re-encoding process for generating a stream B by performing a re-compression encoding process on an image from an image to an edit point and an image from an edit point to an image E2 of the second stream. When, in which as a stream A, B, and a connection step of generating edited stream by connecting C.

According to the configuration and method as described above, the first
The image can be decoded even if the stream editing for connecting the stream of the second stream and the second stream at an arbitrary point is performed.

That is, a stream before the edit point is a stream S1, and a stream after the edit point is a stream S2. Among the images from the head of the stream S1 to the editing point, the image (I-picture or P-picture) closest to the editing point in the display order and for which bidirectional prediction is not performed is designated as E
1, and the range from the head of stream S1 to E1 is stream SA.

Further, among the images from the edit point to the end of the stream S2, the I picture or P picture which is closest to the edit point in the display order is defined as E2, and the range from the image following E2 to the end is defined as the stream. SC.

The image necessary for encoding the image from the image following E1 of the stream S1 to the editing point is converted to the stream S1.
1, the image necessary for encoding the image from the editing point of the stream S2 to E2 is decoded from the stream S2, and the decoded image is encoded again to create the stream SB.

When re-encoding the stream SB,
In the connection between stream SA and stream SC, V
The code amount is set so that the BV buffer does not break down, and the code amount is distributed so that the generated code amount of E2 that comes last in the display order is maximized, and encoding is performed.

Finally, the streams SA, SB and SC are connected to form one stream. At this time, since the stream SA has all the reference images required for decoding in the stream SA, there is no problem in decoding.
Since the stream SB has been re-encoded, there is no image breakdown due to a missing reference image. Also, the stream SC
Uses E2 of the stream SB as a reference image, and generates a stream SB by allocating a large amount of code so that the image becomes an image close to the decoded image of the stream S2. Does not happen.

As described above, by re-encoding a small number of images, it is possible to edit a stream that is not in GOP units.

[0023]

Embodiments of the present invention will be described below in detail with reference to the drawings. In FIG. 1, a stream coming before the editing point is S1, and a stream coming after the editing point is S2. First, a location where a stream is to be edited is determined and input to the stream clipper 11 as an editing point.

The stream cutout unit 11 analyzes the input stream, checks the type of prediction (encoding type) used in the image near the edit point, and, based on the result, determines the start of the stream S1. From the images up to the editing point, the I or P picture closest to the editing point in the display order is searched and set to E1, and a stream (stream SA) ranging from the first image of the stream S1 to the image of E1
Cut out.

The stream cutout unit 11 searches for an I or P picture which is closest to the editing point in the display order from among the images from the editing point to the end of the stream S2, sets it as E2, and sets the next image to E2. Stream (stream SC) ranging from to the last image of stream S2
Cut out.

The streams SA and SC cut out by the stream cutout unit 1 are supplied to a stream connection unit 12. In this case, E1 and E
2 is output to the re-encoder 13 as cutout information.

This re-encoder 13 outputs the stream S
1, a decoder 14 for creating an image necessary for re-encoding the image S1-B from the edit point of the stream S2 to the image S2-B from the edit point of the stream S2, and an output image from the decoder 14 1 for temporarily storing
5 and an encoder 16 for creating a stream.

Here, the image S1-B and the image S2-B to be re-encoded are determined based on the cut-out information supplied to the re-encoder 13, and the images S1-B and S-B are re-encoded.
The image necessary for re-encoding 2-B is decoded by the decoder 14 and the decoded image is stored in the memory 15.
Supplied to

In the decoder 14, information necessary for encoding is extracted and calculated by the encoder 16, and is supplied to the encoder 16. Further, the memory 15 holds the image sent from the decoder 14 and outputs it to the encoder 16 when necessary.

Then, in the encoder 16, based on the cut-out information and the information from the decoder 14, the VBV buffer does not break down in connection with the stream SA and the stream SC, and E2 is an image close to the original image. An encoding parameter in which the code amount is set to a large value is created, an image is received from the memory 15, encoding is performed, and the created stream (stream SB) is output to the stream connector 12.

The stream connection unit 12 connects the streams SA and SC from the stream cutout unit 11 with the stream SB from the re-encoder 13, and changes the encoding information in the stream as necessary. The editing of the stream is completed.

Next, a description will be given of a specific example of a method of connecting a stream by editing the stream at an arbitrary place.

[Specific Example 1] Editing for connecting two streams at an arbitrary point will be described. In this case, a stream before the edit point is S1 and a stream after the edit point is S2. FIG. 2A shows the vicinity of the editing point of the stream S1 and the stream S2, and shows the editing that connects the image in the GOP5 of the stream S1 to the image in the GOPD of the stream S2.

First, the stream cutout unit 11 checks the encoding type of the streams S1 and S2 near the editing point based on the information included in the stream. For the stream S1, an I or P picture closest to the editing point in the display order is searched from the images from the head of the stream S1 to the editing point.

In this case, since the edit point of the stream S1 is between B6 and B7 of GOP5 as shown in FIG. 2B, the I or P picture closest to the edit point is P5.
Therefore, this P5 is set to E1. Then, the stream SA from the leading image of the stream S1 to the image of E1
Cut out.

In the stream S2, the I or P picture closest to the edit point in the display order is searched for from the image from the edit point to the end. The editing point of this stream S2 is, as shown in FIG.
Since it is between B6 and B7 of D, P8 closest to the edit point is E2. Then, a stream SC from the image following E2 to the last image of stream S2 is cut out.

After that, the stream cutout unit 11
The positions of the stream E1 and stream E2 on the stream are sent to the re-encoder 13 as cutout information, so that the image to be re-encoded is B6 (S1-B) of the stream S1.
And B7 and B8 (S2-B) of the stream S2.

Then, in the re-encoder 13, the image used for re-encoding the images S 1 -B and S 2 -B from the streams S 1 and S 2 is decoded by the internal decoder 14 and stored in the memory 15. , Encoder 16
And re-encode to create a stream SB.

Then, as shown in FIG. 2C, each of the streams SA, SB, SC created as described above
Are connected by the stream connector 12, so that
The edit stream is completed.

In this re-encoding process, encoding is executed by allocating a large code amount to E2 of the stream SB, that is, E2 so that the image deterioration of P8 is reduced. These are B9, B10, P1 of the stream SC.
This is because P8, which was in the stream S2, was required for decoding 1, but the image of E2 in the stream SB is used for decoding by editing.

However, since E2 is encoded so as to be an image close to the image obtained by decoding P8 of the original stream S2, E2 of stream SB is
B9, B10, P1 of the stream SC referring to the image of
When decoding 1, there is no problem that the image is broken.

In order to suppress the deterioration of the image of E2,
Since a large amount of code is allocated to E2, other images to be re-encoded (B6 of S1-B and B7 of S2-B) deteriorate, but since this is also a scene change, the decoded image cannot be converted to a normal image. It doesn't matter much to see.

[Specific Example 2] In the re-encoding described in the specific example 1, the re-encoder 13 does not use the encoder 16 to detect the motion prediction performed in the encoding of each image, as shown in FIG. , The information described in the original stream is extracted from the decoder 14 and sent to the encoder 16, and the encoder 16 performs the encoding using this information instead of detecting the motion prediction. ing.

When the editing as described with reference to FIG. 2 is performed, the decoder 14 performs the following processing on the S1-B image.
Only the forward prediction information is extracted and sent to the encoder 16, and if there is prediction information from the decoder 14, the encoder 16 performs prediction processing using that information, and there is no prediction information from the decoder 14. In this case, an intra-frame compression encoding process is performed.

For this reason, the encoding of B6 of S1-B is performed using only the forward prediction information from E1, ie, P5. The decoder 14 sends only backward prediction information to the encoder 16 for the S2-B image, and the encoder 16 performs the same encoding as described above.

B7 of S2-B is coded by using only the prediction information in the backward direction from E2, ie, P8.
Since No. 2 originally does not use the later prediction, an intra-frame compression encoding process that does not use prediction is performed.

Since the reference image used for the prediction of B7 of S2-B is E2, different images are used at the time of encoding and at the time of decoding. However, the image of E2 is the E2 of the stream S2.
Because it is encoded so as to be close to the image of the second image,
This is not a problem for motion estimation.

As described above, the image existing in the stream S1 is a prediction process using only the image existing in the stream S1, and the image existing in the stream S2 is a prediction process using only the image existing in the stream S2. Since the images of the streams S1 and S2 are connected by editing, the correlation between the images is small.
Since one image is rarely used for predicting the image of the stream S2, the influence on the code amount, image quality, and the like is small.

The same can be said for the stream S2. Furthermore, since the prediction process which requires a lot of load in the encoder 16 is reduced, the speed of re-encoding can be increased and the configuration of the encoder 16 can be simplified.

[Embodiment 3] In the re-encoding described in the embodiment 1, the re-encoder 13 has an encoder 16 which does not perform the image prediction processing. That is, the encoder 16 encodes only I pictures for which prediction is not performed for each image. As a result, since the prediction is not used, the image quality of the re-encoded stream is degraded. However, since no prediction processing is performed, the speed of re-encoding can be increased and the configuration of the encoder 16 can be simplified.

It should be noted that the present invention is not limited to the above-described embodiment, and can be variously modified and implemented without departing from the scope of the invention.

[0052]

As described in detail above, according to the present invention,
It is possible to provide a very good stream editing apparatus and a very good stream editing method that can decode an image even if a stream is edited at any point.

[Brief description of the drawings]

FIG. 1 is a block diagram showing an embodiment of the present invention.

FIG. 2 is an exemplary view for explaining re-encoding processing according to the embodiment;

FIG. 3 is a block diagram showing a modification of the re-encoder according to the embodiment;

FIG. 4 shows each picture and GO in MPEG compression encoding.
The figure shown for demonstrating the relationship with P.

FIG. 5 is a view for explaining a problem in stream editing.

[Explanation of Code] 11: Stream cutout unit, 12: Stream connection unit, 13: Re-encoder, 14: Decoder, 15: Memory, 16: Encoder

Claims (6)

[Claims] 1. A method according to claim 1, further comprising, after a predetermined image of a first stream including an image compression-encoded using bidirectional prediction and an image compression-encoded without bidirectional prediction, performing compression encoding using bidirectional prediction. A stream editing apparatus for connecting a second stream including an encoded image and an image compressed and encoded without using bidirectional prediction from the predetermined image, wherein the stream exists from the beginning of the first stream to an editing point. Among the images to be searched, a compression-encoded image E1 is searched without using bidirectional prediction closest to the editing point in display order, and a stream A from the first image of the first stream to the image E1 is cut out. 1 and compression encoding without using bidirectional prediction closest to the editing point in the display order among images existing from the editing point to the end of the second stream. A second clipping means for retrieving the extracted image E2 and clipping from the next image of the image E2 to the last image of the second stream as a stream C; and the next image of the image E1 of the first stream. Re-encoding means for performing a re-compression encoding process on the image from the editing point of the second stream to the image E2 and the image from the editing point of the second stream to the image E2; , B, and C, and a connection means for generating an edit stream. 2. The method according to claim 1, wherein the re-encoding unit obtains prediction information used in a re-compression encoding process from information included in the first and second streams.
The stream editing device as described. 3. The image processing apparatus according to claim 1, wherein said re-encoding means performs a compression encoding process on the image to be subjected to the re-compression encoding process by using only an intra-frame process without using prediction. Stream editing device. 4. After a predetermined image of a first stream including an image that has been compression-encoded using bidirectional prediction and an image that has been compression-encoded without using bidirectional prediction, a compression code is generated using bidirectional prediction. A stream editing method for connecting a second stream including an encoded image and an image compressed and encoded without using bidirectional prediction from the predetermined image, wherein the stream exists from the beginning of the first stream to an editing point. Among the images to be searched, a compression-encoded image E1 is searched without using bidirectional prediction closest to the editing point in display order, and a stream A from the first image of the first stream to the image E1 is cut out. 1 and compression encoding without using bidirectional prediction closest to the edit point in display order among images existing from the edit point to the end of the second stream. A second cropping step of retrieving the extracted image E2, and cropping from the image following the image E2 to the last image of the second stream as a stream C; and an image following the image E1 of the first stream. A re-encoding step of performing a re-compression encoding process on the image from the editing point of the second stream to the image E2 to generate the stream B; , B, and C to generate an edit stream. 5. The method according to claim 4, wherein in the re-encoding step, prediction information used in a re-compression encoding process is obtained from information included in the first and second streams.
The stream editing method described. 6. The re-encoding step according to claim 4, wherein the image subjected to the re-compression encoding process is subjected to the compression encoding process using only intra-frame processing without using prediction. Stream editing method.