JPH0923430A - Method and device for editing moving picture - Google Patents

Abstract

PROBLEM TO BE SOLVED: To highly precisely reproduce all frames by reproducing all the frames even if they are edited in an encoded state as the are. SOLUTION: A central operation processor 4 supplies a control signal to a signal source 1 through an interface 2 and supplies the control signal to a buffer manager 3. The buffer manager 3 writes a data string which is read from the signal source 1 through a data bus 9 and the interface 2 into a buffer memory 5 on a real time and writes the data string after an editing processing into the signal source 1 through the data bus 9 and the interface 2 on a real time. When a B-frame is judged to exist between a first I-frame and a next frame or a P-frame by an initial GOP (the group of videos) header, data of the B-frame between the first I-frame and the next I-frame or the P-frame can be substituted for data of the first I-frame with the first GOP header.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a moving image editing method and an apparatus thereof, and more particularly to an image frame compressed by an intraframe prediction method and an interframe prediction method which are encoded by using prediction. The present invention relates to a method and an apparatus for editing a bit stream composed of a coded moving image in which compressed image frames are mixed, while maintaining the code state.

[0002]

2. Description of the Related Art Conventionally, MPEG (Moving P)
A method for transmitting, recording and reproducing high-efficiency-compressed image data using prediction (image data highly-compressed by the MPEG method) according to the data format proposed by the Picture Coding Group) has been proposed. Further, a method has been proposed in which a reproduced image in a good state can be obtained even when the image is reproduced from the middle of the bit stream (see Japanese Patent Laid-Open No. 6-268995).

According to this method, when reproduction is performed from the middle of a bit stream, a GOP (Group Of Picture) that first appears after the point when the middle reproduction is started.
e) Close (a series of data recorded sequentially following the MPEG system header part in each sector)
De GOP {GOP header → I frame (IntraPi
image information of an image frame in which image data is compressed by applying the intra-frame prediction method → P frame (Predicted Picture)
s, which is an image frame in which the image data is compressed by applying the inter-frame prediction method based on the image data of the past frame, the image information of the arrow corresponding to the forward prediction in the lower part of FIG. 3) → is a GOP having an array of image information such as, that is, a GOP that does not have a frame that may refer to the data of another GOP and can be reproduced from the middle}. If the GOP that first appeared after the point where the midway playback was started is not a closed GOP, the broken link (whether it is a closed GOP or not) in the GOP header attached to the GOP. Is a flag that is rewritten when clipping is performed. Normally, 1 is set by clipping, and then the bit stream is supplied to the MPEG video decoder. It is that way.

Here, when the broken link is 1, the GOP to which the GOP header is attached
B-frame (Bi-directional P) existing between the I-frame and the P-frame constituting the
The image data is compressed by applying the inter-frame prediction method that is based on the image data of both the past frame and the image data of the future frame. The MPEG video decoder is made not to perform the decoding operation for the image frame (see the arrow corresponding to the bidirectional prediction in the upper part of FIG. 3).

[0005]

[Patent Document 1] Japanese Patent Application Laid-Open No. 6-26899
The image reproduction method described in Japanese Patent Publication No. 5 includes a method for reproducing an image from the middle of a bitstream including moving image information, and thus can be applied to editing a bitstream including moving image information. it can. That is, 1
It is possible to edit a moving image by cutting out image frame data from the middle of one or more bit streams and connecting two or more cut out data.
The moving image edited in this way can be reproduced by the MPEG video decoder.

However, if the GOP first appearing after the point when the halfway reproduction is started is not a closed GOP, the broken link in the GOP header attached to the GOP is set to 1 and then the MPEG video is set. Since the bitstream is supplied to the decoder, the MPEG video decoder does not perform the decoding process on the corresponding GOP. That is, the relevant GO
As for P, the B frame existing between the I frame and the P frame is not reproduced. If this method is adopted and all the frames after the point when the halfway playback is started are played back without this method, the playback image is affected by the frames belonging to the irrelevant bitstream. It is possible to eliminate the inconvenience that is incorrect.

However, when the broken link is set to 1 and the bit stream is supplied to the MPEG video decoder as described above, a frame which is not reproduced is included in the bit stream. When both the voice and the image are played back, there is a disadvantage that only the voice is played back, and a frame in which the image is not played back occurs. Further, in order to eliminate such inconvenience, it is conceivable to adopt a measure to lengthen the reproduction display time of other frames by a time corresponding to the number of frames not reproduced. If the measures are adopted, the processing becomes extremely complicated.

[0008]

[Object] The present invention has been made in view of the above problems, and it is possible to reproduce all frames even when editing is performed in an encoded state, and at the same time, all frames can be reproduced. It is an object of the present invention to provide a moving image editing method and an apparatus thereof capable of reproducing an image with considerably high accuracy.

[0009]

According to a first aspect of the present invention, there is provided a moving image editing method, wherein an image frame compressed by an intra-frame prediction method and an image frame compressed by an inter-frame prediction method are coded using prediction. A method for editing a bitstream consisting of mixed coded moving images in the coded state as it is, and cutting out the bitstream before the image frame compressed by the intraframe prediction method, and cutting out the bit It is determined whether or not all the image frames compressed by the inter-frame prediction method in the stream can be displayed correctly, and the image frames compressed by the inter-frame prediction method that cannot be displayed correctly are generated accurately Replace image frames compressed by interframe prediction that cannot be displayed with image frames compressed by intraframe prediction that are most likely to be similar It is that way.

According to a second aspect of the present invention, there is provided a moving image editing method in which an image frame compressed by the intra-frame prediction method and an image frame compressed by the inter-frame prediction method, which are coded using prediction, are mixed. A method for editing a bit stream consisting of a moving image as it is in a coded state, and cutting out the bit stream before the image frame compressed by the intra-frame prediction method, and all in the cut out bit stream, An inter-frame prediction method that cannot be displayed correctly by determining whether an image frame compressed by the inter-frame prediction method can be displayed correctly and the image frame compressed by the inter-frame prediction method that cannot be displayed correctly The image frame compressed with is most likely to be similar to the image frame compressed with the intra-frame prediction method. It is a method of replacing the image frames compressed by equivalent inter-frame prediction method.

According to another aspect of the present invention, a moving image editing apparatus is an encoding method in which image frames compressed by the intra-frame prediction method and image frames compressed by the inter-frame prediction method, which are coded using prediction, are mixed. A device for editing a bit stream consisting of a moving image as it is in a coded state, and cutting out the bit stream before the image frame compressed by the intra-frame prediction method, and all in the cut out bit stream, Condition determining means for determining whether or not an image frame compressed by the inter-frame prediction method can be accurately displayed, and condition that the determining means determines that an image frame compressed by the inter-frame prediction method that cannot be accurately displayed has occurred As an example, an image frame compressed by the inter-frame prediction method that cannot be displayed accurately is compressed by the intra-frame prediction method that is most likely to be similar. And a replacement means for replacing the image frames.

According to another aspect of the moving image editing apparatus of the present invention, there is a mixture of image frames compressed by the intra-frame prediction method and image frames compressed by the inter-frame prediction method, which are coded using prediction. A device for editing a bit stream consisting of a moving image as it is in a coded state, and cutting out the bit stream before the image frame compressed by the intra-frame prediction method, and all in the cut out bit stream, Condition determining means for determining whether or not an image frame compressed by the inter-frame prediction method can be accurately displayed, and condition that the determining means determines that an image frame compressed by the inter-frame prediction method that cannot be accurately displayed has occurred As an example, an image frame compressed by the inter-frame prediction method that cannot be displayed accurately is compressed by the intra-frame prediction method that is most likely to be similar. And a replacement means for replacing at image frames compressed as it to display the equivalent inter-frame prediction method image frames.

[0013]

According to the moving image editing method of the first aspect, the image frames compressed by the intra-frame prediction method and the image frames compressed by the inter-frame prediction method, which are coded using prediction, are mixed. When editing a bitstream consisting of a coded moving image in the coded state, the bitstream is cut out before the image frame compressed by the intra-frame prediction method, and all of the cutout bitstream is extracted. , It is judged whether the image frame compressed by the inter-frame prediction method can be displayed correctly, and if the image frame compressed by the inter-frame prediction method cannot be displayed correctly, the inter-frame prediction cannot be displayed correctly. Since the image frame compressed by the method is replaced with the image frame compressed by the intra-frame prediction method that is most likely to be similar, When playing a collected bitstream, it is possible to prevent the occurrence of a frame that should be played but should not be played, and yet the accuracy of the replaced image frame is maintained to a very high level. An edit process that can be done can be achieved. Of course, since there is no frame that is not reproduced, it is not necessary to perform compression / expansion processing, and it is possible to prevent processing from becoming complicated.

According to the moving image editing method of the second aspect, the image frames compressed by the intra-frame prediction method and the image frames compressed by the inter-frame prediction method, which are encoded by using the prediction, are mixed. When editing a bitstream consisting of a coded moving image in the coded state, the bitstream is cut out before the image frame compressed by the intra-frame prediction method, and all of the cutout bitstream is extracted. , It is judged whether the image frame compressed by the inter-frame prediction method can be displayed correctly, and if the image frame compressed by the inter-frame prediction method cannot be displayed correctly, the inter-frame prediction cannot be displayed correctly. The image frame compressed by the method is displayed as the image frame compressed by the intra-frame prediction method that is most likely to be similar. Since it is replaced with the image frame compressed by the inter-frame prediction method equivalent to, it prevents the occurrence of a frame that should be played but not played when playing the edited bitstream. Editing processes can be achieved that can be done and yet the accuracy of the replaced image frame can be kept fairly high. Of course, since there is no frame that is not reproduced, it is not necessary to perform compression / expansion processing, and it is possible to prevent processing from becoming complicated.

According to another aspect of the moving image editing apparatus of the present invention, the image frames compressed by the intra-frame prediction method and the image frames compressed by the inter-frame prediction method, which are coded using the prediction, are mixed. When editing a bitstream consisting of a coded moving image in the coded state, the bitstream is cut out before the image frame compressed by the intra-frame prediction method, and all of the cutout bitstream is extracted. The determining unit determines whether or not the image frame compressed by the inter-frame prediction method can be accurately displayed, and the determining unit determines that the image frame compressed by the inter-frame prediction method that cannot be accurately displayed has occurred. As a condition, an image frame compressed by the inter-frame prediction method that cannot be displayed accurately by the replacement means is most likely to be similar to the frame. Since it is replaced with an image frame compressed by the intra-frame prediction method, when an edited bitstream is played back, it is possible to prevent the occurrence of a frame that should be played but is not played back. It is possible to achieve an editing process in which the accuracy of the replaced image frame can be kept quite high. Of course, since there is no frame that is not reproduced, it is not necessary to perform compression / expansion processing, and it is possible to prevent processing from becoming complicated.

According to another aspect of the present invention, in the moving image editing apparatus, there is a mixture of image frames compressed by the intra-frame prediction method and image frames compressed by the inter-frame prediction method, which are coded using prediction. When editing a bitstream consisting of a selected moving image in the coded state, the bitstream is cut out before the image frame compressed by the intraframe prediction method, and all the frames in the cutout bitstream are extracted. The determination means determines whether or not the image frame compressed by the inter prediction method can be accurately displayed, and the determination means determines that an image frame compressed by the inter frame prediction method that cannot be accurately displayed has occurred. , The image frame compressed by the inter-frame prediction method that cannot be displayed accurately by the replacement means is the frame that is most likely to be similar. Since the image frame compressed by the inner prediction method is replaced with the image frame compressed by the inter-frame prediction method, which is equivalent to displaying the image frame as it is, it should be reproduced when the edited bitstream is reproduced. Regardless, it is possible to prevent occurrence of a frame that is not reproduced regardless of the fact that the editing process can maintain the accuracy of the replaced image frame to a considerably high level. Of course, since there is no frame that is not reproduced, it is not necessary to perform compression / expansion processing, and it is possible to prevent processing from becoming complicated.

A more detailed description will be given. There is always a "frame that can be accurately reproduced" before the "frame that cannot be accurately reproduced" (before the display order, but in the code order) due to editing. Also, "frames that can be played correctly"
Is used as a “future frame” to predict “frames that cannot be reproduced accurately”. This “frame that can be accurately reproduced” is before the “frame that cannot be accurately reproduced” in the code order, but is after the “frame that cannot be accurately reproduced” in the display order. Therefore, it can be said that the frames are temporally closest in display order, have the highest degree of similarity, and are “frames that can be accurately reproduced”. In addition,
Taking the bitstream shown in FIG. 3 as an example, the display order is
P0, B1, B2, I3, B4, B5, P6, B7, B
, P9, B10, ..., but the code order is P0,
I3, B1, B2, P6, B4, B5, P9, B7, B
8, ...

Further, in general, the handling of "frames that cannot be accurately reproduced" by editing depends on the reproducing apparatus. However, in the present invention, the image data of "frames that cannot be accurately reproduced" remains in the code state in the editing process. Since it is replaced with another image data, it does not depend on the playback device,
All frames can be played back accurately without being missed.

[0019]

BEST MODE FOR CARRYING OUT THE INVENTION Embodiments of the present invention will be described in detail below with reference to the accompanying drawings showing embodiments. It should be noted that, although the description will be made in conformity with the MPEG system below, the invention is not limited to the case in which the MPEG system is complied with, and image data is encoded by adopting an intra-frame prediction method or an inter-frame prediction method. Any method can be applied similarly.

1 and 2 are flow charts for explaining one mode of the moving image editing method of the present invention. The flowchart of FIG. 1 explains a normal editing process. In step SP1, data is cut out in GOP units in the code state, and in step SP2 it is determined whether the first GOP is a closed GOP. Of the GOP header arranged prior to the head portion of the GOP. If it is determined that the first GOP is not a closed GOP, it is arranged prior to the head portion of the GOP in step SP3. Rewrite the broken link in the GOP header that has been set to 1. Then, when the processing of step SP3 is performed, or the first GOP in step SP2 is a closed GOP.
If it is determined that the data is edited, the data edited in step SP4 is stored in the code state as it is, and the series of processes is ended.

The flowchart of FIG. 2 is for explaining the editing process which is a feature of the present invention. In step SP1, the data is cut out in GOP units in the code state,
In step SP2, the first GOP is a closed G
Whether or not it is an OP is judged by the information content of the GOP header arranged prior to the head part of the GOP. If it is judged that the first GOP is not a closed GOP, in step SP3. Information indicating that the GOP is not a closed GOP in the GOP header arranged prior to the beginning of the GOP is closed.
Rewrite as a de GOP. And step SP4
In the first GOP, the data of the B frame between the first I frame and the next I frame or P frame is replaced with the data of the first I frame. Then, when the processing of step SP4 is performed, when it is determined that the first GOP is a closed GOP in step SP2, the edited data is stored in the code state in step SP5, and Ends the process.

However, in the processing in step SP4, although the data may be replaced as described above, it is present between the first I frame and the next I frame or P frame which compose the first GOP. The first I frame may be transferred to the MPEG video decoder instead of the B frame being processed. Further, the B frame may be replaced with an image frame for which interframe prediction is performed based on image data of a frame which is a future frame so that the I frame is displayed as it is.

Specifically, the GOP shown in FIG. 3 (display order is P0, B1, B2, I3, B4, B5, P6, B
7, B8, P9, B10, ... And the code order is P
0, I3, B1, B2, P6, B4, B5, P9, B
, B8, ...), and when the I3 frame is cut out in the code order by the editing process, P0
Since the frame does not exist, the B1 frame and the B2 frame cannot be accurately reproduced as they are. However, in this embodiment, the B1 frame,
Since the data of the B2 frame is replaced with the data of the I3 frame, the B1 frame and the B2 frame can be remarkably accurately reproduced as compared with the case of reproducing the B1 frame and the B2 frame as they are. However, if the increase of the data amount becomes a problem by replacing the B frame with the I frame, I3
The frame may be displayed as it is, and the frame may be replaced with the data of the B frame for which the inter-frame prediction is performed based on the data of the future frame (I3 frame).
Since the I3 frame is displayed as it is, the frame data to be replaced is the frame data having a difference of 0 with respect to the I3 frame, and the image frame compressed by the intra-frame prediction method that is most likely to be similar is displayed as it is. Corresponds to an image frame compressed by the inter-frame prediction method equivalent to. Further, it is preferable that the frame data to be replaced is provided in advance in the video editing system,
The above substitutions can be easily accomplished if desired.

FIG. 4 is a block diagram showing an aspect of the moving image editing apparatus of the present invention. This moving image editing apparatus includes a signal source 1, an interface 2, and a buffer manager 3.
And a central processing unit 4, a buffer memory 5, and a memory 6. In addition, 7, 8 and 9 are data buses. The signal source 1 is high-efficiency-coded moving image information (video information) according to the MPEG system in which at least I frame and P frame or B frame are mixed.
And a time series data string (bit stream) including data of audio information (audio information), type information indicating whether each of the above information is video information or audio information,
An information recording medium in which a data string to be edited is recorded in a state in which a predetermined header including at least time information (time stamp) for each information is added, for example, , Optical disks, magneto-optical disks, and the like. Further, this information recording medium is capable of writing the edited data string as well as reading the data string.

The buffer memory 5 temporarily holds a data string read from the signal source 1 through the data bus 9, the interface 2, and the buffer manager 3, and also temporarily holds the edited data string. To do. The memory 6 holds a program for performing the processing of the flowcharts shown in FIGS.

The central processing unit 4 reads the program from the memory 6 and performs the processing shown in the flow chart of FIG. 1 or 2. In addition, the central processing unit 4 uses the interface 2 in accordance with the timing required to perform the processing of the flowchart of FIG. 1 or 2.
Supply necessary control signals (read control signal, write control signal, etc.) to the signal source 1 via
Control signals required for the buffer manager 3 (control signals for reading, control signals for writing, etc.)
Supply.

The buffer manager 3 writes the data string read from the signal source 1 through the data bus 9 and the interface 2 to the buffer memory 5 in real time, and sends the edited data string through the data bus 9 and the interface 2 as a signal. The processing such as writing to the source 1 in real time is performed. Therefore, by using the moving image editing apparatus having this configuration, the same processing as in the flowchart of FIG. 1 or 2 is performed, and the first I frame and the next I frame or P frame are inserted in the first GOP header. When it is determined that there is a B frame, the data of the B frame existing between the first I frame and the next I frame or P frame in the first GOP header can be replaced with the data of the first I frame. .

[0028]

According to the first aspect of the present invention, when an edited bit stream is played back, it is possible to prevent the occurrence of a frame which should be played but is not played back. It is possible to achieve an editing process that can maintain the accuracy of the replaced image frame fairly high, and further, since there are no frames that are not played back, there is no need to perform compression / decompression processing, which makes the processing complicated. It has a unique effect that it is possible to prevent the deterioration.

According to the second aspect of the present invention, when the edited bit stream is reproduced, it is possible to prevent the occurrence of a frame which is not reproduced although it should be reproduced. It is possible to achieve an editing process that can maintain the accuracy of the captured image frame to a very high level, and since there are no frames that are not played back, there is no need to perform compression / decompression processing, which complicates the processing. It has a unique effect that it can be prevented.

According to the third aspect of the present invention, when the edited bit stream is reproduced, it is possible to prevent the occurrence of a frame which is not reproduced although it should be reproduced, and the frame is replaced. It is possible to achieve an editing process that can maintain the accuracy of the captured image frame to a very high level, and since there are no frames that are not played back, there is no need to perform compression / decompression processing, which complicates the processing. It has a unique effect that it can be prevented.

According to the fourth aspect of the present invention, when the edited bit stream is reproduced, it is possible to prevent the occurrence of a frame which should be reproduced but is not reproduced. It is possible to achieve an editing process that can maintain the accuracy of the captured image frame to a very high level, and since there are no frames that are not played back, there is no need to perform compression / decompression processing, which complicates the processing. It has a unique effect that it can be prevented.

[Brief description of drawings]

FIG. 1 is a flowchart illustrating a normal editing process.

FIG. 2 is a flowchart illustrating an embodiment of a moving image editing method of the present invention.

FIG. 3 is a schematic diagram showing an example of a GOP.

FIG. 4 is a block diagram showing an embodiment of a moving image editing apparatus of the present invention.

[Explanation of symbols]

 4 Central processing unit 6 Memory

Claims (4)

[Claims] 1. A bitstream consisting of an encoded moving image in which image frames compressed by the intra-frame prediction method and image frames compressed by the inter-frame prediction method are mixed, which are encoded using prediction. This is a method for editing in the code state with the target as the target, the bitstream is cut out before the image frame compressed by the intraframe prediction method, and all the images compressed by the interframe prediction method in the cutout bitstream are extracted. Image frames compressed by the inter-frame prediction method that cannot be displayed accurately are the most similar, provided that it is judged whether the frame can be displayed correctly and that an image frame compressed by the inter-frame prediction method that cannot be displayed correctly occurs. A moving image editing method characterized by replacing with an image frame compressed by an intra-frame prediction method that is highly likely to occur. 2. A bitstream composed of an encoded moving image in which image frames compressed by the intra-frame prediction method and image frames compressed by the inter-frame prediction method are mixed, which are encoded using prediction. This is a method for editing in the code state with the target as the target, the bitstream is cut out before the image frame compressed by the intraframe prediction method, and all the images compressed by the interframe prediction method in the cutout bitstream are extracted. It is determined whether or not a frame can be displayed accurately, and if an image frame compressed by the inter-frame prediction method that cannot be displayed correctly occurs, the image frame compressed by the inter-frame prediction method It is an inter-frame prediction method that is equivalent to displaying the image frame compressed by the intra-frame prediction method that is likely to be similar Moving picture editing method characterized by substituting reduced image frames. 3. A bitstream composed of an encoded moving image in which image frames compressed by the intra-frame prediction method and image frames compressed by the inter-frame prediction method are mixed, which are encoded using prediction. This is a device that edits in the code state with the target as the target, and cuts out the bitstream before the image frame compressed by the intraframe prediction method, and all the images compressed by the interframe prediction method in the cutout bitstream. Judgment means (4) for judging whether or not the frame can be displayed accurately
The condition (6) is similar to the image frame compressed by the inter-frame prediction method that cannot be accurately displayed, provided that the determination unit determines that an image frame compressed by the inter-frame prediction method that cannot be accurately displayed has occurred. A moving image editing apparatus comprising: replacement means (4) and (6) for replacing with an image frame compressed by an intra-frame prediction method having a high possibility. 4. A bitstream composed of an encoded moving image in which image frames compressed by the intra-frame prediction method and image frames compressed by the inter-frame prediction method, which are encoded using prediction, are mixed. This is a device that edits in the code state with the target as the target, and cuts out the bitstream before the image frame compressed by the intraframe prediction method, and all the images compressed by the interframe prediction method in the cutout bitstream. Judgment means (4) for judging whether or not the frame can be displayed accurately
The condition (6) is similar to the image frame compressed by the interframe prediction method that cannot be accurately displayed, provided that the determination means determines that an image frame compressed by the interframe prediction method that cannot be accurately displayed has occurred. It is characterized by including replacement means (4) (6) for replacing an image frame compressed by the intra-frame prediction method that is likely to be displayed as it is with an image frame compressed by the inter-frame prediction method. Video editing method.