JPH08307860A - Scene re-encoder - Google Patents

Abstract

PURPOSE: To facilitate edition of an image in the unit of scenes and re-coding. CONSTITUTION: When a scent change detection circuit 32 detects a scene change, a scene change flag is generated. A motion evaluation circuit 33 revises the coding method with respect to each image frame so as to inhibit prediction between scenes. A frame sequence rearrangement circuit 31 revises a frame sequence in response to the revised coding method and provides an output. Thus, a GOP is made to stop just before a scenes change and the coding frame in a scene change timing is set to a head of the GOP. Moreover, an overhead data generating circuit 34 sets a closed GOP just after the scene change. Thus, coding is conducted without predicting between scenes and the edit and re-coding in the unit of scenes are facilitated.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a scene re-encoding device capable of editing and re-encoding scene by scene.

[0002]

2. Description of the Related Art In recent years, digital processing of images has become popular with the establishment of high-efficiency image coding technology. The high-efficiency coding technique is for coding image data at a low bit rate in order to improve the efficiency of digital transmission and recording. In this high efficiency coding,
Orthogonal transformation such as DCT (discrete cosine transformation) processing is performed in block units of m × n pixels. Orthogonal transformation is to transform an input sample value into an orthogonal component such as a spatial frequency component. This makes it possible to reduce spatial correlation components. The orthogonally transformed components are quantized to reduce the redundancy of the signal of the block.

Further, variable length coding such as Huffman coding is applied to the quantized output to further reduce the data amount. Huffman coding performs coding based on the result calculated from the statistical code amount of the quantized output, assigning short bits to data with a high appearance probability and long bits to data with a low appearance probability. The variable length coding to be assigned reduces the total amount of data.

Further, in a device for performing high efficiency coding, a hybrid system which is being studied by MPEG (Moving Picture experts group) and the like is predominant. In this method, inter-frame compression (predictive coding) that reduces redundancy in the time axis direction by utilizing correlation between frames is adopted in addition to intra-frame compression that performs DCT processing on an image within a frame. Inter-frame compression further reduces the bit rate by obtaining the difference between the previous and next frames and encoding the difference value (prediction error) by using the property that general moving images are similar to each other. It is what makes me. In particular, motion-compensated interframe predictive coding that reduces the prediction error by predicting the motion of an image and obtaining the interframe difference is effective.

As described above, in the hybrid system, the image data of a predetermined frame and the reference image data of the frames before and after this frame are subjected to the intra-frame coding in which the image data of the predetermined frame is directly DCT processed and coded. Predictive coding in which only difference data is DCT processed and coded is adopted. The predictive coding method includes forward predictive coding for motion-compensating temporally forward reference image data to obtain a prediction error, and backward predictive coding for temporally motion-compensating backward-oriented reference image data to obtain a predictive error. There are predictive coding and bidirectional predictive coding using an average of either forward or backward or both directions in consideration of coding efficiency.

A reference image is required to decode a frame encoded in the interframe compression mode. That is, the original image cannot be restored only with the inter-frame compressed frame. On the other hand, a frame coded by intra-frame coding (hereinafter referred to as an I picture) is coded only by intra-frame information, and thus can be decoded by only single coded data. Therefore, M
In the PEG standard, I
One picture is inserted in a fixed cycle (for example, 12 frames). In the MPEG standard, an inter-frame coded frame (hereinafter referred to as P picture) is obtained by forward predictive coding using this I picture. Note that P
The picture can also be obtained by performing forward predictive coding on the forward P picture. In addition, a bidirectional prediction adaptive switching frame (hereinafter, referred to as B) by bidirectional predictive coding using either forward or backward I or P pictures in both directions.
Picture).

FIG. 10 is an explanatory diagram for explaining the compression method of this system. 10A shows an input frame image, FIG. 10B shows encoded data, and FIG.
(C) shows the decrypted data.

The frame image of frame number 0 is intra-frame coded. Using this frame image as a reference image, the frame image of frame number 3 is forward predictively encoded. The arrow in FIG. 10A indicates the prediction direction of such encoding, and the frame image of frame number 6 is also subjected to forward predictive encoding using the frame image of frame number 3 ahead as a reference image. Further, the frame images of frame numbers 1 and 2 are bidirectionally predictively coded using the frame images of frame numbers 0 and 3 as reference images. Also, frame numbers 4, 5
The bidirectional predictive coding is performed on the frame image of No. 3 using the frame images of frame numbers 3 and 6 as reference images.

That is, first, as shown in FIG.
The image data of frame number 0 is intra-frame coded to I
Get the picture. In this case, the image data of frame number 0 is framed by a memory or the like, and for example, it is divided into blocks of 8 pixels × 8 lines, and DCT processing is performed in block units. The DCT transform coefficient obtained by the DCT process is quantized using a predetermined quantized coefficient, and then variable length coding is performed to obtain coded data.

For the frame image with the frame number 1 input next, bidirectional predictive coding using the frame images with the frame numbers 0 and 3 is performed. Therefore, until the frame image with the frame number 3 is coded, it is stored in the memory. Hold. Similarly, the frame image of frame number 2 is also encoded after the frame image of frame number 3 is encoded. For the frame image of frame number 3, forward prediction coding is performed using the frame image of frame number 0 as a reference image to obtain a P picture (FIG. 10 (b)). That is,
The image data of frame number 0 is motion-compensated using the motion vector, and the difference (prediction error) between the motion-compensated reference image data and the image data of the current frame (frame of frame number 3) is DCT-processed. The variable length coding after quantizing the DCT transform coefficient is the same as the intraframe coding.

Next, the already encoded frame numbers 0, 3
I and P pictures of frame numbers 1 and 2
Frame images are sequentially bidirectionally predictively encoded. In this way, two B pictures are obtained as shown in FIG. Thereafter, similarly, as shown in FIG. 10B, the frame images of frame numbers 6, 4, 5, ... Are encoded in order to obtain P picture, B picture, B picture ,.

Thus, at the time of encoding, the encoding is performed in a frame order different from the actually input frame order. At the time of decoding, the frame order of the encoded data is returned to the original, and as shown in FIG.
Decoded data is output in the order of 1, 2, ....

FIG. 11 is a block diagram showing a compression device compatible with such an MPEG system.

Video data input via the input terminal 1 is supplied to the frame order rearrangement circuit 2. The frame order rearrangement circuit 2 changes the frame order of the input video data according to the above-described encoding method and outputs the frame order to the subtraction circuit 3 and the motion evaluation circuit 4. The frame order rearrangement circuit 2 outputs video data in units of two-dimensional data (hereinafter, referred to as block data) of horizontal 8 pixels × vertical 8 lines, for example.

Now, it is assumed that the intra-frame compression mode is designated. In this case, the switch 5 is off, and the subtraction circuit 3 outputs the input block data as it is to the DCT circuit 6. The DCT circuit 6 converts an input signal from a spatial coordinate axis component into a frequency component by 8 × 8 two-dimensional DCT processing. This makes it possible to reduce spatial correlation components. That is, the output (transformation coefficient) of the DCT circuit 6 is given to the quantization circuit 7, and the quantization circuit 7 requantizes the transformation coefficient with a predetermined quantization width to reduce the redundancy of the signal of one block. To do. The quantization circuit 7
Of the plurality of quantization tables having different quantization widths (quantization levels) is used to quantize using a table based on the generated code amount, the assigned set code amount, etc. It can be controlled.

The quantized data from the quantizing circuit 7 is zigzag-scanned from the low band to the high band in the horizontal and vertical directions for each block and is given to the variable length coding circuit 8. The variable length coding circuit 8 two-dimensionally Huffman-codes the quantized output based on a predetermined variable-length code table, such as a Huffman code table, and outputs a coded output. In the two-dimensional Huffman coding, a pair of data of the number of zero quantized outputs (zero run length) and the number of bits of non-zero coefficient is coded. As a result, short bits are assigned to data having a high appearance probability and long bits are assigned to data having a low appearance probability, thereby further reducing the transmission amount.

The coded output from the variable length coding circuit 8 is given to the rate buffer 9. The rate buffer 9 is composed of, for example, a FIFO (first in first out) memory, and outputs the input encoded output to the multiplexer 10 at a predetermined speed. The generation rate of the encoded output from the variable length encoding circuit 8 is a variable rate, and the rate buffer 9 absorbs the difference between the generation rate and the transmission rate. On the other hand, the overhead data generation circuit 11 generates overhead data such as various settings in encoding, such as a prediction method, a quantization scale, and a frame order, and outputs the overhead data to the multiplexer 10. Multiplexer 10
, Multiplexes the overhead data with the output of the rate buffer 9 and outputs the multiplexed data from the output terminal 12 to a transmission line (not shown).

On the other hand, in the interframe compression coding mode, the switch 5 is turned on. The block data from the frame order rearrangement circuit 2 is supplied to the subtraction circuit 2, and the subtraction circuit 2 is supplied with pixel data of a motion-compensated reference image block (hereinafter referred to as a reference block) from the switch 5, The difference from the block of the current frame is output to the DCT circuit 6 as a prediction error. In this case,
The DCT circuit 6 will perform DCT processing on the difference data.

The reference block is obtained by decoding the quantized output. That is, the output of the quantization circuit 7 is also given to the inverse quantization circuit 13. The inverse quantization circuit 13 inversely quantizes the quantized output, and the inverse DCT circuit 14 performs inverse DCT processing to restore the original video data. Since the output of the subtraction circuit 2 is the difference information, the output of the inverse DCT circuit 14 is also the difference information. The output of the inverse DCT circuit 14 is given to the adder 15. The output of the adder 15 is added through the frame memory 16, the motion compensation circuits 17 and 18, the selection circuit 19 and the switch 5.
The adder 15 adds the difference data to the data of the reference block from the selection circuit 19 to reproduce the block data (local decoded data) of the current frame, and outputs it to the frame memory 16. The frame memory 16 stores the block data from the adder 15 in association with the position on the screen.

The frame memory 16 has a capacity for storing the local decoded data for two frames (two images), and the stored local decoded data of the reference image is stored in the motion compensation circuits 17 and 18 and the motion evaluation circuit 4. Output. The motion evaluation circuit 4 detects a motion vector from the block data of the current frame from the frame rearrangement circuit 2 and the block data of the reference image from the frame memory 16 and outputs it to the motion compensation circuits 17 and 18. The motion compensation circuits 17 and 18 are
The block position of the local decoded data of the reference image stored in the frame memory 16 is corrected by the motion vector and output as a motion-compensated reference block.

The outputs of the motion compensation circuits 17 and 18 are coefficient multipliers 21 and 22.
Also given to. The coefficient units 21 and 22 multiply the input data by the coefficients α and β, and output it to the adder 23. Note that generally α = β = 1/2. The adder 23 adds the outputs of the coefficient units 21 and 22 and outputs the result to the selection circuit 19. The selection circuit 19 is
When the coding mode is designated by the motion evaluation circuit 4 and the forward prediction coding is performed, the output of the motion compensation circuit 17 is selected, and when the backward prediction coding is performed, the output of the motion compensation circuit 18 is selected. , In the case of performing bidirectional predictive coding, one of the motion compensation circuits 17 and 18 or the output of the adder 23 is selected. The reference block data from the selection circuit 19 is given to the subtraction circuit 3 and the adder 15 via the switch 5.

In this way, the motion-compensated reference block data is supplied to the subtraction circuit 2, and the subtraction circuit 2
Calculates the prediction error from the difference between the block of the current frame and the reference block. This allows the DCT for the prediction error.
Processing is performed. The motion vector from the motion evaluation circuit 4 is also given to the variable length coding circuit 8 via the terminal 20, and is variable length coded by the variable length coding circuit 8 based on a predetermined variable length code table and output. To be done.

As described above, in order to suppress error propagation and the like, the intra-frame compressed frame is inserted in a predetermined frame unit (hereinafter, referred to as GOP (Group of Picture)). FIG. 12 is an explanatory diagram showing the structure of a GOP. 12A shows an image frame input to the compression device, FIG. 12B shows an encoded frame output from the compression device, and FIG. 12C shows a prediction direction by an arrow. The subscripts I, P and B in FIG. 12 indicate frame numbers.

With the compression device of FIG. 11, the structure shown in FIG.
If the image frames B-2 and B-1 shown in (1) are to be encoded as B pictures, the image frames P-3 and I0 before and after these frames must be encoded first. Therefore, in the compression device, as shown in FIG. 12B, the image frames P-3, I0, B-2, and B-1 are encoded in this order and the encoded frame is transmitted. Similarly, in other frames, rearrangement in the frame order shown by the arrow between FIGS. 12A and 12B is performed according to the coding mode. In the example of FIG. 12, 1 GOP is composed of 9 frames. For example, GOP1 is a coded frame I0
To encoded frame B5. The GOP has an I picture arranged at the head. Further, in order to complete the encoding in GOP units, the last image frame (see FIG. 12A) of the original image frame group corresponding to 1 GOP is P
It is designed to be encoded as a picture.

By the way, in order to change the image quality of a part of the encoded data output from the compression device, it is possible to re-encode a part of the encoded data. By changing the coding parameter, some image data is coded again. Further, it is possible to edit a plurality of pieces of encoded data or one piece of encoded data.

According to the MPEG standard, editing is performed in GOP units. Also in this case, as shown in FIG. 12, the second and third encoded frames from the beginning of the GOP use the last P picture of the previous GOP as a reference image. For example, the coded frames B-2 and B-1 of GOP1 use the coded frame P-3 of GOP0 as a reference image as a reference image. Like this, GOP
Some coded frames of GOP may depend on the previous GOP, which is not completely independent. Therefore, when the GOP is used as a unit of editing, the second and third coded frames from the head of the GOP must be B pictures with only the coded frame at the head of the GOP as a reference image, and the code amount is May be slightly higher than without editing.

By the way, as a starting point of a portion to be re-encoded or an editing starting point (hereinafter referred to as an editing point), a scene switching timing can be considered. However, it has not been determined at which position in the GOP the scene change occurs, and if the scene change is the edit point, the edit point is G
It may not be the start position of OP. FIG. 12B shows that the edit point is immediately before the encoded frame P3. That is, in this case, the input image frame B1
The image frames after (see FIG. 12A) are re-encoded.

However, in this case, the GO before and after the edit point
In an attempt to maintain the continuity of P, the encoded frame P3
The coded frame I0 before the edit point is required to create the. As described above, when the scene change is used as the editing point, the editing cannot be performed while maintaining the continuity of the GOP. That is, when a scene change is used as an editing point, a coded frame that was a P or B picture before editing needs to be an I picture, and the code amount may be significantly different before and after editing or recoding. is there.

[0029]

As described above, conventionally, in the case of re-encoding or editing coded data, since the independence of each scene is low, the edit point is maintained while maintaining the continuity of GOP. There was a problem that it was not possible to set the scene change timing.

The present invention has been made in view of the above problems, and it is possible to maintain the independence of each scene and maintain the continuity of GOP even when performing the editing and re-encoding of each scene. It is an object of the present invention to provide a scene re-encoding device capable of performing the same.

[0031]

A scene re-encoding apparatus according to the present invention is an intra-frame encoding process for an input image frame based on input image data, and a unidirectional predictive encoding using a forward reference image. Processing or coding means for performing coding by bidirectional predictive coding processing using forward and backward reference images to output a coded frame; and controlling the coding means to perform the intra-frame coding processing, Coding control means for switching each coding mode of the directional predictive coding process and the bidirectional predictive coding process by a predetermined algorithm on a frame-by-frame basis and repeating the cycle in a predetermined number of frames,
A scene change detecting means for detecting a scene change of the input image frame, and a scene change detecting means for changing the control of the encoding mode of the encoding means by the encoding control means based on the detection result of the scene change detecting means. Predictive coding of the input image frame after the scene change timing using the input image frame before the timing as the reference image and predictive coding of the input image frame before the scene change timing using the input image frame after the scene change timing as the reference image Predictive coding control means for prohibiting predictive coding between certain scenes is provided.

[0032]

According to the present invention, the input image data is given to the coding means, and the coding mode is controlled under the control of the coding control means, that is, the intraframe coding processing, the unidirectional prediction coding processing or the bidirectional prediction coding. Encoding is performed according to each encoding mode of the encoding process. When a scene change occurs, predictive coding using the image frame immediately after the scene change timing is prohibited for the image frame immediately before the scene change timing, and the image frame immediately before the scene change timing is used for the image frame immediately after the scene change timing. Predictive coding that was previously used is prohibited. This allows
Inter-scene prediction is not performed, and independent encoding is performed for each scene. This facilitates editing and re-encoding on a scene-by-scene basis.

[0033]

Embodiments of the present invention will be described below with reference to the drawings. FIG. 1 is a block diagram showing an embodiment of a scene re-encoding device according to the present invention. In FIG. 1, the same components as those in FIG. 11 are designated by the same reference numerals.

In this embodiment, when a scene change occurs, the GOP boundary position can be changed by changing the picture type and coding order of the coded frame so that the scene change timing and the GOP boundary match. It is a thing. As a result, even when the scene change timing is the editing point, GOPs can be made continuous before and after editing or re-encoding, and the code amount control becomes easy.

Video data that has been subjected to predetermined preprocessing is input to the input terminal 1. This video data is supplied to the frame rearrangement circuit 31 and also to the scene change detection circuit 32. The frame order rearrangement circuit 31
The frame order of the input video data is changed according to the encoding method and output to the subtraction circuit 3 and the motion evaluation circuit 33.
The frame rearrangement circuit 31 outputs video data in units of two-dimensional data (hereinafter, referred to as block data) of, for example, horizontal 8 pixels × vertical 8 lines.

In this embodiment, the frame rearrangement circuit 31 is controlled by the scene change detection circuit 32. The scene change detection circuit 32 has a frame memory (not shown), detects a scene change by comparing front and rear images, and rearranges the scene change flag into a frame order rearrangement circuit 31, a motion evaluation circuit 33, and overhead data generation. It is designed to output to the circuit 34.

The frame order rearrangement circuit 31 changes the frame order according to the encoding method for each image frame. In this embodiment, the image frame immediately before the scene change is encoded as a P picture, the image frame immediately after the scene change is encoded as an I picture, and the GOP immediately after the scene change is closed G.
Set to OP. In addition, closed GOP is other GO
It is a closed GOP that does not use the P image frame as a reference image. As a result, encoding is performed so that prediction between scenes is prohibited.

For example, the frame order rearrangement circuit 31 uses P
The I picture or P picture that is the reference image is output before the image frame that is encoded as a picture, and the I picture or P picture that is the reference image is output before the image frame that is encoded as the B picture. The frame order rearrangement circuit 31 outputs information on the frame output order to the motion evaluation circuit 33.

The motion evaluation circuit 33 is a frame order rearrangement circuit.
The block data from 31 is given, and the data of the reference image is given from the frame memory 16 described later, and the motion vector between the current frame and the reference frame is obtained and output to the motion compensation circuits 17 and 18. ing. Further, the motion evaluation circuit 33 determines the picture type of the current frame from the frame rearrangement circuit 31 based on the information on the output order of the frames and the scene change flag, and outputs a control signal based on the picture type to the selection circuit 19. To do.

The subtraction circuit 3 is supplied with the motion-compensated image data of the reference block from the switch 5, and in the inter-frame compression mode, the difference between the block data from the frame order rearrangement circuit 31 and the reference block data is used as the prediction error DCT. Supply to the circuit 6. In the intra-frame compression mode, the switch 5 is turned off, and the subtraction circuit 3 gives the output of the frame order rearrangement circuit 31 to the DCT circuit 6 as it is. The DCT circuit 6 DCT-processes the output of the subtraction circuit 3 and outputs it to the quantization circuit 7. The quantizing circuit 7 quantizes the DCT transform coefficient, and the variable length coding circuit 8 and the inverse quantizing circuit 13
Output to. The variable length coding circuit 8 performs variable length coding on the quantized output from the quantization circuit 7 and outputs it to the rate buffer 9.

The rate buffer 9 is composed of, for example, a FIFO memory, absorbs the difference between the generation rate and the transmission rate of the coded output from the variable length coding circuit 8 and multiplexes the coded output at a predetermined speed. Output to 10. The overhead data generation circuit 34 generates overhead data such as various settings in encoding, such as a prediction method, a quantization scale, and a frame order.

In the present embodiment, the overhead data generation circuit 34 is given a scene change flag from the scene change detection circuit 32. If it is indicated that a scene change has occurred, the overhead data generation circuit 34 determines that each coded frame of the GOP immediately after the scene change does not use the coded frame of the GOP before the scene change as a reference image, that is, A closed flag indicating that the GOPs are independent is included in the overhead data and output. The multiplexer 10 multiplexes the overhead data on the output of the rate buffer 9 and outputs it from the output terminal 12 to a transmission line (not shown).

The inverse quantization circuit 13 inversely quantizes the quantized output, and the inverse DCT circuit 14 inversely DCT-processes the inversely quantized output to restore the pixel data before the DCT processing, and supplies the pixel data to the adder 15. The reference image data is given to the adder 15 through the switch 5, and the prediction error from the inverse DCT circuit 14 and the reference image data are added to restore the data of the current frame (local decoded data) to obtain the frame. Output to memory 16.

The frame memory 16 can store reference image data for two frames. Frame memory 16
Is provided with the restored image data of the I and P pictures to be the reference image and held by the adder 15. Frame memory 16
Output the reference image data held at the decoding timing of the corresponding P and B pictures to the motion compensation circuits 17 and 18 and the motion evaluation circuit 33.

The motion compensation circuits 17 and 18 are frame memories, respectively.
The reference image data from 16 is motion-compensated based on the motion vector from the motion evaluation circuit 33 and output. Motion compensation circuit
The outputs of 17 and 18 are given to coefficient units 21 and 22, respectively, and
It is also given to the selection circuit 19. The coefficient units 21 and 22 multiply the output of the motion compensation circuit 17 by a predetermined coefficient and output it to the adder 23. The adder 23 adds the outputs of the coefficient units 21 and 22 to select circuit
Output to 19.

Under the control of the motion evaluation circuit 33, the selection circuit 19 selects the output of the motion compensation circuit 17 in the case of encoding as a P picture and the prediction in the case of encoding as a B picture. The output of the motion compensation circuits 17 and 18 or the adder 23 is selected according to the direction and output to the switch 5. The switch 5 is controlled by the mode signal at the terminal 24 and is turned off in the intra-frame compression mode and turned on in the inter-frame compression mode.

Next, the operation of the embodiment thus constructed will be described with reference to the explanatory views of FIGS. 2 and 3 show the planned picture type of the input image frame and the picture type of the output encoded frame by the symbols I, B, and P, and the frame numbers are indicated by the subscripts. 2A shows an input image frame, FIG. 2B shows a change in picture type due to a scene change, and FIG. 2C shows an output and transmitted encoded frame.

The video data which has been subjected to the predetermined pre-processing is input to the frame rearrangement circuit 31 and the scene change detection circuit 32 via the input terminal 1. As shown in FIG. 2 (a), the image frame I0 of frame number 0 is set as an I picture, and thereafter, B, B, P, B, and
The picture types are set in the order of B, P, B, B, I, .... That is, in this case, one GOP is composed of 9 frames.

Now, assume that the scene change detection circuit 32 has not detected a scene change. In this case, the frame order rearrangement circuit 31 changes the frame order according to the same rule as in FIG. 10, for example. The block data from the frame order rearrangement circuit 31 is given to the subtraction circuit 3 and the motion evaluation circuit 33. Assuming that the image frame of the input video data is set to the I picture, the switch 5 is off and the subtraction circuit 3 outputs the input block data to the DCT circuit 6 as it is.

The DCT circuit 6 converts the block data into a two-dimensional D
The CT processing is performed and the result is output to the quantization circuit 7. The DCT transform coefficient is quantized by the quantization circuit 7 to reduce the code amount. The quantized output is subjected to variable length coding by the variable length coding circuit 8 and given to the rate buffer 9. The rate buffer 9 provides the variable length coded output to the multiplexer 10 at a predetermined speed. The multiplexer 10 multiplexes the overhead data from the overhead data generation circuit 34 on the variable length coded output and outputs it.

On the other hand, the output of the quantizing circuit 7 is also output to the inverse quantizing circuit 13 in order to create a reference image. The quantized output is inversely quantized by the inverse quantization circuit 13, and inverse DC
Inverse DCT processing is performed by the T circuit 14 to restore the data before DCT processing, and the data is supplied to the adder 15. Since the switch 5 is off when creating an I picture, the adder 15 stores the output of the inverse DCT circuit 14 in the frame memory 16 as it is as local decoded data.

Next, it is assumed that the video data set in the P picture is input. In this case, the switch 5 is turned on. The motion evaluation circuit 33 is a frame rearrangement circuit
A motion vector is obtained from the output of 31 and the output of the frame memory 16 and output to the motion compensation circuit 17. The motion compensation circuit 17 corrects the block position of the reference image stored in the frame memory 16 by the motion vector, and outputs the motion-compensated reference block data via the selection circuit 19. This reference block data is sent to the subtraction circuit 3 via the switch 5.
The subtraction circuit 3 outputs the difference from the current block data to the DCT circuit 6 as a prediction error.

Thus, in this case, the prediction error is DC
A P picture is created by performing T processing, quantization processing, and variable length coding processing. In addition, the quantization output is an inverse quantization circuit.
13 and the inverse DCT circuit 14 restores the data before the DCT processing and supplies it to the adder 15. Since the output of the subtraction circuit 3 is the prediction error, the output of the inverse DCT circuit 14 is also the prediction error. The adder 15 is supplied with the motion-compensated reference block data up to the previous frame via the switch 5, and the adder 15 adds the reference block data to the prediction error to restore the image data of the current frame. . This image data is stored in the frame memory 16 as reference image data.

Next, it is assumed that the video data set for the B picture is input. Also in this case, the switch 5 is turned on. The motion evaluation circuit 33 is a frame rearrangement circuit
A motion vector is obtained from the output of 31 and the output of the frame memory 16 and output to the motion compensation circuits 17 and 18. Motion compensation circuit
A block 17 corrects the blocking position of the forward reference image stored in the frame memory 16 with a motion vector, and outputs motion-compensated reference block data. Further, the motion compensation circuit 18 corrects the blocking position of the backward reference image stored in the frame memory 16 by the motion vector and outputs the motion-compensated reference block data.

The outputs of the motion compensation circuits 17 and 18 are coefficient units 2 respectively.
After being multiplied by the coefficients α and β by 1 and 22, they are added by the adder 23 and given to the selection circuit 19. Selection circuit 19
Is controlled by the motion evaluation circuit 33 to select the output of the motion compensation circuits 17, 18 or the adder 23 according to the prediction direction. In this way, the motion-compensated reference block data is given to the subtraction circuit 3 via the switch 5, and the subtraction circuit 3 outputs a prediction error from the current frame. This prediction error is DCT processing,
Quantization processing and variable length coding processing are performed to create a B picture using reference images in the front, rear, or both directions.

Here, the frame number 3 shown in FIG.
It is assumed that a scene change occurs at the timing of the image frame P3. The image frame P of FIG.
-3 corresponds to GOP0, and image frames B-2 to P6
Corresponds to GOP1, and frames after the image frame B7 correspond to GOP2. Scene change detection circuit 32
When detecting a scene change, outputs a scene change flag to the frame rearrangement circuit 31, the motion evaluation circuit 33, and the overhead data generation circuit 34.

In this embodiment, when a scene change occurs during GOP, the GOP is divided at this timing. That is, as shown in FIG. 2B, GO from the -2nd image frame to the 2nd image frame among the -2nd image frame to the 6th image frame corresponding to GOP1.
P1,1 and G from the third frame to the sixth frame
OP1 and OP2. And these GOP 1,1 and G
The encoding method is changed so as not to perform predictive encoding between OP1 and OP2, that is, between scenes. That is, as shown in FIG. 2B, the last second image frame of GOP1, 1 is changed from the B picture to the P picture (or I picture). Similarly, the first third image frame of GOP1 and GOP2 is changed from a P picture to an I picture. Also, the frame order is changed in accordance with the change of the picture type.

When the scene change flag occurs at the timing of the image frame P3 which was to be encoded as a P picture in the middle of GOP, the frame order rearrangement circuit 31 generates the last image frame P2 of GOP1,1 and the immediately preceding image frame. Swap the output order with B1. Further, the third image frame I3 is output as the first frame of GOP1 and 2 after the scene change, and the image frame P6 is output second as the reference image of the image frames B4 and B5 after the scene change. The frame order rearrangement circuit 31 outputs information on the output order of frames to the motion evaluation circuit 33.

The video data from the frame order rearrangement circuit 31 is given to the subtraction circuit 3 and the motion evaluation circuit 33. When the image frame P2 is input, the motion evaluation circuit 33 obtains a motion vector with the image frame I0 and outputs it to the motion compensation circuit 17. In this case, the motion evaluation circuit 33
Causes the selection circuit 19 to select the output of the motion compensation circuit 17. Thus, the subtraction circuit 3 is supplied with the motion-compensated reference block data with the image frame I0 to obtain the prediction error. As a result, the image frame P2 is forward predictively encoded and output to the multiplexer 10.

When the next image frame B1 is input, the motion evaluation circuit 33 obtains a motion vector between the image frames I0 and P2 and outputs it to the motion compensation circuits 17 and 18. Further, in this case, the motion evaluation circuit 33 causes the selection circuit 19 to select the output of either the motion compensation circuits 17 and 18 or the adder 23. In this way, the subtraction circuit 3 operates the image frames I0 and P2.
The motion-compensated reference block data is given between and to obtain the prediction error. As a result, the image frame B
1 is bidirectionally predictively coded and output to the multiplexer 10.

The next image frame I3 is directly supplied to the DCT circuit 6 via the subtraction circuit 3 and subjected to intra-frame compression processing. Next, when the image frame P6 is input, the motion evaluation circuit 33 obtains a motion vector with the image frame I3 and gives it to the motion compensation circuit 17,
The motion-compensated reference block data from 17 is output to the subtraction circuit 3. In this way, the subtraction circuit 3 obtains the prediction error between the image frame P6 and the image frame I3, and calculates the DC error.
Output to the T circuit 6. The image frame P6 is forward predictively encoded and output as an encoded frame P6.

Similarly, the next image frame B4 is coded by bidirectional predictive coding using the image frames I3 and P6 as reference images to obtain a coded frame B4. The image frame B5 to be encoded next is also encoded by bidirectional predictive encoding using the image frames I3 and P6 as reference images.

As described above, each image frame of GOP1 and GOP2 is subjected to predictive coding using only the image frame in GOP1 and GOP2. That is, GOP1,1 and GOP1,2
It does not use inter-scene prediction and is independent of each other.

The scene change flag of the scene change detection circuit 32 is also supplied to the overhead data generation circuit 34. The overhead data generation circuit 34 uses the GOP
A closed flag indicating that 1 and 2 are closed GOPs is generated, and the multiplexer is used as overhead data.
Output to 10. The multiplexer 10 multiplexes the encoded data from the rate buffer 9 with the overhead data and outputs it.

It is assumed that the encoded output created in this way is edited and re-encoded at the scene change timing. That is, in this case, re-encoding is started at the timing of the encoded frame I3 in FIG. 2 (c). In this case, the image frames I3 to P6 corresponding to the coded frames I3 to B5 forming the GOPs 1 and 2 are shown in FIG.
Re-encoding with the same picture type as in (b). Since GOP 1,1 before editing and GOP 1,2 are independent from each other, almost the same code amount control can be performed with and without editing.

Next, assume that a scene change occurs at the timing of a B picture in a GOP. Also in this case, GOP1 is divided into GOP1,1 and GOP1,2 as shown in FIGS.
The picture type of the last image frame is changed to P, and the picture type of the image frame initially set to the P picture in GOP1 and GOP2 is changed to I. The frame order rearrangement circuit 31 first sets P after the scene change.
The image frame P that was to be encoded as a picture
3 is output as the first image frame I3 after the scene change, and then the image frame B2 at the scene change timing is output.

That is, when a scene change occurs, the frame order rearrangement circuit 31 outputs the third image frame I3 as the first image frame of GOP1 and GOP2. This image frame I3 is directly supplied to the DCT circuit 6 via the subtraction circuit 3 and is compressed in the frame. The scene change flag is also supplied to the overhead data generation circuit 34, and the closed flag is output as overhead data.

Next, the frame order rearrangement circuit 31 outputs the image frame B2. When no scene change occurs, the image frame B2 is the front image frame P1.
And the subsequent image frame I3 are coded as reference images. In this case, since the inter-scene prediction is prohibited by the closed flag, the image frame P1 is not used as the reference image. Only the image frame I3 in GOP1 and GOP2 is used as the reference image. Motion evaluation circuit
A motion vector 33 is used to obtain a motion vector with respect to the image frame I3, and the motion compensation circuit 18 motion-compensates the locally decoded data of the image frame I3 and outputs it as reference block data. As a result, the image frame B2 is backward predictively coded.

In the subsequent image frames in GOP1 and GOP2, the image frames in GOP1 and GOP2 are used as reference images for predictive coding. As a result, even in this case, the GOP 1,1 and the GOP 1,2 are independent of each other.

In this way, at the time of editing, it is possible to edit in scene change units.

As described above, in this embodiment, when a scene change is detected, the GOP is divided at this timing, the picture type of the image frame immediately before the scene change is set to P, and the P picture is first displayed after the scene change. The picture type of the image frame that was to be encoded as is set to I. Also, the frame order of encoding is changed according to the change of the picture type so that the GOPs before and after the scene change are mutually independent. This enables editing at the scene change timing.

In this embodiment, the image frames corresponding to 1 GOP are B, B, I, B, B, P, B,
Although described as B and P, it is obvious that image frames having different configurations may be used.

By the way, in FIG. 2, the GOP is divided at the scene change timing to create two GOPs which are independent of each other from the image frame corresponding to one GOP. On the other hand, a method of completing the GOP with the images up to the scene change and newly creating a GOP similar to the normal after the scene change is also conceivable. Even in this case,
It can be realized by a circuit configuration similar to that described above, and only the algorithm for determining the picture type and changing the frame order by the frame rearrangement circuit 31 and the motion estimation circuit 33 need be changed.

FIGS. 4 and 5 are explanatory views for explaining the determination of the picture type and the change of the frame order in this case. FIGS. 4 and 5A to 5C correspond to FIGS. 2 and 3A to 3C, respectively.

When a scene change is detected during the GOP, the GOP is completed with the image frames up to the scene change. For example, even if one GOP is composed of N coded frames, the GOP is completed by a scene change, and GO is executed with a coded frame of less than N.
Configure P. Then, a new 1 is added using N image frames after the image frame in which the scene change is detected.
Configure GOP. In order to complete the GOP, as described above, the image frame immediately before the scene change may be changed to the P picture (or I picture).

Now, assume that the image frame shown in FIG. 4A is input. If no scene change occurs, nine image frames B-2 to P6 correspond to GOP1. When a scene change occurs at the timing of image frame P3, image frame B
Complete GOP1 by 2 That is, the prediction is not performed between GOP1 and the next GOP2 ', as shown in FIG.
The picture type of the last image frame B2 is changed to P as shown in FIG.

Since the next GOP 2'is a normal GOP using 9 image frames, the picture type of the 3rd and 4th image frames is set to B and the picture of the 5th image frame is set to B, as shown in FIG. 4B. Let type be I. Also, GOP
In order to prohibit the prediction using one encoded frame,
OP2 'is closed.

The frame order rearrangement circuit 31 is shown in FIG.
When the scene change flag is generated at the timing of the image frame P3 shown in FIG.
The output order of 2 and the immediately preceding image frame B1 is exchanged.
Further, the motion evaluation circuit 33 uses the image frame I0 as a reference image of the image frame P2 and the image frames I0 and P2 as a reference image of the image frame B1. As a result, as shown in FIG. 4C, the coded frames I0,
A GOP1 composed of five frames B-2, B-1, P2 and B1 is created.

Next, the frame order rearrangement circuit 31 outputs the image frame I5. This image frame I5 is compressed in the frame. Next, the frame order rearrangement circuit 31 outputs the image frame B3. In this case, the overhead data generation circuit 34 has generated the closed flag, and the motion evaluation circuit 33 enables backward prediction using the image frame I5. Similarly, backward prediction using the image frame I5 is performed for the image frame B4. The encoding of the next image frame P8 and the subsequent ones is the same as the normal GOP creation.

Thus, in this case, GOP1 is completed at the scene change timing, and the next GO
Coding is performed with P2 'closed. Therefore, even when the editing is performed at the scene change timing, the continuity of GOP can be maintained as in the case before the editing, and the code amount control and the like becomes easy.

FIG. 5 shows an example in which a scene change occurs at the timing of the image frame B2 in the middle of GOP1. In this case, the motion evaluation circuit 33 uses the GOP
The last image frame P1 (FIG. 5 (a)) corresponding to 1 is directly used as a P picture (FIG. 5 (b)). This completes GOP1.

A GOP2 'is created by nine frames after the next image frame B2. Frame order rearrangement circuit
31 first outputs the image frame I4 as the head frame of GOP2 '. This image frame I4 is intraframe compressed. A closed flag is set in GOP2 '. Next, the image frame B2 is output from the frame order rearrangement circuit 31. The motion evaluation circuit 33 specifies only the rear image frame I4 as the reference image of the image frame B2. Similarly, only the image frame I4 is used as the reference image of the image frame B3 to be output next.
As a result, GO is applied to all image frames of GOP2 '.
Closed encoding is performed using only the image frames in P2 '.

Thus, also in this case, it is possible to edit at the scene change timing while maintaining the continuity of GOP.

Further, it is possible to edit the scene unit by closing the GOP in which the scene change has occurred. 6 and 7 are explanatory diagrams for explaining the determination of the picture type and the change of the frame order in this case. FIGS. 6 and 7A to 7C correspond to FIGS. 2 and 3A to 3C, respectively. In addition,
Also in this case, it can be realized by changing the algorithm for determining the picture type and changing the frame order by the frame order rearrangement circuit 31 and the motion estimation circuit 33.

When a scene change is detected in the middle of a GOP, the GOP to which the scene change belongs is closed and encoded. That is, in encoding each image frame of this GOP, prediction using the image frame of the previous GOP is not performed. Further, each image frame before the scene change is subjected to the predictive coding using only the image frame before the scene change, and each image frame after the scene change is subjected to the predictive coding using only the image frame after the scene change.

Now, assume that the image frame shown in FIG. 6A is input. If no scene change occurs, nine image frames B-2 to P6 correspond to GOP1. It is assumed that a scene change occurs at the timing of the image frame P3. Then, the overhead data generation circuit 34 responds to the GO by the scene change flag from the scene change detection circuit 32.
Overhead data indicating that P1 is closed is generated. Also, in order not to predict between scenes,
As shown in FIG. 6B, the picture type of the image frames B1 and B2 is changed to P. Similarly, the picture type of the image frame P3 immediately after the scene change is changed to I.

The frame order rearrangement circuit 31 outputs the image frame I0 as the first frame of the closed GOP1. This image frame I0 is output after being compressed in the frame. Next, the frame order rearrangement circuit 31 sequentially outputs the image frames B-2 and B-1. Since GOP1 is set to be closed, the motion estimation circuit 33 selects the motion compensation circuit 18 at the time of encoding these image frames and adopts the predictive encoding using only the backward reference image.

Next, the coded frames P1 and P2 are created using the image frame I0. Encoded frames P1 and P
2 does not use the image frame I3 for prediction, and the coded frames B-2 to P2 of the closed GOP1 are independently coded in GOP1.

Next, the image frame I after the scene change
Compress 3 in frame. Further, the image frame P6 is forward predictively encoded using the image frame I3. Next, using these image frames I3 and P6, the image frames B4 and B5 are bidirectionally predictively coded. These image frames I3 to P6 are also independent of each other without using the image frames before the scene change in GOP1 and the image frame of GOP2.

When re-encoding the encoded data created in this way, re-encoding is started from the first frame of the closed GOP. In this case, since GOP1 is closed, re-encoding is possible from the first frame of GOP1. By performing re-encoding without changing the encoding parameter, each encoded frame before the scene change becomes the same as before re-encoding. After the image frame I3, new encoding parameters are set and re-encoding is performed. Also in this case, each image frame after the scene change of the closed GOP 1 is independently coded, and re-coding is possible.

As described above, also in this case, the same effect as in the case of FIGS. 2 to 5 can be obtained.

FIG. 7 shows an example in which a scene change occurs at the timing of image frame B2 in the middle of GOP1. In this case, the picture type of the image frame B1 immediately before the scene change is changed to P, and the picture type of the image frame B2 immediately after the scene change is I.
Change to By changing the image frame B1 to the P picture, the prediction using the image frame P3 immediately after the scene change becomes unnecessary, and each image frame before the scene change of GOP1 can be closed and encoded. Also,
Similarly, with respect to each image frame after the scene change of GOP1, since the image frame B2 is changed to the I picture, closed encoding is possible.

Therefore, even when the coded data thus created is edited at the scene change timing, the continuity of GOP can be maintained.

Further, in this embodiment, it is possible to adopt the method shown in FIGS. 8 and 9. 8 and 9 are explanatory diagrams for explaining the determination of the picture type and the change of the frame order, and FIGS. 8 and 9A to 9C are FIGS. 2 and 3A to 3C, respectively. ) Is supported.

When the edit point is set at the boundary of the GOP, conventionally, the closed flag is set to the GOP immediately after the edit point, which indicates that the prediction using the image frame of the previous GOP is not performed. However, as described above, prediction is used before editing, GOPs are not independent, and the same code amount control cannot be performed with and without editing.

Therefore, in order to easily handle editing,
When a scene change occurs at the boundary of GOP, the GOP immediately after the scene change is set to closed at the time of encoding. Furthermore, when a scene change occurs during GOP, the GO next to the GOP including the scene change
Set P to closed. In this case, editing cannot be performed by changing the scene, but can be surely performed by GOP.

Now, assume that the image frame shown in FIG. 8A is input. The image frames B-2 to P6
9 image frames correspond to GOP1. GOP1
It is assumed that a scene change occurs at the timing of the image frame B2 which is the first frame of the. Then, the overhead data generation circuit 34 becomes the scene change detection circuit.
The scene change flag from 32 generates overhead data indicating that GOP1 is closed. That is, the image frame of GOP0 is not used for the prediction of GOP1.

The frame order rearrangement circuit 31 outputs the image frame I0 as the first frame of the closed GOP1. This image frame I0 is output after being compressed in the frame. Next, the frame order rearrangement circuit 31 sequentially outputs the image frames B-2 and B-1. Since GOP1 is set to be closed, the motion estimation circuit 33 selects the motion compensation circuit 18 at the time of encoding these image frames and adopts the predictive encoding using only the backward reference image. Thus, the closed encoding is performed on GOP1.

It is assumed that the coded output thus constructed is edited at the scene change timing. In this case, since GOP1 is set to closed,
Even if a scene change is performed, that is, re-encoding is performed from the first frame of GOP1, the continuity of GOP is maintained before and after editing. In this way, since the scenes are encoded independently, editing and re-encoding for each scene becomes easy.

FIG. 9 shows an example in which a scene change occurs at the timing of the image frame B4 in the middle of GOP1. In this case, the scene change is GOP
It is determined that the error occurs at the timing of the first frame of No. 2, and GOP2 is set to closed (FIG. 9 (c)).
That is, regarding the image frames B7 and B8 of GOP2,
Predictive coding is performed using the image frame I9 as the backward reference image.

When editing the encoded data thus configured, the scene change is assumed to be the head of GOP2. In this way, by performing editing from the first frame of GOP2, the continuity of GOP can be maintained before and after editing. GO decision on scene change
By setting the P unit, it is not necessary to change the frame order due to the scene change, and there is an advantage that the circuit configuration can be simplified.

[0102]

As described above, according to the present invention, it is possible to maintain the independence of each scene and maintain the continuity of GOP even when performing the editing and re-encoding of each scene. Have.

[Brief description of drawings]

FIG. 1 is a block diagram showing an embodiment of a scene re-encoding device according to the present invention.

FIG. 2 is an explanatory diagram for explaining the operation of the embodiment.

FIG. 3 is an explanatory diagram for explaining the operation of the embodiment.

FIG. 4 is an explanatory diagram for explaining the operation of the embodiment.

FIG. 5 is an explanatory diagram for explaining the operation of the embodiment.

FIG. 6 is an explanatory diagram for explaining the operation of the embodiment.

FIG. 7 is an explanatory diagram for explaining the operation of the embodiment.

FIG. 8 is an explanatory diagram for explaining the operation of the embodiment.

FIG. 9 is an explanatory diagram for explaining the operation of the embodiment.

FIG. 10 is an explanatory diagram for explaining a compression method of the MPEG system.

FIG. 11 is a block diagram showing a compression device compatible with the MPEG system.

FIG. 12 is an explanatory diagram showing the structure of a GOP.

[Explanation of symbols]

17, 18 ... Motion compensation circuit, 19 ... Selection circuit, 31 ... Frame order rearrangement circuit, 32 ... Scene change detection circuit, 33 ... Motion evaluation circuit, 34 ... Overhead data generation circuit

Claims (9)

[Claims] 1. An intraframe coding process for an input image frame based on input image data, a unidirectional predictive coding process using a forward reference image, or a bidirectional predictive code using a forward and backward reference image. Coding means for performing coding by coding processing and outputting a coded frame; and controlling the coding means to code each of the intra-frame coding processing, unidirectional predictive coding processing and bidirectional predictive coding processing. Encoding control means for switching the encoding mode on a frame-by-frame basis by a predetermined algorithm and repeating the cycle at a predetermined number of frames; scene change detecting means for detecting a scene change of the input image frame; and a detection result of the scene change detecting means. Based on the above, the control of the encoding mode of the encoding means by the encoding control means is changed so that the input before the scene change timing is changed. Between the scenes that are predictive coding of the input image frame after the scene change timing using the input image frame as the reference image and the predictive coding of the input image frame before the scene change timing that use the input image frame after the scene change timing as the reference image And a predictive coding control means for prohibiting predictive coding of the scene re-encoding device. 2. A group of pictures in which a plurality of coded frames of the predetermined number of frame cycles output from the coding means have an intra-frame coded frame created at the beginning by an intra-frame coding process. The scene re-encoding device according to claim 1, wherein 3. The predictive coding control means, a frame order changing means for changing a frame order of a coding process of the coding means, a selection means for selecting a coding mode of the coding means, and a predetermined group. 3. The scene re-encoding apparatus according to claim 2, further comprising: a closed group of picture setting unit that sets the of picture to a closed group of picture for which prediction encoding using another group of picture is prohibited. . 4. The scene re-encoding apparatus according to claim 2, further comprising a multiplexing unit that multiplexes information indicating the closed group of pictures to an output of the encoding unit. 5. The predictive coding control means creates the last frame of the group of pictures immediately before the scene change timing by intra-frame coding processing or unidirectional predictive coding processing, thereby inputting after the scene change timing. The scene re-encoding apparatus according to claim 2, wherein predictive encoding of an input image frame before a scene change timing using an image frame as a reference image is prohibited. 6. The predictive coding control means divides the group of pictures at a scene change timing,
The group of pictures immediately after the scene change timing is set to a closed group of pictures for which prediction coding using another group of pictures is prohibited to prohibit prediction coding between the scenes. The scene re-encoding device according to 1. 7. The predictive coding control means sets the group of pictures including a scene change to a closed group of picture for which prediction coding using another group of pictures is prohibited, and predicts between the scenes. The scene re-encoding device according to claim 2, wherein encoding is prohibited. 8. The predictive coding control means sets a group of picture immediately after a scene change to a closed group of picture in which predictive coding using another group of picture is prohibited, and the predictive coding between scenes is performed. The scene re-encoding device according to claim 2, wherein the encoding is prohibited. 9. The predictive coding control means determines that the group of picture immediately after the scene change is the scene change timing when the scene change does not occur in the first frame of the group of picture. Item 9. The scene re-encoding device according to item 8.