JPH11239319A - High efficiency coded image signal compositing device and editor - Google Patents

Abstract

PROBLEM TO BE SOLVED: To obtain an image signal that is overlap-processed without the need for an orthogonal transform means and a quantization means and without deterioration by special conversion processing. SOLUTION: The number of frames for overlap processing is set to a parameter register 56. A sequencer 57 activates a frame counter based on a total frame number for overlap processing set in the parameter register 56. An orthogonal transform coefficient composite circuit 59 applies composite processing to two referenced image data stored in a data memory 55 based on their discrete cosine transform DCT coefficients by taking a frame decided by a syntax setting circuit 58 into account and the composite result is fed to a multiplexer 62 via a quantization circuit 60 and a variable length coding circuit 61. The composite stream is outputted for only the overlapped periods.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a high-efficiency coded image signal synthesizing apparatus and an editing apparatus, and more particularly to a high-efficiency coded image signal synthesizing apparatus and an editing apparatus for handling a high-efficiency coded image signal.

[0002]

2. Description of the Related Art In recent years, information obtained by compressing a digitized image signal by high-efficiency coding on a medium such as an optical disk or a magnetic medium is reproduced. A recording / reproducing system has been developed and put into practical use for performing decoding processing to restore and reproduce an image signal. As a typical high-efficiency encoding method used in these recording and reproducing systems, there is MPEG2 which is encoding for moving images.

In MPEG2, a reference image (I frame) is first determined, and the image data is horizontally
DCT (Discrete Cosine Transform), which is a type of orthogonal transform, is divided into vertical two-dimensional blocks and the divided blocks are united
After processing, each of the DCT-transformed coefficients is subjected to variable-length coding.

Since the DCT-converted component has a value corresponding to the frequency component of the image, the amount of information is obtained by coarsely quantizing the high frequency component and finely quantizing the low frequency component in consideration of human visual characteristics. Reduce. The DC component, which is the lowest frequency component, represents the average value of the block, and the difference data is subjected to variable-length encoding using the correlation of the image between the blocks. In the quantized high-frequency component (AC component), 0s are often continuous, and the DCT transform coefficients are sequentially scanned from the low frequency to the high frequency, and the number of consecutive 0s and the non-zero component. Is variable-length coded. With the above processing, the DCT transform coefficient can be represented with a small amount of information.

Further, the motion component of another image (P frame, B frame) is obtained by using the reference image, and predictive coding is performed. Motion vector detection is performed in divided two-dimensional block units, and horizontal and vertical motion components are obtained as two-dimensional vectors. A prediction block is constructed by subtracting the value of the reference image block at the position shifted by the obtained motion vector from the value of the block of the image to be encoded, and performing orthogonal transform on the prediction block in the same manner as the reference image to perform variable length encoding. .
Larger information compression can be performed by performing predictive coding. The P frame can further serve as a reference image, and is used for predictive encoding of other P frames and B frames.

FIG. 2 shows the basic encoding syntax of MPEG2. As a device for performing the above-described high-efficiency encoding, a configuration as shown in a block diagram of FIG. 3 is known. In FIG. 3, an input digital image signal is stored in a frame memory 1 and is delayed in order to perform rearrangement in an encoding order according to an encoding syntax. The digital signal output from the frame memory 1 is supplied to a two-dimensional block conversion circuit 3 through a subtractor 2, where 2 pixels of N pixels in the vertical direction and M pixels in the horizontal direction (N, M are usually 8) in the reference frame. Converted to dimensional blocks. The transformed data is subjected to DCT transformation in the orthogonal transformation circuit 4 and sent to the quantization circuit 5.

The DCT quantized in the quantization circuit 5
The transform coefficient is encoded by an encoding circuit 6 into an encoding table 7.
The variable-length or fixed-length encoding is performed by referring to the address corresponding to the coefficient of, and the coded data and the information indicating the location of the block in the screen are multiplexed by the multiplexer 15 to form a bit stream. And recorded on the recording medium 17.

The code amount of the stream is compared with a target code amount in the rate control circuit 16, and the quantization level of the quantization circuit 5 is adjusted and feedback-controlled in order to approach the target code amount. Block encoding processing is performed.

Along with the encoding process, the DCT transform coefficient quantized by the quantization circuit 5 is subjected to inverse quantization and inverse DCT transform in an inverse quantization circuit 8 and an inverse orthogonal transform circuit 9 to decode an encoded bit stream. The image signal is restored to the image signal in the state as described above, and further input to the prediction memory 12 through the deblocking circuit 10 and the adder 11 and stored.

Subsequently, in the predicted frame, a motion vector between the images is obtained by the motion vector detection circuit 13 between the image output from the frame memory 1 and the image stored in the prediction memory 12. The motion vector detection circuit 13 generally obtains a two-dimensional block of an image to be coded and an image of the prediction memory 12 by block matching, and obtains a block having the smallest sum of absolute differences (or the sum of squared differences) for each pixel. Is output as a motion vector.

A prediction block is cut out from the prediction memory 12 by the motion compensation prediction circuit 14 based on the output motion vector. The motion compensation prediction circuit 14 determines whether to use prediction or not, and selects a prediction mode (in the case of a B frame, since it has two reference frames, it determines which prediction to apply and performs encoding. After subtracting the difference from the image block by the subtracter 2, the difference is sent to the two-dimensional block conversion circuit 3. In the subsequent processing, the same processing as that of each block of the frame is performed, and the DCT transform coefficient is calculated together with the motion vector and the prediction mode. Output as a bit stream.

On the other hand, using digital recording media,
When distributing various types of image information, a process of performing various special effects processing on an original image to broaden a range of video expression is generally performed. Conventionally, such special effect processing is often performed as an editing processing before high-efficiency coding, and a trial processing is performed to determine a place where the effect is to be applied, and the final created edited image is processed. Used and recorded after being subjected to a high-efficiency encoding process.

In order to perform such editing processing, a medium capable of recording unedited large-capacity digital image data in an uncompressed state is required, and a large-scale editing system for business use is required. Become.

However, it is difficult to prepare a large-scale editing system as described above for personal use such as a video letter, and for personal use, a high-efficiency coded image signal is simply edited. A system that can do this is needed.

As editing processing used for personal use, 2
An overlap process is used to connect two image signals smoothly. The overlap process mixes the images when joining the end of one image signal and the beginning of another image signal, and changes the mix level over time to realize a smooth scene transition. it can.

Therefore, as a method of performing the above-mentioned overlap editing processing, based on an encoded bit stream,
There is a method of performing overlap processing on the restored image data, performing coding processing again, and re-recording the bit stream to which the overlap effect has been applied. The configuration of a conventional editing apparatus using this technique will be described with reference to the block diagram of FIG. 3, the same components as those of FIG. 3 are denoted by the same reference numerals.

In FIG. 4, an encoded bit stream obtained from a recording medium 20 such as a hard disk is
The demultiplexer 21 divides the information into additional information such as a motion vector and screen position information, and DCT coefficient coding information, and inputs the information to a decoding circuit 22. The decoding circuit 22 decodes the variable-length or fixed-length coded DCT coefficient coding information with reference to the decoding table 23, and
Extract the conversion coefficient. The extracted coefficients are subjected to inverse quantization by an inverse quantization circuit 24, inverse DCT transform by an inverse orthogonal transform circuit 25, and 2 by a deblocking circuit 26.
After the dimensional blocks are rearranged in the scan line order of the display, if they are prediction blocks, the values extracted from the prediction memory 28 by the motion compensation prediction circuit 27 are added by the adder 29 and restored as digital image data. And stored in the prediction memory 28.

An overlap process is performed on the restored image data. Since the overlap processing requires two screen reference images, the image data is stored in a two-screen decoded image memory (frame memory) 30. The special effect processing circuit 31 generates an overlap screen by adding pixels at the same spatial position in the two screens according to the overlap parameter. The overlap parameter changes from an image 100% existing before the image to an image 100% existing after the time, and each time the special effect processing circuit 31 generates an overlap screen. The generated coded image data of the overlap screen is sequentially input to the two-dimensional block conversion circuit 3 through the subtractor 32 in accordance with the syntax of MPEG2, and the same coding circuit as shown in FIG. An encoding process is performed, and encoded data is output and recorded on the recording medium 33.

[0019]

However, in the above-described conventional editing apparatus, when the overlap processing is performed, the image quality is deteriorated by repeating the inverse orthogonal transform and the orthogonal transform. In addition, since the overlapped data is an image obtained by mixing two images, the luminance level and the resolution may be different from the prediction reference image. The probability of inducing detection is increased, and the error or code amount due to coding increases. In addition, there is a disadvantage that the circuit configuration or the amount of calculation is increased by requiring a circuit having the same scale as the encoding circuit in the editing apparatus.

The present invention has been made in view of the above points,
It is an object of the present invention to provide a high-efficiency coded image signal synthesizing apparatus and an editing apparatus capable of performing overlap processing with little image deterioration on a coded bit stream of a high-efficiency coded image with a simpler apparatus and means. Aim.

[0021]

In order to achieve the above object, a synthesizing apparatus according to the present invention converts an image signal of a reference frame into an orthogonal signal in a predetermined block unit, quantizes the image signal, and encodes the image signal. After calculating the motion component of the image signal of the other frame using to generate a prediction block,
A stream obtained by quantizing and encoding is input, and a separating unit that separates a reference frame, and an image signal that is orthogonally transformed by decoding and dequantizing the image data of the reference frame separated by the separating unit , A memory for storing the orthogonally transformed image signal from the restoration means, a setting means for externally setting the encoding syntax of the overlap period, and an encoding syntax set by the setting means. And a synthesizing means for synthesizing two orthogonally transformed image signals read from the memory.

The setting means in the synthesizing apparatus according to the present invention comprises a storage means for storing the number of overlap frames, and a frame for predicting each of the number of overlap frames from the storage means from a previously existing frame. Determining means for determining whether or not there is a frame, or determining whether the frame is a frame for which bidirectional prediction is to be performed between the preceding and succeeding frames. It is a means for combining the converted image signals at a ratio according to the frame type.

The editing apparatus according to the present invention further comprises: a quantizing / encoding means for quantizing the synthesized image signal taken out from the synthesizing means and encoding the quantized image signal, and a quantizing / encoding means. Recording means for recording the output composite image code on a recording medium.

According to the present invention, in a synthesizing apparatus and an editing apparatus for handling a highly efficient coded bit stream, a synthesized image based on the syntax of the overlap section is automatically obtained from the image signals of the preceding and succeeding decoded orthogonal transform areas. A signal can be obtained.

[0025]

Next, embodiments of the present invention will be described with reference to the drawings. FIG. 1 is a block diagram showing an embodiment of a high-efficiency image signal editing apparatus according to the present invention. The coded bit stream to be edited is supplied from a recording medium 51 such as a hard disk to a demultiplexer 52, where an I frame is detected, and additional information such as a motion vector and screen position information and coding information of DCT coefficients are converted. The separated encoded information of the DCT coefficient is supplied to the variable length decoding circuit 53.
Is input to The variable length decoding circuit 53 receives the input DC
Variable-length decoding of the encoded information of the T coefficient is performed, and the obtained decoded data is supplied to the inverse quantization circuit 54. Inverse quantization circuit 5
The image data in the DCT area that has been dequantized and extracted by 4 is stored in the orthogonal transformation area data memory 55.

Only the data of the last I frame of the first half stream to be edited is left, the first I frame of the second half coded stream is extracted, and the coding information of the DCT coefficient is separated and sent to the variable length decoding circuit 53. Then, the data is decoded and stored in the data memory 55 in the same manner. In the parameter register 56, the number of frames to be subjected to the overlap processing is set. The sequencer 57 operates a frame counter based on the total number of frames for which overlap processing is set in the parameter register 56.

The syntax setting circuit 58 determines the frame type for each frame based on the count value of the frame counter output from the sequencer 57, based on the number of frames to be subjected to the overlap processing set in the parameter register 56. Further, how to set additional information such as a motion vector value at that time, and how to set a synthesis formula of orthogonal transform coefficients are set as described later, and orthogonality is set together with the determined frame type. The data is output to the transform coefficient combining circuit 59 and the additional information encoding circuit 65.

Then, based on the determined frame type, the orthogonal transform coefficient synthesizing circuit 59 performs a synthesizing process from the DCT coefficients of the two reference image data, and supplies the result to the quantization circuit 60. The quantization circuit 60 performs a quantization process on the synthesized DCT coefficient, and causes the variable length coding circuit 61 to perform variable length coding on the obtained quantized data. The variable length coded data is supplied to the multiplexer 6
2 is supplied.

The additional information encoding circuit 65 encodes additional information such as a motion vector and screen position information supplied from the syntax setting circuit 58 and supplies the encoded additional information to the multiplexer 62. The multiplexer 62 includes a variable length encoding circuit 61
Is output as a stream obtained by combining the encoded information of the DCT coefficient supplied from, and the encoded additional information, and is recorded on the recording medium 64 via the switch 63.

The switch 63 is switched for the coded data (image data not to be synthesized) before and after the two set reference frames, and the additional information extracted from the demultiplexer 52 is not separated. The encoded stream of the image data is directly recorded on the recording medium 6.
4 and the combined stream is output from the multiplexer 62 and recorded only for the overlap section.

Hereinafter, assuming the syntax of MPEG2 as shown in FIG.
Frame interval is 15 frames, P frame interval is 3
The operation in the case of a frame will be described. The syntax setting circuit 58 sets the I frame when the remainder obtained by dividing the value of the frame counter given from the sequencer 57 by 15 is 0, and otherwise sets the I frame when the remainder obtained by dividing by 3 is 0. Set a P frame and set B
The frame is set.

Next, in the case of an I frame (when the count value of the frame counter is 15), in the case of a P frame (when the count value of the frame counter is 6), and in the case of a B frame (when the count value of the frame counter is 10). The operation of the orthogonal transform coefficient combining circuit 59 will be described.

Assuming that two pieces of reference image data (orthogonal transform areas) input from the orthogonal transform area data memory 55 are A and B, respectively, the count value of the frame counter becomes 15
Is an I frame, and in the case of an I frame,
Since the data to be encoded is data for which motion compensation prediction is not performed, the two pieces of reference image data A and B may be combined at a ratio of distance.

Accordingly, in this case, assuming that the total number of overlap frames is X and the number of frame counters is Y, the combined image data C is expressed by (1)
Based on the calculation of the equation, C = A × {(XY) / X} + B × (Y / X) = A × (15/30) + B × (15/30) = (A + B) / 2 (1 ).

In the case of a P frame, prediction is performed from an I frame or a P frame existing before, so that displacement of an image at P frame intervals becomes error data. Therefore, assuming that the motion vector is 0, it suffices to divide the difference value between the two pieces of reference image data A and B by the ratio between the number of overlap frames and the P frame interval.

Accordingly, when the count value of the frame counter is 6, since the frame is a P frame, the composite image data C from the two pieces of reference image data has a P frame interval of Z
Then, it is calculated by the orthogonal transformation coefficient synthesis circuit 59 based on the calculation of the equation (2), and C = (BA) × (Z / X) = (BA) × (3/30) = (BA) / 10 (2).

Further, in the case of a B frame, the I
Since bidirectional prediction can be performed between a frame and a P frame, the distance between the preceding and succeeding I or P frames and the B frame to be encoded and the time corresponding to the center point (that is, the time corresponding to the bidirectional prediction) The displacement corresponding to the difference between the two reference images (the center point of the two reference images) is the error data. As in the case of the P frame, the motion vector is assumed to be 0 in both directions, and the displacement amount is used as encoded data. When the count value of the frame counter is 10, the frame is a B frame, and if the P frame interval is Z and the distance between the reference image and the B frame in the forward direction is T, the orthogonal transformation coefficient synthesis circuit 5
9, the composite image data C of the two pieces of reference image data A and B is calculated as follows: C = (B−A) × {(1/2) − (T / Z)} / 30 = (BA) * {(1/2)-(1/3)} / 30 = (BA) * (1/6) / 30 = (BA) / 180 (3) .

As described above, the DCT coefficients (image data) to be coded for each successive overlap period can be derived from the DCT coefficients of two pieces of reference image data. Overlap editing can be realized without including a motion vector detection circuit, a motion compensation circuit, and an orthogonal transformation circuit for conversion and encoding.

When a quantization process is performed on the output DCT coefficient, a coding error occurs according to the quantization level. In the case of a P frame, since the prediction is of a cyclic type, errors are accumulated when encoding is performed ignoring errors. However, in the case of a P frame, the accumulation can be prevented by storing the quantization error in the orthogonal transform domain in a memory and adding the quantization error to the DCT coefficient of the error value for the next P frame.

When the interval between the P frames is set to two frames, the value of the equation (3) for deriving the error value of the B frame is as follows: C = (BA) × ｛(1/2) − (1/2)｝ / 30 =
0, no error value is generated, and no error is sent with the motion vector 0 in the bidirectional prediction mode, so that encoded data can be output. Therefore, the overlap processing can be realized with a small code amount.

[0041]

As described above, according to the present invention,
In a synthesizing device and an editing device that handle a highly efficient coded bit stream, a synthesized image signal can be automatically obtained based on the syntax of the overlap section from the decoded image signals in the preceding and following decoded orthogonal transform areas. It is possible to obtain an overlap-processed image signal that does not require orthogonal transformation means or quantization means and does not cause deterioration due to special transformation processing.

[Brief description of the drawings]

FIG. 1 is a block diagram of an embodiment of the present invention.

FIG. 2 is a diagram illustrating a basic encoding syntax of MPEG2.

FIG. 3 is a block diagram illustrating an example of a high-efficiency encoding device.

FIG. 4 is a block diagram of an example of a conventional high efficiency coded image signal editing device.

[Explanation of symbols]

 51, 64 Recording media 52 Demultiplexer 53 Variable length decoding circuit 54 Inverse quantization circuit 55 Orthogonal transformation area data memory 56 Parameter register 57 Sequencer 58 Syntax setting circuit 59 Orthogonal transformation coefficient synthesizing circuit 60 Quantization circuit 61 Variable length encoding circuit 62 multiplexer 63 selector switch 65 additional information encoding circuit

Claims (3)

[Claims] 1. An image signal of a reference frame is orthogonally transformed in a predetermined block unit, quantized and encoded, and a motion component of an image signal of another frame is obtained using the image signal of the reference frame. Is generated, a stream obtained by quantizing and encoding is input, and a separating unit that separates the reference frame; decoding and inverse quantization of the image data of the reference frame separated by the separating unit Restoring means for obtaining the orthogonally transformed image signal by converting the orthogonally transformed image signal from the restoring means, a memory for storing the orthogonally transformed image signal from the restoring means, setting means for externally setting an encoding syntax of the overlap period, The two orthogonally transformed image signals read from the memory are combined based on the encoding syntax set by the setting unit. High efficiency coding the image signal synthesizing apparatus characterized by having a synthesizing means for. 2. The method according to claim 1, wherein the setting unit is a storage unit that stores the number of overlap frames, and a frame that predicts each of the overlap frame numbers from the storage unit from a previous frame. Deciding means for deciding whether or not the frame is a frame for which bidirectional prediction is to be performed between the preceding and succeeding frames. 2. The high-efficiency coded image signal synthesizing apparatus according to claim 1, wherein said orthogonally transformed image signals are synthesized at a ratio corresponding to the frame type. 3. An image signal of a reference frame is orthogonally transformed in a predetermined block unit, quantized and encoded, and a motion component of an image signal of another frame is obtained by using the image signal of the reference frame. Is generated, a stream obtained by quantizing and encoding is input, and a separating unit that separates the reference frame; decoding and inverse quantization of the image data of the reference frame separated by the separating unit Restoring means for obtaining the orthogonally transformed image signal by converting the orthogonally transformed image signal from the restoring means, a memory for storing the orthogonally transformed image signal from the restoring means, setting means for externally setting an encoding syntax of the overlap period, The two orthogonally transformed image signals read from the memory are combined based on the encoding syntax set by the setting unit. A quantizing unit that quantizes a synthesized image signal extracted from the synthesizing unit, and then encodes the synthesized image signal. The synthesized image code output from the quantizing and encoding unit is recorded on a recording medium. And a recording unit for recording.