JP3495040B1 - Digital image encoding / decoding method and digital image encoding / decoding apparatus using the same - Google Patents

Abstract

The present invention provides a digital image data encoding device capable of realizing accurate conversion with a smaller data transmission amount when its parameter has a large number of non-integer digits such as affine conversion. An image predictive encoding apparatus includes: an image compressing unit that encodes an input image and compresses data; and performing coordinate conversion on an image obtained by decoding an image compressed by the image compressing unit and performing coordinate conversion. A coordinate conversion means for outputting coordinate data generated by the coordinate conversion means; a conversion parameter generation means for generating a conversion parameter from the coordinate data of the coordinate conversion means; and a conversion parameter generated by the conversion parameter generation means. There is provided a predicted image generating means for generating a predicted image, and a transmitting means for transmitting a compressed image and coordinate data.

Description

Detailed Description of the Invention

【Technical field】

The present invention relates to an encoding / decoding method and apparatus for accumulating or transmitting digital image data, and particularly encoding / decoding motion information in image prediction to generate a predicted image with high accuracy. A method and apparatus.

[Background technology]

To efficiently store or transmit digital images, it is necessary to perform compression coding. Conventional methods such as JPEG and M have been used as compression encoding methods for digital images.
In addition to the discrete cosine transform (DCT) represented by PEG, there are waveform coding methods such as subband, wear bullet, and fractal.

Further, in order to remove a redundant signal between images, inter-picture prediction using motion compensation is performed, and the difference signal is waveform-coded.

Here, the MP based on the motion compensation DCT
The EG method will be described. 16x1 input images
It is divided into 6 macroblocks for processing. One macroblock is further divided into 8x8 blocks, subjected to 8x8 DCT, and then quantized. This is called intraframe coding.

On the other hand, with a motion detection method such as block matching, a predicted macroblock having the smallest error in a target macroblock is detected from another frame adjacent in time, and a past macroblock is detected based on the detected motion. Motion compensation is performed from the image of to obtain the optimum prediction block.
The signal indicating the prediction macroblock with the smallest error is the motion vector.

Next, the difference between the target block and the corresponding prediction block is obtained, DCT is performed, the transform coefficient is quantized, and transmitted or stored together with the motion information. This is called interframe coding.

On the receiving side, after the quantized transform coefficient is restored to the original difference signal, a prediction block is obtained based on the motion vector, added to the difference signal, and the image is reproduced.

Although the prediction image is generated in block units, the entire image may move like pan and zoom. In this case, the entire image is motion-compensated. Motion compensation or generation of a predicted image may be accompanied by deformation such as simple translation to enlargement / reduction / rotation.

(Formula 1) to (Formula 4) show equations representing movement / deformation. (X, y) is the pixel coordinate, (u, v)
Is the transformed coordinates, which is the motion vector at (x, y). Other variables are conversion parameters indicating movement and deformation.

(Equation 1) (u, v) = (x + e, y + f) (Equation 2) (u, v) = (ax + e, dy + f) (Equation 3) (u, v) = (ax + by + e, cx + dy + f) (Equation) 4) (u, v) = (gx2 + pxy + ry2 + ax + by +
e, hx2 + qxy + sy2cx + dy + f)

The transformation of (Equation 3) is called an affine transformation, which will be described below as an example. The affine transformation parameters (a, b, c, d, e, f) can be obtained as follows.

First, the image is divided into a plurality of blocks (2 × 2,
(4 × 4, 8 × 8, etc.), and the motion vector of each block is obtained by the block matching method. An affine parameter is obtained by selecting at least three highly reliable motion vectors from the obtained motion vectors and solving the six simultaneous equations of Equation 3. Generally, in order to reduce the error, more points are selected and the affine parameter is obtained by the least square method.

The affine parameters obtained in this way are used for predictive image generation. It is necessary to transmit the affine parameters to the receiving side so that the same predicted image can be generated.

DISCLOSURE OF THE INVENTION [Problems to be Solved by the Invention]

However, the conventional interframe coding is based on the assumption that the target image and the reference image have the same size, and is not sufficiently compatible with images of different sizes. The change in size of two adjacent images is often due to the movement of an object in the images.

For example, when a person standing with both hands down (FIG. 7A) raises both hands, the size of the rectangle surrounding the person changes (FIG. 7B). In consideration of coding efficiency, it is necessary to convert the target image and the reference image into the same coordinate space so that the code amount of the motion vector becomes small.

In addition, the arrangement of macroblocks in the divided image changes due to the change in the size of the image. For example, when changing from FIG. 7 (A) to FIG. 7 (B), the macroblock 701 is divided into two macroblocks 703 and 704 and compression-coded, so in the reproduced image of FIG. 7B. , Vertical distortion due to quantization appears on a person's face, and the visual quality is degraded.

Further, since it is necessary to perform the affine transformation with high accuracy, the affine parameters (a, b,
(c, d, e, f, etc.) are generally real numbers below the decimal point, and therefore, in order to transmit with high precision, it is necessary to transmit with a long number of bits.

Conventionally, since the affine parameter is simply quantized and transmitted with a fixed-length or variable-length code, the precision of the affine parameter is reduced, high-precision affine transformation cannot be obtained, and a desired predicted image cannot be generated.

As can be seen from (Equation 1) to (Equation 4), there are 2 to 10 or more conversion parameters. To transfer the conversion parameters,
If the coding is performed according to the maximum number, there is a problem that a redundant code will be sent when transmitting a small number of parameters.

[Means for Solving the Problems]

A first object of the present invention is to perform digital image data encoding / decoding which can realize accurate conversion with a smaller data transmission amount when the parameter has a large number of digits which are not integers like affine conversion. It is to provide an oxidization device.

To achieve the object, an image compression unit that encodes an input image and compresses the data and an image obtained by decoding the image compressed by the image compression unit are subjected to coordinate conversion and coordinate conversion. From the input image using the coordinate conversion means for outputting the coordinate data generated by the conversion parameter, the conversion parameter generation means for generating the conversion parameter from the coordinate data of the coordinate conversion means, and the conversion parameter generated by the conversion parameter generation means. An image predictive coding apparatus is configured by including a predicted image generating unit that generates a predicted image and a transmitting unit that transmits the compressed image and coordinate data.

Also, variable length decoding means for inputting compressed image data and coordinate data and performing variable length decoding, conversion parameter generation means for generating conversion parameters from the coordinate data decoded by the variable length decoding means, and Prediction image generation means for generating prediction image data using the conversion parameter generated by the conversion parameter generation means, and decoding by adding the prediction image from the prediction image generation means and the variable length decoded compressed image data It is a digital image decoding device having an addition means for generating an image.

Particularly, the conversion parameter generating means converts the conversion parameter from the coordinate points of N (N is a natural number) pixels and the N converted coordinate points obtained by converting the N coordinate points by a predetermined linear polynomial. Digital encoding, which produces
It is a decoding device. Further, the conversion parameter generating means is
The first to Nth target images of different sizes are input, common space coordinates are set for the first to Nth target images, and the first to Nth target images are set by a predetermined method. When compression encoding is performed to generate first to Nth compressed images, the first to Nth compressed images are decoded, converted into the common space coordinates, and first to Nth decompressed images are generated and stored. At the same time, the digital image encoding / encoding device outputs conversion parameters generated by converting the first to Nth expanded images into the common space coordinates.

A second object of the present invention is to perform motion by transmitting coordinate data obtained by converting a target image and a reference image into the same coordinate space, particularly when predictively encoding images of different sizes as coordinate data. The digital image encoding / decoding device improves the detection accuracy and at the same time reduces the code amount of the motion vector to improve the image quality.

In order to achieve this object, the image predictive coding apparatus of the present invention inputs first to Nth target images of different sizes, and performs common space for the first to Nth target images. The coordinates are set, the first target image is compression-encoded by a predetermined method to generate the first compressed image, and then the first compressed image is decoded and converted into common space coordinates, and the first decompressed image. Is generated and stored, and at the same time, a first offset signal (coordinate data) generated by converting the first target image into common space coordinates is encoded and transmitted together with the first compressed image. = 2,3, ... N) target images are converted into common space coordinates, a predicted image is generated by referring to the (n-1) th decompressed image, and the nth target image and the predicted image are generated. Generates a difference image from and compression-encodes it to generate the nth compressed image After that, the nth compressed image is decoded, converted into common space coordinates, and the nth decompressed image is generated and stored, and at the same time, the nth target image is converted into common space coordinates. n offset signal
(Coordinate data) is encoded and transmitted together with the nth compressed image.

Further, the image predictive decoding device of the present invention is
An input terminal, a data analyzer, a decoder, an adder,
A coordinate converter, a motion compensator, and a frame memory are provided, and the first to Nth target images having different sizes are encoded at the input terminal, and the nth (n = 1, 2, 3 ,. ., N), the first to Nth compressed image data including the nth offset signal generated by converting the target image of the target image into the common space coordinates are input, and the first compression is performed by the data analyzer. After analyzing the image data, outputting the first compressed image signal and the first offset signal, inputting the first compressed image signal to the decoder, and restoring the first reproduced image, the coordinate converter Then, the first reproduction image is subjected to coordinate conversion based on the first offset signal and stored in the frame memory, and the n-th (n = 2,
3 ,. ． ． , N) of the compressed image data, the data analyzer analyzes the nth compressed image data, outputs the nth compressed image signal, the nth offset signal, and the nth motion signal for decoding. Input the nth compressed image signal to the
Restored to the decompression differential image of No. 1, input the nth offset signal and the nth motion signal to the motion compensator, and store them in the frame memory based on the nth offset signal and the nth motion signal. When the n-th predicted image is acquired from the (n-1) -th reproduced image and the adder adds the n-th expanded difference image and the n-th predicted image to restore and output the n-th reproduced image At the same time, in the coordinate converter, based on the nth offset signal,
The coordinate of the nth reproduced image is converted and stored in the frame memory.

A third object of the present invention is to transmit a conversion parameter including an affine parameter in the case of performing an affine transformation to data to be transmitted as coordinate data with high accuracy, and to enable a highly accurate prediction image generation to obtain a digital image. Coding,
It is to provide a decoding device.

According to the present invention, a variable length decoding unit,
A digital image decoding device including a differential image decompression unit, an addition unit, a conversion parameter generation unit, a predicted image generation unit, and a frame memory, in which data is input to the variable length decoding unit and the differential image is converted from the data. At the same time as separating the data and transmitting it to the difference image decompression unit, the coordinate data is separated and input to the conversion parameter generation unit, the difference image decompression unit decompresses the difference image data and transmits it to the addition unit, and the conversion parameter generation unit. At, a conversion parameter is generated from the coordinate data and transmitted to the prediction image generation unit, at the prediction image generation unit, a prediction image is generated from the conversion parameter and the image input from the frame memory, and transmitted to the addition unit, The adding unit adds the predicted image to the decompressed difference image to generate and output the image, and at the same time stores it in the frame memory.

The coordinate data is the coordinate points of N pixels and N transformed coordinate points obtained by transforming the N coordinate points by a predetermined linear polynomial, or the coordinates of the N pixels. A difference value between a point and N converted coordinate points obtained by converting N coordinate points by a predetermined linear polynomial, or N obtained by converting predetermined N coordinate points by a predetermined linear polynomial. Transformed coordinate points, or N obtained by transforming predetermined N coordinate points by a predetermined linear polynomial.
It is a difference value between the converted coordinate points and the predicted value. The predicted value is a predetermined N coordinate points or N converted coordinate points of the previous frame.

Further, according to the present invention, the conversion parameter estimation unit, the predicted image generation unit, the first addition unit, the difference image compression unit, the difference image decompression unit, the second addition unit, the frame memory, A digital image encoding device including a transmission unit inputs a digital image, a conversion parameter estimation unit estimates a conversion parameter from the image stored in the frame memory and the digital image, and a predicted image generation unit The estimated conversion parameter and the image stored in the frame memory are input, a predicted image is generated based on the estimated conversion parameter, and the first addition unit calculates the difference between the digital image and the predicted image. , The differential image compression unit compresses the compressed differential data and sends it to the transmission unit. At the same time, the differential image decompression unit decompresses the compressed differential data into decompressed differential data, and the second addition unit performs pre-processing. Image and adding, in the digital image encoding device to be stored in the frame memory, the transformation parameter estimator, sends the coordinate data to the transmission unit, transmits with compressed difference data.

The above coordinate data is the coordinate points of N pixels and N transformed coordinate points obtained by transforming the N coordinate points according to the transformation parameter, or the coordinate points of the N pixels. , The difference between the N transformed coordinate points and the coordinate points of the N pixels, or the N transformed coordinate points obtained by transforming the coordinate points of the predetermined N pixels by the transformation parameter, It is the difference between the N transformed coordinate points and the coordinate points of predetermined N pixels, or the difference between the N transformed coordinate points and the N transformed coordinate points of the past frame.

Further, according to the present invention, a digital image decoding device including a variable length decoding unit, a difference image decompression unit, an addition unit, a conversion parameter generation unit, a prediction image generation unit and a frame memory is provided. , The data is input to the variable length decoding unit, the differential image data is separated from the data and transmitted to the differential image decompression unit, and at the same time, the number of coordinate data and the coordinate data are input to the conversion parameter generation unit, and the difference is calculated. The image decompression unit decompresses the difference image data and transmits it to the addition unit, and the conversion parameter generation unit switches the conversion parameter generation method based on the number of conversion parameters, generates conversion parameters from coordinate data, and predicts. The predicted image is transmitted to the image generation unit, the predicted image generation unit generates a predicted image from the conversion parameter and the image input from the frame memory, and transmits the predicted image to the addition unit. At, the predicted image is added to the decompressed differential image to generate an image, which is output and simultaneously stored in the frame memory.

The coordinate data is the coordinate points of N pixels and the N transformed points obtained by converting the N coordinate points by a predetermined linear polynomial, or the coordinate data is N pixels. Is the difference between N coordinate points converted by a predetermined linear polynomial and N coordinate points converted by a predetermined linear polynomial, or the coordinate points of N pixels, or the coordinate data of N pixel Differences between the coordinate points and the coordinate points of N images in the past frame, N transformed coordinate points obtained by transforming the N coordinate points by a predetermined linear polynomial, and N in the past frame.
Difference from the converted coordinate points, or a predetermined N
N coordinate points which have been converted by a predetermined linear polynomial, or N predetermined coordinate points which have been converted by a predetermined linear polynomial. Difference from N coordinate points of, or
It is the difference between the N converted coordinate points obtained by converting the predetermined N coordinate points by the predetermined linear polynomial and the N converted coordinate points of the past frame.

Further, when the conversion parameter is transmitted as it is, the conversion parameter is scaled by the image size and then quantized and encoded, or the exponent part of the maximum value of the conversion parameter is obtained, and the conversion parameter is normalized by the exponent part. , The exponent part and the normalized conversion parameter are transmitted.

【The invention's effect】

As described above, according to the present invention, by converting images of different sizes into the same coordinate system and then performing motion detection to generate a predicted image, the accuracy of motion detection is improved and at the same time, the code amount of the motion vector is increased. The effect of reducing is obtained.

Further, by obtaining the conversion parameter from the coordinate data on the decoding side, a highly accurate conversion parameter can be obtained and a highly accurate predicted image can be generated. Further, by normalizing the conversion parameter or multiplying it by the image size, the parameter can be transmitted with accuracy according to the image.

Further, by switching the generation of the conversion parameter depending on the number of coordinate data, the conversion parameter generation can be optimally processed and the coordinate data can be efficiently transmitted.

BEST MODE FOR CARRYING OUT THE INVENTION

Embodiments of the present invention will be described below with reference to FIGS. 1 to 12. (Embodiment 1) FIG. 1 is a block diagram of an image predictive coding apparatus according to Embodiment 1 of the present invention. In FIG.
2 is a first adder, 103 is an encoder, 106 is an output terminal, 107 is a decoder, 110 is a second adder, 111 is a first coordinate converter, 112 is a second coordinate converter, and 113 is motion detection. , 114 is a motion compensator, and 115 is a frame memory.

The operation of the image predictive coding apparatus configured as described above will be described below. Input terminal 10
1, the first to Nth target images having different sizes are input. N is determined by the length of the image.

First, the first target image is input to the input terminal 101, passes through the first adder 102, and then the encoder 103.
Compress and encode at. In this case, the subtraction in the first adder 102 is not performed. In addition, in the present embodiment, the target image is divided into a plurality of adjacent blocks (8 × 8 pixels), and the discrete cosine transformer DCT (104) transforms the spatial domain signal into a frequency domain signal to generate a transformed block. .

The transform block is quantized by a quantizer Q (105) to generate a first compressed image, which is output to an output terminal 106 and converted into a fixed-length or variable-length code for transmission (illustrated in the figure). Not). At the same time, the decoder 107 restores the first compressed image to a decompressed image. In the present embodiment, the inverse quantizer IQ (108) dequantizes and the inverse discrete cosine transformer (IDCT) 109 transforms the signal into a spatial signal.

The reproduced image thus obtained is subjected to coordinate conversion by the first coordinate converter 111, and the first reproduced image is stored in the frame memory 115.

Next, the operation of the first coordinate converter 111 will be described. Let FIG. 2 (A) be a first target image. Image 20
The pixel a1 of 1 has coordinates (0, 0) in the coordinate system 203. In FIG. 2C, a new coordinate system 205 is set. This coordinate system can be the coordinate system of the display screen,
Alternatively, a coordinate system whose origin is the center point of the target image may be used.

In any case, the coordinate system 205 is preset before the encoding is started. A mapping of the target image 201 to the coordinate system 205 is shown in FIG.
It is shown in (C).

By this coordinate conversion, the coordinates of the pixel a1 of the target image 201 become (x_a, y_a). Note that coordinate conversion including rotation may be performed. The values of x_a and y_a are encoded with a fixed length of 8 bits and transmitted together with the first compressed image.

Next, the nth (n = 2, 3, ..., N)
The target image of is input to the input terminal 101. The nth target image is input to the second coordinate converter 112 via the line 126 and converted into the coordinate system 205. Image 20 of FIG. 2 (B)
Let 2 be the nth target image.

It is mapped to the coordinate system 205 and the coordinates of the pixel b1 are converted into (x_b, y_b) (FIG. 2).
(C)). The coordinate-transformed target image 202 is input to the motion detector 113, divided into a plurality of blocks, the (n-1) th reproduced image stored in the frame memory 115 is referred to, and motion is performed by a method such as block matching. Detect and generate a motion vector.

The generated motion vector is set to line 128.
The (n-1) th reproduced image stored in the frame memory 115 and accessed to the motion compensator 114 at the same time is output and encoded and transmitted (not shown). Examples of motion detection / compensation are disclosed, for example, in USP 5,193,004.

The block of the n-th target image and its prediction block are input to the first adder 102 to generate a difference block. The difference block is compressed by the encoder 103,
The nth compressed image is generated and output to the output terminal 106, and at the same time, the decoder 107 restores the expanded differential block.

The second adder 110 adds the prediction block sent via the line 125 to the decompression difference block to reproduce the image. The image reproduced in this way is input to the first coordinate converter 111, and the image 2 of FIG.
The coordinates are converted in the same manner as 02 and stored in the frame memory 115 as the nth reproduced image, and at the same time, the coordinates (x_b, y_b) of the pixel b1 are encoded and transmitted together with the nth compressed image.

FIG. 3 is a schematic diagram showing a coded image data series by the image predictive coding apparatus according to the embodiment of the present invention.
An image synchronization signal 303 is added to the head of the encoded image data, and then parameters x_a (304) and y_a by coordinate conversion are used.
(305), image sizes 306 and 307, and a step value 308 used for quantization.

After that, the motion vector and the compressed data of the image follow. That is, the parameter x_a is used as coordinate data.
(304), y_a (305), image size 306,
307 will be transmitted.

FIG. 4 shows another form of image coordinate conversion in the embodiment of the present invention. In this case, the target image is divided into a plurality of areas, and coordinate conversion is performed for each area.

For example, the image 201 is divided into three regions R
1, R2, R3 are divided, each area is compressed and expanded, and then R reproduced by the first coordinate converter 111.
The coordinates of the areas R1, R2, and R3 are converted and stored in the frame memory 115. Parameters (x_a1, y_a1), (x_a2, y_a) used for coordinate conversion
2) and (x_a3, y_a3) are simultaneously encoded and transmitted.

Next, the image 202 is input, and the area R4
It is divided into R5 and R6, and the respective regions are subjected to coordinate conversion by the second coordinate converter 112. For each coordinate-converted area, the area stored in the frame memory 115 is referred to for motion detection / compensation to generate a prediction signal, and the first adder 102 generates a difference signal for compression / expansion. Then, the second adder adds the prediction signals.

The respective regions thus reproduced are subjected to coordinate conversion and stored in the frame memory 115. Parameters (x_b1, y_b1) used for coordinate conversion,
(X_b2, y_b2) and (x_b3, y_b3) are simultaneously encoded and transmitted.

By converting images of different sizes into common space coordinates, it is possible to improve the accuracy of motion detection and at the same time reduce the code amount of the motion vector to improve the image quality.

The images of FIGS. 6 (A) and 6 (B) are represented by dot 60.
By matching the coordinates to 5, the block 601 and the block 603 are coincident with each other, and the block 602 and the block 604 are coincident with each other, so that the motion detection can be accurately obtained.

Further, in this example, since the motion vectors of the block 603 and the block 604 are close to zero, the code amount of the motion vector can be reduced. The same can be said for a general image and two adjacent images. Further, unlike FIG. 7B, since the face of the block 603 in FIG. 6B fits into one block, vertical distortion due to quantization does not appear on the face.

(Second Embodiment) FIG. 5 is a block diagram of an image predictive decoding apparatus according to a second embodiment of the present invention. In FIG. 5, 501 is an input terminal and 50 is an input terminal.
2 is a data analyzer, 503 is a decoder, 506 is an adder, 507 is an output terminal, 508 is a coordinate converter, 509 is a motion detector, and 510 is a frame memory.

The operation of the image predictive coding apparatus configured as described above will be described below. Input terminal 50
1 is generated by encoding first to Nth target images of different sizes and converting the nth (n = 1, 2, 3, ..., N) target image into common space coordinates. The first to Nth compressed image data including the nth conversion parameter are input. The schematic diagram of FIG. 3 shows an example of compressed image data. The compressed image data input by the data analyzer 502 is analyzed.

First, the first compressed image data is analyzed by the data analyzer 502, and the first compressed image is decoded by the decoder 503.
Output to. In addition, the first conversion parameter (x_ in FIG. 2C) generated by converting to the common space coordinates.
a, y_a) via line 520 to coordinate converter 50
Send to 8.

The decoder 503 decompresses the first compressed image into a decompressed image and outputs it to the output terminal 507. At the same time, the expanded image is input to the coordinate converter 508. In this embodiment, inverse quantization is performed and IDCT is applied to restore a signal in the spatial domain.

The coordinate converter 508 maps the decompressed image in the common space coordinate system based on the first conversion parameter and outputs it as the first reproduced image.
Store in 0. The coordinate conversion is the same as that described in the first embodiment.

Next, the nth (n = 2, 3, ..., N) compressed image data is analyzed by the data analyzer 502,
The compressed difference image of is output to the decoder 503. In addition, the nth motion data is transmitted via the line 521 to the motion compensator 50.
9, and the n-th conversion parameter (x_b, y_ in FIG. 2C) generated by converting into the common space coordinates.
(corresponding to b) via line 520 to coordinate converter 508
And to the motion compensator 509.

The decoder 503 restores the nth compressed difference image to the nth expanded difference image and outputs it to the adder 506. In the present embodiment, the differential signal of the target block is inversely quantized, IDCT is performed, and output as an expanded differential block.

On the other hand, the motion compensator 509 acquires a prediction block from the frame memory 510 using the nth transformation parameter and the motion vector of the block.

In the present embodiment, the coordinates of the target block are converted using the conversion parameters, that is, the coordinates of the target block are converted into the nth conversion parameter (for example, x in FIG. 2C).
_B, y_b) is added and the motion vector is added to the sum to determine the address of the frame memory 510. The prediction block thus obtained is added to the adder 50.
6 and add it to the expansion difference block to reproduce the image. The reproduced image is output to the output terminal 507, and at the same time, the coordinate converter 508 performs coordinate conversion using the nth conversion parameter and stores it in the frame memory 510.

In place of the coordinate converter 508, the difference (x_b−) between the conversion parameters of the n-th image and the (n-1) -th image at the coordinates of the target block in the motion compensator 509 or before and after the motion compensator 509. It is needless to say that the same effect can be obtained by determining the address of the frame memory 510 by adding another device having a function of adding x_a, y_b−y_a) and adding a motion vector to it.

Next, consider the case where another compressed image data is input to the input terminal 501. That is, the first to Nth target images having different sizes are divided into a plurality of target regions, coded, and each target region is converted into common space coordinates. Input the Nth compressed image data.

First, the first compressed image data is analyzed by the data analyzer 502, and the m-th (m = 1, 2, ...,) is analyzed.
The M) compressed area is output to the decoder 503. Figure 4
In (A), M = 3. In addition, the m-th conversion parameter (FIG. 4 (A)) generated by converting into the common space coordinates.
X_am, y_am, m = 1, 2, 3) of line 52
It is sent to the coordinate converter 508 via 0. Decoder 503
Then, the m-th compression area is restored to the m-th expansion area and output to the output terminal 507. At the same time, the m-th extension area is input to the coordinate converter 508.

Therefore, based on the m-th conversion parameter,
Mapping the m-th extension area to the common space coordinate system,
Is output as a reproduction area of and is stored in the frame memory 510. The method is the same as described above.

Next, the nth (n = 2, 3, ..., N) compressed image data is analyzed by the data analyzer 502, and the kth (k = 1, 2, ... , K) and outputs the compressed difference area to the decoder 503. In FIG. 4B, K = 3.

In addition, the corresponding motion data is set to the line 52.
1 is sent to the motion detector 509 and is converted into the common space coordinates to generate the k-th conversion parameter (x_bk, y_bk, k = 1, 2, 3 in FIG. 4B).
Is sent to the coordinate converter 508 and the motion compensator 509 via the line 520.

The decoder 503 restores the k-th compression difference area to the k-th expansion difference area and outputs it to the adder 506. In the present embodiment, the differential signal of the target block is inversely quantized, IDCT is performed, and output as an expanded differential block.

On the other hand, the motion compensator 509 uses the k-th conversion parameter and the motion vector of the corresponding block,
A prediction block is acquired from the frame memory 510. In this embodiment, the coordinates of the target block are converted using the kth conversion parameter, that is, the coordinates of the target block are converted to the kth coordinates.
Conversion parameters (for example, x_bk, y_ in FIG. 4B)
bk, k = 1, 2, 3) and the motion vector is added to the sum to determine the address of the frame memory 510.

The prediction block thus obtained is sent to the adder 506 to be added to the decompression difference block to reproduce the image. The reproduced image is output to the output terminal 507, and at the same time, the coordinates are converted by the coordinate converter 508 and stored in the frame memory 510.

(Third Embodiment) FIG. 8 shows a block diagram of a decoding apparatus according to a third embodiment of the present invention. The input terminal 801, the variable length decoding unit 802, the difference image expansion unit 803, the addition unit 804, the output terminal 805, the conversion parameter generation unit 806, the frame memory 807, and the predicted image generation unit 808 are included.

The compression-coded image data is input to the input terminal 801. The variable length decoding 802 analyzes the input data, separates the difference image data and the coordinate data, and sends them to the difference image decompression unit 803 and the conversion parameter generation unit 806 via lines 8002 and 8003, respectively.

The differential image data includes a DCT-quantized transform coefficient and a quantization width. The difference image decompression unit 803 dequantizes the transform coefficient using the quantization width and then performs inverse DCT to decompress the difference image.

On the other hand, the coordinate data includes data for generating the conversion parameter, and the conversion parameter generating unit 806 generates the conversion parameter. For example,
In the case of the affine transformation shown in (Equation 3), (a, b, c, d,
e, f) are generated. Details will be described below.

The conversion parameters generated by the conversion parameter generation unit 806 and the image stored in the frame memory are input to the predicted image generation unit 808. In the case of the affine transformation shown in (Equation 3), using (a, b, c, d, e, f) sent from the transformation parameter generation unit 806, it is in (x, y) according to (Equation 3). The predicted value of the pixel is the pixel at (u, v) of the image stored in the frame memory. The same applies to the cases of (Equation 1), (Equation 2), and (Equation 4).

The prediction image generated in this way is sent to the addition unit 804, added to the difference image, and the image is reproduced. The reproduced image is output to the output terminal 805 and simultaneously stored in the frame memory 807.

The coordinate data described above can take a plurality of forms, which will be described below. Consider a case where the coordinate data is composed of coordinate points of N pixels and N converted coordinate points obtained by converting the N coordinate points by a predetermined linear polynomial.

Here, N is the number of points required to obtain the conversion parameter. In the case of affine parameters, there are 6 parameters, so 6 equations are needed to solve 6 variables. (X, y) for one coordinate point
Since there is a component of, if N = 3, six affine transformation parameters can be solved.

In the case of (Equation 1), N = 1, in the case of (Equation 2) N = 2, and in the case of (Equation 4) N = 5. The N converted coordinate points are motion vectors, and are represented by (Equation 1) to (Equation 4)
Corresponds to (u, v) on the left side of.

In the case of affine transformation, three coordinate points (x
0, y0), (x1, y1), (x2, y2) and transformed coordinate points (u0, v0), (u1, v1), (u2, v)
2) passes through line 8003 and the conversion parameter generation unit 8
Enter in 06. The conversion parameter generation unit 806 obtains the affine parameters by solving the simultaneous equations below.

(Equation 5) (U0, v0) = (ax0 + by0 + e, cx0 + dy0 + f) (U1, v1) = (ax1 + by1 + e, cx1 + dy1 + f) (U2, v2) = (ax2 + by2 + e, cx2 + dy2 + f)

Note that it is also possible to obtain the conversion parameter by using more coordinate data. In other cases, the conversion parameters can be solved similarly. By selecting N (x, y) well, it is possible to obtain a highly accurate conversion parameter. N (x, y) Ns arranged at right angles are preferred.

It should be noted that the transformed coordinate points (u0, v0),
When calculating corresponding coordinate points (x0, y0), (x1, y1), (x2, y2) for (u1, v1), (u2, v2), (Equation 6) instead of (Equation 5) The simultaneous equation of may be obtained.

(Equation 6) (X0, y0) = (Au0 + Bv0 + E, Cu0 + Dv0 + F) (X1, y1) = (Au1 + Bv1 + E, Cu1 + Dv1 + F) (X2, y2) = (Au2 + Bv2 + E, Cu2 + Dv2 + F)

Next, consider a case where the coordinate data is the difference value between the coordinate points of N pixels and the N transformed coordinate points obtained by transforming the N coordinate points by a predetermined linear polynomial. When the predicted value for obtaining the difference is the coordinate points of N pixels, the conversion parameter generation unit 806 adds the coordinate points of the N pixels and the difference values of the N converted coordinate points, and N A conversion parameter is generated from the coordinate points of the pixels and the N converted coordinate points added.

When the predicted value for obtaining the difference is the converted coordinate points of N pixels of the previous frame, the conversion parameter generation unit 806 outputs N converted coordinate points of the previous frame and N converted coordinate points. Add the difference value of the transformed coordinate points,
A conversion parameter is generated from the coordinate points of N pixels and the N converted coordinate points that have been added. The N converted coordinate points that have been added are stored as predicted values for the next frame.

Next, consider a case where the coordinate data is N converted coordinate points obtained by converting predetermined N coordinate points by a predetermined linear polynomial. The predetermined N coordinate points are predetermined coordinate points and need not be transmitted. The conversion parameter generation unit 806 generates a conversion parameter from the predetermined N coordinate points of pixels and the N converted coordinate points.

Next, consider a case where the coordinate points are difference values of N converted coordinate points obtained by converting the predetermined N coordinate points by a predetermined linear polynomial. When the predicted value for obtaining the difference is the coordinate points of N pixels, the conversion parameter generation unit 806 adds the coordinate points of the N pixels and the difference values of the N converted coordinate points, and N A conversion parameter is generated from the coordinate points of the pixels and the N converted coordinate points added.

When the predicted value for obtaining the difference is the converted coordinate points of N pixels of the previous frame, the conversion parameter generation unit 806 outputs N converted coordinate points of the previous frame and N converted coordinate points. Add the difference value of the transformed coordinate points,
A conversion parameter is generated from the coordinate points of N pixels and the N converted coordinate points that have been added. The N converted coordinate points that have been added are stored as predicted values for the next frame.

FIG. 9 is a block diagram of an encoding device according to the third embodiment of the present invention. The input terminal 901, the conversion parameter estimation unit 903, the predicted image generation unit 908, the first addition unit 904, the difference image compression unit 905, and the difference image decompression unit 9
10, the second addition unit 911, and the frame memory 909
And a transmission unit 906. A digital image is input to the input terminal 901. Conversion parameter estimation unit 903
At, the conversion parameters are estimated from the image stored in the frame memory and the digital image. The method for estimating the affine parameters is as described above.

The original image may be used instead of the image stored in the frame memory. The conversion parameter estimated by the conversion parameter estimation unit 903 is sent to the predicted image generation unit 908 via the line 9002.

Also, the coordinate data converted using the conversion parameter is sent to the transmission unit 906 via the line 9009. The coordinate data can have a plurality of forms, as described above.

The predicted image generation unit 908 inputs the estimated conversion parameter and the image stored in the frame memory 909, and generates a predicted image as described above based on the estimated conversion parameter. Next, the first addition unit 9
At 04, the difference between the digital image and the predicted image is obtained, and the difference image compression unit 905 compresses the compressed difference data and sends it to the transmission unit 906.

In the differential image compression 905, the differential image is D
CT and quantize. At the same time, the differential image expansion unit 910 expands the compressed differential data into expanded differential data. In the difference image expansion unit 910, inverse quantization and inverse DCT are performed. The second addition unit adds the decompression difference data to the predicted image and stores it in the frame memory. The transmission unit 906 encodes the compression difference data, the quantization width, and the coordinate data, multiplexes them, and then transmits / stores them.

(Fourth Embodiment) FIG. 10 shows a digital image decoding apparatus according to a fourth embodiment of the present invention. Input terminal 1001 and variable length decoding unit 100
2, a difference image decompression unit 1003, an addition unit 1004,
The conversion parameter generation unit 1006 and the predicted image generation unit 10
08 and a frame memory 1007. The basic operation is the same as in FIG. Only different points will be described.

The conversion parameter generation unit 1006 is configured to generate a plurality of types of conversion parameters.
The parameter generation unit 1006a has the parameters (a, e, d, f) shown in (Equation 2), and the parameter generation unit 1006b has the parameters (a, b, e, c, d, shown in (Equation 3).
f), the parameter generation unit 1006c causes the parameters (g, p, r, a, b, e, h, q, s, c,
d, f).

(Equation 2) is two coordinate points, and (Equation 3) is 6
A parameter can be generated if there are 12 coordinate points, one coordinate point (Equation 4). The number of these coordinate points is line 1001
The switches 1009 and 1010 are controlled via 0. When the number of coordinate points is 2, switches 1009 and 101
0 to terminals 1011a and 1012a respectively,
The coordinate data is sent to the parameter generation unit 1006a via the line 10003 and the simultaneous equations are solved to generate the parameters of (Equation 2) and output from the terminal 1012a.

When the number of coordinate points is 3 and 6, the coordinate points are connected to the parameter generation units 1006b and 1006c, respectively.
In this way, the type of coordinate data to be transmitted can be known from the information on the number of coordinate points, and it is possible to switch and generate conversion parameters.

The form of the coordinate data passing through the line 10003 is as described above. If (x, y) on the right-hand side of (Equation 2) to (Equation 4) is known, it is not necessary to transmit, so the number of coordinate points passing through the line 10010 is 1 with respect to (Equation 2), It is also possible to correspond to 3 for (Equation 3) and 6 for (Equation 4). Furthermore, the number of conversion parameter generation units is not limited to three, and there may be more.

(Fifth Embodiment) FIGS. 11 and 12 are block diagrams of a digital image decoding apparatus and an encoding apparatus according to a fifth embodiment of the present invention. It is basically the same as FIG. 8 and FIG. The difference is that the conversion parameter generating unit 806 is replaced by a conversion parameter decompressing unit 1106, and the operations of the conversion parameter estimating units 903 and 1203 are slightly different. This will be described.

The conversion parameter estimation unit 1203 of FIG.
Then, after estimating the conversion parameter, it is multiplied by the image size, quantized, and then sent to the transmission unit 1206 via the line 12009. The conversion parameters are real numbers,
It is necessary to make a constant and then convert it to an integer.

In the case of affine parameters, (a, b,
c, d) must be expressed with high precision. a and c are vertical coordinate parameters, which are multiplied by the vertical pixel number V of the image,
Further, b and d are parameters of horizontal coordinates, which are multiplied by the number H of horizontal pixels of the image.

When there is a square term as in (Equation 4), the image size to be scaled is the same square (H2, V2, H).
V) is also possible. The conversion parameter decompression unit 1106 in FIG. 11 divides the scaled parameter and reproduces the parameter.

Further, the conversion parameter estimation unit 1 of FIG.
In 203, after estimating the conversion parameter, the maximum value of the conversion parameter is obtained. The maximum absolute value is preferable. The conversion parameter is normalized by the exponent part (preferably exponent part of power of 2) of the maximum value. That is, each conversion parameter is multiplied by the value of the exponent.

The conversion parameter and exponent part thus normalized are sent to the transmission part 1206, converted into a fixed length code and transmitted. The conversion parameter expansion unit 1106 in FIG. 11 divides the normalized conversion parameter by the exponent part and expands it into the conversion parameter.

Affine parameters (a, b, c, d)
In the case of, the maximum value is obtained from (a, b, c, d).
The translation parameters (e, f) may be included, but they are not included because the size of the ordinary value has a different number of digits. The same applies to the parameters of Equation 4, and it is preferable to normalize by dividing the parameters of the squared term and the squared term, but not limited to this.

In all the above-described embodiments, the case where the difference image is non-zero has been described, but the same applies when the difference image is completely zero. In this case, the predicted image is output as it is. Further, although the conversion of the entire image has been described, the same can be applied to a case where a two-dimensional or three-dimensional image is divided into a plurality of small areas and each small area is subjected to conversion such as affine transformation.

[Industrial availability]

As described above, according to the present invention, by converting images of different sizes into the same coordinate system and then performing motion detection to generate a predicted image, the accuracy of motion detection is improved, and at the same time, the code amount of the motion vector is increased. The effect of reducing is obtained.

Further, by obtaining the conversion parameter from the coordinate data on the decoding side, a highly accurate conversion parameter can be obtained and a highly accurate predicted image can be generated. Further, by normalizing the conversion parameter or multiplying it by the image size, the parameter can be transmitted with accuracy according to the image.

Further, by switching the generation of the conversion parameter according to the number of coordinate data, the conversion parameter generation can be optimally processed and the coordinate data can be efficiently transmitted.

[Brief description of drawings]

[0118]

FIG. 1 is a block diagram showing an image predictive coding apparatus according to a first embodiment of the present invention.

FIG. 2 is a first schematic diagram showing image coordinate conversion in Embodiments 1 and 2 of the present invention.

FIG. 3 is a schematic diagram showing a coded image data sequence by the image predictive coding apparatus according to the first embodiment of the present invention.

FIG. 4 is a second schematic diagram showing image coordinate conversion in the first and second embodiments of the present invention.

FIG. 5 is a block diagram showing an image predictive decoding device according to a second embodiment of the present invention.

FIG. 6 is a schematic diagram showing images divided in Examples 1 and 2 of the present invention.

FIG. 7 is a schematic diagram showing an image divided by a conventional method.

FIG. 8 is a block diagram of a digital image decoding apparatus according to a third embodiment of the present invention.

FIG. 9 is a block diagram of a digital image coding apparatus according to a third embodiment of the present invention.

FIG. 10 is a block diagram of a digital image decoding apparatus according to a fourth embodiment of the present invention.

FIG. 11 is a block diagram of a digital image decoding device according to a fifth embodiment of the present invention.

FIG. 12 is a block diagram of a digital image coding apparatus according to a fifth embodiment of the present invention.

[Explanation of symbols]

[0119] 101 input terminal 102 first adder 103 encoder 106 output terminals 107 Decoder 110 Second adder 111 First coordinate converter 112 Second coordinate converter 113 motion detector 114 motion compensator 115 frame memory 201 Target image 203 coordinate system 304 Parameter by coordinate conversion 305 Parameter by coordinate conversion 501 input terminal 502 Data analyzer 503 Decoder 506 adder 507 output terminal 508 coordinate converter 509 motion detector 510 frame memory 801 input terminal 802 variable length decoding unit 803 Difference image decompression unit 804 Adder 805 output terminal 806 Conversion parameter generation unit 807 frame memory 808 prediction image generation unit 901 input terminal 903 conversion parameter estimation unit 904 First addition unit 905 Difference image compression unit 906 Transmission unit 907 output terminal 908 Prediction image generation unit 909 frame memory 910 Difference image decompression unit 911 Second addition unit 1001 input terminal 1002 variable length decoding unit 1003 Difference image decompression unit 1004 adder 1005 output terminal 1006 conversion parameter generation unit 1007 frame memory 1008 prediction image generation unit 1009 switch 1010 switch 1101 input terminal 1102 variable length decoding unit 1103 Difference image decompression unit 1104 First addition unit 1105 output terminal 1106 conversion parameter decompression unit 1107 frame memory 1201 input terminal 1203 conversion parameter estimation unit 1204 adder 1205 Difference image compression unit 1206 Transmission unit 1207 Second addition unit 1208 prediction image generation unit 1209 Frame memory 1210 Difference image decompression unit

Claims (2)

(57) [Claims] 1. A predictive decoding method for decoding compressed image data encoded with reference to a reference image, comprising: generating a predicted image from the compressed image data by coordinate conversion from a reference image at the time of encoding. Coordinate data information regarding the transformed coordinate points, number information of the transformed coordinate points, and a compressed image signal of the decoding target image are extracted, and a predicted image conversion method is selected based on the number information, and the selected predicted image Generate a prediction image at the time of decoding from the reference image based on the conversion method and the coordinate data information, to generate a difference image by decoding the compressed image signal, the prediction image at the time of decoding and the A predictive decoding method characterized by adding a difference image and generating a decoded image. 2. A predictive coding method for coding a target image with reference to a reference image, wherein a predictive image transforming method is estimated from the reference image and the target image, and based on the estimated predictive image transforming method. Generate a predicted image from the reference image, determine the coordinate data information about the number of transformation coordinate points corresponding to the predicted image conversion method, generate a difference image from the target image and the predicted image, the difference image A predictive coding method, comprising: compressing and coding to generate a compressed image signal, and transmitting coordinate data information regarding the number of converted coordinate points corresponding to the predictive image converting method together with the compressed image signal.