JPH0344270A - Picture interpolation system and picture coding system - Google Patents

Abstract

PURPOSE:To attain a smooth magnification interpolation picture white avoiding the increase in the scale of the hardware by applying similar processing to discrete inverse cosine conversion except revision of a conversion matrix in a frequency region where a picture is subject to discrete cosine conversion. CONSTITUTION:A picture data is split into a size of 1XN by a 1XN block segmentation section 1 and a linear discrete cosine conversion is applied in a linear DCT section 2 and the result is stored in a linear frequency buffer 3. A row direction magnification section 7 generates a picture subject to magnification and interpolation at a multiple of (k) in the row direction directly from the linear frequency and consists of a 1X2N block segmentation section, a 2NXkN matrix data section 5 and a matrix multiplication section 6. Since a smooth magnification interpolation picture subject to linear interpolation is obtained, the increase in the scale of hardware is not almost caused.

Description

[Detailed Description of the Invention] Industrial Application Field The present invention relates to image encoding. Conventional technology Orthogonal transform encoding is known as a redundant compression method for images. In recent years, orthogonal transformations include Hadamard transform, Fourier transform, and discrete cosine transform. Discrete cosine transform, in which much energy is concentrated in the low-order frequency region, is often used.

A> Image enlargement/interpolation Is it possible to enlarge the image using discrete cosine transform? Now, divide the image into N×N blocks,
A two-dimensional discrete cosine transform is performed for each block to transform the image into a two-dimensional frequency domain. ×kN
By expanding into blocks of
Although it is possible to obtain an image that has been enlarged twice, Figure 11 is a diagram that explains the doubling of an image. ×N
The two-dimensional discrete cosine transformation of the size is also performed in the same figure (b
) is converted into the N×N two-dimensional frequency domain F shown in (b). The enlarged 2N x 2N image block shown in (d) of the same figure is obtained by enlarging the 2N x 2N enlarged image block shown in (d) to the 2D frequency domain. The enlarged image obtained by the enlargement method is kN×
Within each block of kN, interpolation is performed smoothly. (a) Interpolated image from low frequency components Obtain an interpolated image that approximates the original image from the low frequency components in the frequency domain obtained by discrete cosine transformation of the reproduced image. Figure 12 shows this, and for each block obtained by dividing the image shown in Figure (a) into N
A two-dimensional discrete cosine transform of N size is performed and the same figure (b)
Then, as shown in the same figure (c), only the low frequency component is made effective, and the high frequency component is set to 0. For this low frequency component, the N×N size 2 By performing dimensional discrete inverse cosine transformation, the same figure (d
) The N×N interpolated/regenerated image block shown in ) is obtained, and interpolation is performed smoothly within each N×N block on the image 1 reconstructed from this low frequency component.) Image Coding Discrete Cosine Transformation When performing hierarchical encoding using A method is used in which the difference between the image and the original image is taken, and the second stage encoding is performed on the frequency component obtained by performing discrete cosine transform on the difference component.At the time of decoding,
First, the encoded data of the first stage is decoded and the DC component is focused.Based on this, an interpolated image is reproduced to display the outline of the image.Next, the encoded data of the second stage is decoded and the interpolated image is generated. It is possible to reconstruct the original image by restoring the difference component from the original image and adding the previous interpolated image to it.However, the problem to be solved by the invention is 1, conventional technology (a). It is closed in block units, and smooth expansion interpolation is performed within each block. α Continuity between blocks is not taken into consideration. Therefore, block distortion occurs in the enlarged image, especially when the enlargement rate is large. The more noticeable the block distortion becomes, the more conspicuous the block distortion becomes.2, Conventional technology (a) The process of interpolating and reproducing the image from low-frequency components is also the same as the problem of conventional technology (a).The processing is closed on a block-by-block basis. Smooth interpolation/reproduction is performed within each block.a Continuity between blocks is not considered.t,%
For this reason, block distortion occurs in interpolated/reproduced images.In particular, the fewer low-frequency regions are used, the more noticeable block distortion becomes.Momo 3, Conventional Technique 1) Encoding images hierarchically In this case, it is desirable that the DC component in the frequency domain obtained by performing discrete cosine transform to find the difference component between the interpolated image and the original image by interpolating the DC component is zero. The coding efficiency can be improved by targeting only the alternating current component in the frequency domain of the difference component.For this purpose, Ct is the average value of the interpolated image and the average value of the original image in each block, which is the unit of discrete cosine transform. It is necessary to match the above, and it is necessary to generate an interpolated image that satisfies this. An example of such an interpolation method is "Hiroko Kato, "Improving Coding Efficiency of Mean Separated Block Coding Method," 1986 Institute of Electronics and Communication Engineers. Although a method using quadratic surfaces has been proposed in General National University A 115a, 1986-34, however, the generation of interpolated images that takes into account the average value of each block is generally complex, and requires a large amount of hardware and processing. It has the disadvantage of increasing time.

The present invention takes into consideration this point. Regarding image enlargement, Ct
Image interpolation that allows you to obtain a smooth enlarged/interpolated image while avoiding an increase in hardware scale.For image reproduction from square or low frequency components, generate an interpolated image that approximates the image without block distortion while avoiding an increase in hardware scale. It is an object of the present invention to provide an image coding method that can achieve high coding efficiency while avoiding an increase in hardware scale. a) For image enlargement and interpolation, the original image is divided into blocks of 1 row x N columns, and each block is multiplied by an N x N transformation matrix to perform a one-dimensional discrete cosine transformation to create the original image. When converting to the one-dimensional frequency domain and restoring the image from the one-dimensional frequency domain, 1
A first block B1 of 1 row x N columns in the dimensional frequency domain and a second block B1 of 1 row x N columns adjacent to the first block Bl in the row direction
Combine block B2 with block B2 to form a combined block B12 with 1 row and 2N columns. Multiply the combined block B12 with matrix C consisting of 2N rows and kN columns to generate image block R1 with 1 row and kN columns. Then, each element of the matrix C with 2N rows and kN columns is represented by a linear combination of the elements of the NxN inverse transformation matrix that expresses the one-dimensional discrete inverse cosine transformation for 1 row and N columns.For the linear combination, 1 The first block B1 of the original image corresponding to the first block B1 in the dimensional frequency domain
All elements of block AI and the second one-dimensional frequency domain
A second block A2 of the original image corresponding to block B2 of
The first block AI of the original image using the first element of
A) Low Regarding interpolated image reproduction from frequency components, the G1 original image is divided into blocks of 1 row x kN columns, and each block is multiplied by a kN x kN transformation matrix to perform a one-dimensional discrete cosine transformation to transform the original image. The one-dimensional frequency domain is converted to a one-dimensional frequency domain, and only N elements are selected from the low frequency components of each 1-row x kN-column block in the one-dimensional frequency domain to form a 1-row x N-column block. The reduced one-dimensional frequency domain, which is reduced by a factor of 1 in the row direction, is specially reduced by 1
The processing procedure for restoring an image that approximates the original image from the dimensional frequency domain is to perform a one-dimensional discrete inverse cosine transform of N x N size for each 1 row x N column block in the reduced one-dimensional frequency domain. A procedure is used to restore an interpolated image that approximates the original image by enlarging the reduced image to a factor of 1 in the row direction and performing linear interpolation on the reduced image. % The realization of these steps is
The first block E1 of 1 row × N columns in the reduced one-dimensional frequency domain and the 1 row × N adjacent in the row direction of the first block El
The second block E2 in the column is combined to form a combined block E12 with 1 row x 2N columns, and the combined block E12 and 2N rows x
1 by performing matrix operation with matrix C consisting of kN columns.
An image block TI with rows and kN columns is generated. At this time, each element of the matrix C with 2N rows and kN columns is an N×N inverse transformation matrix that expresses a one-dimensional discrete inverse cosine transformation for 1 row and N columns. Represented by a linear combination of each element.For the linear combination, all elements of the first block DI of the reduced image corresponding to the first block El of the reduced one-dimensional frequency domain and reduced 1
Using the first element of the second block D2 of the reduced image corresponding to the second block E2 of the dimensional frequency domain, the first block D1 of the reduced image is enlarged twice and linearly interpolated into 1 row×kN Each element of the block of columns has a means for determining a linear combination so that each element of the image block TI has 1 row x kN columns. 1 obtained by dividing into columns of blocks and performing one-dimensional discrete cosine transformation on each block.
In the dimensional frequency domain, select only the elements of the DC component of each block of 1 row x N columns and record the 1-dimensional DC acid content domain by reducing the 1-dimensional frequency domain to 1/N in the row direction.1-dimensional DC A reduced image in which the original image is reduced to 1/N in the row direction by multiplying each element of the acid region by an l × 1 inverse transformation matrix, that is, a constant, and performing one-dimensional discrete inverse cosine transformation. The first step is to restore an interpolated image that approximates the original image by enlarging this N times in the row direction and performing linear interpolation.
Realization of the first process (i) The first element 1''1 of the one-dimensional DC acid region and the second element F2 adjacent in the row direction of the first element F1 are combined to form one row x Assuming a 2-column combined block F12, perform matrix operations on the combined block F12 and a 2-row x N-column matrix C' to generate a 1-row x N-column image block Wl. At this time, a 2-row x N-column matrix C is generated. .

Each element [Above: Represented by a linear combination of constants expressing a one-dimensional discrete inverse cosine transform for 1 row x 1 column About the linear combination & Above: The first element F of the one-dimensional DC component domain
The first element Gl of the reduced image corresponding to 1 and the second element Gl of the reduced image corresponding to the second element F2 of the one-dimensional DC component region
The first element G1 of the reduced image is expanded N times using element G2 of
A linear combination is determined so that each element of an image block Wl of 1 row by N columns is determined, and a matrix C is determined by the linear combination.

(b) Divide the original image into blocks of N rows x N columns, and perform two-dimensional discrete cosine transformation on each block to obtain the two-dimensional frequency domain. Select only the DC component elements of each N row x N column block and reduce the two-dimensional frequency domain to 1/N in both the row and column directions using the two-dimensional DC domain method. Or, by performing the first process in the order of column and row directions, 2
The second process is the process of restoring an interpolated image that approximates the original image from the dimensional DC component area. Find the difference component from the average value of the elements in the block of N rows x N columns for each block,
For each block of the interpolated image obtained in the second process, when the image obtained by subtracting the previous average value difference component from the pixels in the block is used as the average value corrected interpolated image, the average value corrected interpolated image is obtained from the two-dimensional DC component area. The process to obtain this is the third process.The third process is realized by changing the linear combination that defines each element of the 2N rows by N columns matrix C° in the first process and applying this to the second process. (d) When encoding the original image, the DC component in the two-dimensional frequency domain obtained by performing two-dimensional discrete cosine transform on the original image in block units of N rows x N columns is After encoding it to the first stage code data, generate an interpolated image from the DC component using the second process, find the difference component between the interpolated image and the original image, and calculate N rows x N for the difference component. Only the alternating current component of the two-dimensional frequency domain obtained by performing two-dimensional discrete cosine transform on a block-by-column block basis is encoded as second-stage encoded data. During decoding, the first stage is Based on the DC component obtained by decoding the encoded data, a second process restores an interpolated image that approximates the original image.
In the second stage of decoding, a mean value corrected interpolated image is obtained from the DC component through third processing. Furthermore, a DC component O is added to the AC component obtained by decoding the encoded data of the second stage, resulting in a two-dimensional discrete inverse cosine. The present invention also works by adding the data obtained by the conversion to the average value corrected interpolated image. Then, perform the same processing as discrete inverse cosine transformation except for changing the transformation matrix. Generate an enlarged and linearly interpolated image. A) Regarding interpolated image reproduction from low frequency components, convert the top image to discrete cosine By performing processing similar to discrete inverse cosine transformation on the low frequency components of the transformed frequency domain, except for changing the transformation matrix, an interpolated image that approximates the original image is generated from the low frequency components. For encoding, perform the first stage encoding on the DC component in the frequency domain obtained by discrete cosine transform of the image. A hierarchical code structure is created by performing second-stage encoding on the AC components of the area. During decoding, first the first-stage encoded data is decoded and interpolated to display the outline of the image, and then the first and second stages are encoded. Based on the two-stage encoded data, DC component information not included in the second-stage encoded data is reproduced and corrected to restore the original image, but interpolated image generation processing from the DC component and second-stage encoded data are also performed. Reproduction and correction processing of DC components not included in (↓
Processing similar to discrete inverse cosine transformation is used except for changing the transformation matrix.

Embodiment A) Embodiment of image enlargement/interpolation processing Figure 1 shows the first embodiment of the present invention, and is a plot diagram of a device for enlarging and interpolating an image twice in the row direction. In the figure, the image data is divided into I×N sizes at one 1×N block extraction section, and further divided into two one-dimensional D
A one-dimensional discrete cosine transform is performed in the CT section and stored in a one-dimensional frequency buffer (No. 3). A row direction enlargement section (No. 7) generates an image that is doubled and interpolated directly from the one-dimensional frequency in the row direction. Q, consists of 4 1x2N block cut outs, 5 2NxkN matrix data elements, and 6 matrix multipliers.The processing of the row direction enlarger 7 will be specifically described below. This is a diagram showing double enlargement and interpolation in the row direction, where the original image data of the I×N block in (a) is doubled by linear interpolation. Linear interpolation data a As is clear from the figure, in order to expand and linearly interpolate the original image data a l ''-' a H in (a), all elements of the block aP...aN, aP...・By using 4b+, which requires the first element b1 of the block adjacent to the am block, it is possible to smoothly interpolate between blocks.One-dimensional discrete cosine transformation for I×N image blocks is expressed as equation (1) (F+, Pg,...
, Fw) (1) Equation Te, fi, f2. ...4-I×N image block.

Cz (1≦t+J≦N) is the transformation matrix f, Ft, Ft, ..., F that expresses the one-dimensional discrete cosine transformation.
M is the frequency component corresponding to the image block, and also (
The discrete inverse cosine transform for equation 1) can be expressed as equation (2) (hereinafter, blank) (f+, b,...,h) Now, the original image data B+,... ..., all
The frequency components for the block are F×, ..., Fa
- Let the frequency components for the blocks of original image data b+,..., bw be h+,..., Fbw.These frequency components are stored in the one-dimensional frequency buffer 3 in FIG. According to the invention, the expansion and linear interpolation shown in FIG. 2(b) can be performed directly from the frequency components, i.e., al
,... Frequency component F necessary to find all pixels
・1. ..., using the frequency components Fbt, ..., Fbm necessary to find the pixels of F@n and bl (3)
By performing matrix operations on the equation, the double enlarged and linearly interpolated data shown in Figure 2 (b) can be obtained. Even if it is expressed by a linear combination of the elements of the N×N matrix in the second term on the right-hand side of The above processing is the first
In the device shown in the figure, F−1, the first term on the right side of equation (3),
..., FsN, Fil, ..., Fbn are taken out from the one-dimensional frequency buffer 3 by the 1x2N block extraction unit 4, and the 2Nx2N matrix data of the second term on the right side of equation (3) is 2N
×kN is stored in the matrix data section 5, and matrix multiplication of both is performed in the matrix multiplication section 6, and the enlarged/linearly interpolated data on the left side of equation (3) is output. Although it is possible to obtain an image that has been enlarged twice and linearly interpolated, Equation (3) can also be extended to perform k times enlargement and linear interpolation. Also, this is on the image data.
Based on N and b+ pixels! Q at,..., aN
In order to generate 1×kN enlarged image data by enlarging and linearly interpolating the blocks of (Fat, “”, Fan, Fbl, “”, FbN)
(4) The left side of the equation is ×N size image data that has been expanded twice and linearly interpolated, and the first term on the right side is F, 1. ..., Fall and F are...
..., 1×2N frequency component consisting of Fbn, second right side
Even if the term is a 2N×kN matrix, the second term on the right side is 2N×kN
Each element αz of the matrix (1≦i≦2N, 1≦j≦kN)
i! N×N of the second term on the right side of equation (3) and equation (2) for the same aircraft
It can also be expressed as a linear combination of matrix elements, i.e. (
4) The condensative combination is determined so that the multiplication result on the right side of the equation (4) becomes the value on the left side of the equation (4). This is a block diagram of the device that doubles the enlargement and interpolation. In the same figure, the image data is divided into N×N sizes in the N×N block cutting section of the IO, and is further divided into two-dimensional discrete pieces in the 11 two-dimensional DCT sections. Based on the two-dimensional frequency data, the data is subjected to cosine transformation and stored in the 12 two-dimensional frequency buffers.The 2-dimensional frequency data is then doubled in the row direction in the 13 row-direction expansion sections, and then the column data is doubled in the 14 column-direction expansion sections. Here, the row direction enlargement unit 13ζ table is similar to the row direction enlargement unit 7 in FIG. 1 in the first embodiment. The column-direction enlarging section 144 may also be constructed as shown in FIG. 4 to carry out the processing of the row-direction enlarging section 7 of FIG. The output part divides the double data in the row direction from the row direction enlarging unit 13 in FIG. 3 into 2Nx l blocks, 4 16 is kNx
2N matrix data part, k for column direction expansion
Even if Nx2N matrix data is stored, both of them are multiplied by 17 matrix multipliers and an image that is doubled in the row and column directions is output. It is possible to generate an image that is doubled in size and linearly interpolated in both column directions.B) Example of interpolated image generation processing from low frequency components Figure 5 shows the third embodiment of the present invention. 1-dimensional DCT
This is a block diagram of a device that generates an interpolated image that approximates the original image using only low frequency components in the frequency domain obtained by The data is divided into blocks of size kN, further subjected to one-dimensional discrete cosine transform in 21 one-dimensional DCT units, and stored in 22 one-dimensional frequency buffers, resulting in 41-dimensional frequency data containing low frequency components for each 1×kN block. An I×N block is cut out by the 1×N low frequency block cutting unit 23, reduced to 1/k, and stored in the reduced one-dimensional frequency buffer 24.The image reproduction unit 28 extracts the I×N block from the reduced one-dimensional frequency. It reproduces an interpolated image that approximates the direct light image, and has 25 lx2n blocks cut out.
In order to explain the processing of the image reproduction unit 28, which consists of 26 2N×kN matrix data and 27 matrix multiplication units, we will first explain the principle of interpolation image reproduction from low frequency components. 1 from the low frequency component in the frequency domain
This refers to the process of reproducing an interpolated image that approximates the original image from a one-dimensional frequency reduced to l/2 by extracting only the It is transformed into the one-dimensional frequency domain of (b) by one-dimensional DCT of divided filX2N size, and then (b)
By selecting 1/2 region from the low frequency component of each 1x2N block in the one-dimensional frequency region of (c) to create an IxN block, the one-dimensional frequency region (low frequency region) reduced by 1/2 is obtained. If we perform I×N size one-dimensional inverse DCT on the 1/2 reduced one-dimensional frequency domain in (c), we can reproduce the l×N size reduced image block in (d).
Even though the reduced image block in (d) is the image block in (a) reduced to 1/2, by further enlarging the reduced image block in (d) to 2 times and performing linear interpolation, the image block in (a) can be Enlarged/interpolated image that approximates the image block (
e) can be played back to (c)
Although it is shown that it is possible to reproduce an interpolated image that approximates the original image from the 1/2 reduced one-dimensional frequency region (low frequency region) of (c), the 1/2 reduced one-dimensional frequency region (low frequency region) The procedure for obtaining the enlarged/interpolated image from ) to (e) can be the same as the enlarged/interpolated image using the frequency domain shown in the first embodiment. The processing at 28 may be configured to perform the same processing as the row direction enlarging unit 7 in FIG. 1 in the first embodiment. Two adjacent ×N low frequency blocks are combined to form an I ×2N block, and A 2N ×
The kN matrix data section 26 is 2N×2 of the second term on the right side of equation (3).
The matrix multiplier 27 stores N matrix data, and performs matrix multiplication of both to generate interpolated image data that approximates the original image. Although it is possible to extend the power strength described in the embodiment to a two-dimensional frequency domain by two-dimensional DCT using the same procedure as shown in the second embodiment.
In other words, it is possible to generate an interpolated image that approximates the original image by using only low frequency components in the two-dimensional frequency domain. The interpolated image generation process from the low frequency components shown in the third embodiment is used. Figure 8 is a block diagram of the equipment that decodes the rough image in the first stage. Figure 9 is a block diagram of the equipment that decodes the final image in the second stage. In the encoding device shown in the figure, the original image is subjected to two-dimensional discrete cosine transform in 30 two-dimensional DCT units, and its two-dimensional frequency base components are converted into three
Furthermore, the DC component of the two-dimensional frequency component is extracted by the DC component extraction unit 32 and encoded by the first stage encoding unit 33, and is converted into first stage encoded data. At the same time, the interpolation image generation section 34 generates an interpolation image from the DC component and stores it in the interpolation image buffer 35. Here, the interpolation image generation section 34 generates the interpolation image from the low frequency component shown in the third embodiment. Since it performs image generation processing, the difference component between the original image and the interpolated image is found, and the difference component is 3.
Although it is converted into a two-dimensional frequency by two-dimensional discrete cosine transformation in the two-dimensional DCT section of No. 6 and stored in the two-dimensional frequency buffer of No. 37, the DC component of the two-dimensional frequency of the difference component is ignored, and only the AC component is converted into an AC of No. 38. AC component extraction unit 38 extracts the AC component extraction unit and encodes it in the second-stage encoding unit of the C39 to obtain second-stage encoded data.
The information on the DC component discarded in can be reproduced and corrected during decoding. Correction of the DC component will be explained in detail later, but as shown above, the second stage of encoding does not target the DC component, so Although it is possible to improve the encoding efficiency, the above processing allows the first stage encoded data to have the DC component information of the original image, and the first stage encoded data to have the difference component information between the original image and the interpolated image from the DC component. What is the purpose of obtaining two-stage encoded data? The first stage decoding unit decodes the DC component and stores it in the DC component buffer 41. Furthermore, the first stage interpolation image is generated by the interpolation image generation unit 42 based on the DC component. Here, the interpolation is performed. The image generation section 42 performs interpolation image generation processing from the low frequency components shown in the third embodiment, and is the same as the interpolation image generation section 34 shown in FIG. 7.The device shown in the block diagram of FIG. In the following explanation, the interpolated image obtained from the DC component will be referred to as the DC component interpolated image. In this method, the difference component between the DC component interpolated image and the original image is reproduced from the second stage code data, and the original image is reproduced by adding the DC component interpolated image reproduced from the first stage code data. In the second stage encoding process, the DC component of the two-dimensional frequency of the difference component is discarded, so
It is necessary to correct this at the time of decoding.I will explain the correction of the DC component here, but Figure 1C is a diagram explaining the correction of the average value in a one-dimensional area.As shown in the figure, the DC component interpolated image is The average value (within a block) and the original image average value (within a block) are generally different values. The image corresponding to the DC component is defined as an average value-corrected interpolated image by giving a constant correction value to the DC component interpolated image so that its average value matches the average value of the original image. As shown in Figure 10, the average corrected interpolated image is obtained by translating the DC component interpolated image, and at this time, the amount of movement, that is, the correction value, is the average of the DC component interpolated image. Even if it is the difference between the value and the average value of the original image.
Figure 0 is for a one-dimensional area.For a two-dimensional area, the principle of Figure 10 is extended in the row and column directions.The difference between the DC component interpolated image and the original image is 2
This is because the DC component of the two-dimensional frequency component obtained by dimensional discrete cosine transformation is set to 0. This is because it is equivalent to the difference component between the mean value corrected interpolated image and the original image, which is subjected to two-dimensional discrete cosine transformation. For example, the average value of the average value corrected interpolated image and the average value of the original image are equal for each block, and the average value corrected interpolated image and the DC component interpolated image are similar within each block. The DC component of the first stage encoded data is decoded in the first stage decoding unit 44 and stored in the DC component buffer 45. The corrected interpolated image is generated in the row and column directions, and this is the third
This can be realized by partially changing the interpolated image generation process from the low frequency components shown in the embodiment. In other words, for equation (4) used in the third embodiment, 2N×k of the second term on the right side.
The corrected interpolated image generation unit 46 can be realized by changing each element of the N matrix into a form that can directly generate an average value corrected interpolated image from the DC component. Even though the AC component in the frequency domain of the difference component between the original image and the DC component interpolated image is decoded in the section, the AC component is left as is, the DC component is set to O, and a discrete inverse cosine transform is performed by the 49 two-dimensional inverse DCT section. By doing this, the difference component between the average value corrected interpolated image and the original image can be obtained according to the principle explained earlier. By adding this to the average value corrected interpolated image which is the output of the corrected interpolated image generation section 46, the original image is restored. However, as described in detail, according to the image interpolation method of the present invention, there are three problems regarding image enlargement. By simply performing processing similar to discrete inverse cosine transformation, it is possible to obtain smooth enlarged and interpolated images with linear interpolation.For this reason, there is almost no increase in hardware scale, and image reproduction from low frequency components is difficult. , Extract only the low-frequency components of the frequency domain by performing discrete cosine transform on the image, and perform the same processing as discrete inverse cosine transform except for changing the transformation matrix for these low-frequency components.
It is possible to generate an interpolated image that is close to the original image and has no block distortion. The image encoding method of the present invention is particularly effective for obtaining images without block distortion, and requires almost no increase in hardware scale.Furthermore, according to the image encoding method of the present invention, the difference component between the interpolated image from the DC component and the original image is Encoding
In addition to improving the coding efficiency by using the correlation between blocks, it is also possible to improve the coding efficiency by not using the DC component in the frequency domain of the difference component as a coding target. Process 4: Correcting the DC component in the frequency domain of the difference component during decoding Process 4: Processing equivalent to discrete inverse cosine transform, except for changing the elements of the transformation matrix. There is almost no increase (chi

[Brief explanation of drawings]

Figure 1 shows the blocks of the device for enlarging and interpolating an image in the row direction in the first embodiment of the present invention. Figure 2 shows the block diagram for explaining linear interpolation. The blocks of the apparatus for enlarging and interpolating the image in the row and column directions in the example are as shown in Fig. 4. The blocks for explaining the column direction enlarging section in Fig. 3 are as shown in Fig. 5. Figure 6 illustrates the principle of interpolated image generation from low frequency components. The blocks in FIG. 8 are the blocks of the first stage decoding section of the image encoding device in the fourth embodiment of the present invention. The block of the two-stage decoding section is
Figure 10 is an explanation of the average value correction interpolated image. Figure 11 is an explanation of double enlargement using frequency components in the conventional example. Figure 12 is interpolation and reproduction of an image from low frequency components in the conventional example. This is a diagram to explain l...I×N×
N block 2... 1-dimensional DCT 3... 1-dimensional frequency buffer 4... 1×2N block cut out a
5... 2N x kN matrix data enemy 6... Matrix multiplication 7... Row direction expansion 20... I x kN block cutting test 21... 1-dimensional DCT starvation 22... 1-dimensional Frequency buffer, 23...1×N low frequency block cutout 24...Reduced one-dimensional frequency buffer, 25...1
×2N block cutting 26...2N×kN matrix data R, 27...Matrix multiplication 28...Image reproduction star 30.
...Two-dimensional DCTK 31... Two-dimensional frequency buffer, 32... DC fraction extraction enemy 33... First stage encoding 34... Interpolated image generation! 35... Interpolation image buffer, 36... 2-dimensional DCT 37... 2-dimensional frequency buffer, 38... AC component sleeve 39.
...Second stage encoding 40...First stage decoding 73
41... DC component buffer, 42... Interpolated image raw T
li, m 44...first stage decoding K 45...
・DC component buffer, 46...Corrected interpolation image generation 4
7...Second stage decoding 48...AC component buffer, 49...2-dimensional inverse DCT!

Claims (5)

[Claims] (1) In the process of enlarging and linearly interpolating the original image by k times in the row direction, the original image is divided into blocks of 1 row x N columns, and each block is multiplied by an N x N transformation matrix. When converting the original image into a one-dimensional frequency domain by performing a one-dimensional discrete cosine transformation, and restoring the image from the one-dimensional frequency domain, the first A block B1 and a second block B2 of 1 row x N columns adjacent to the first block B1 in the row direction are combined to form a combined block B12 of 1 row x 2N columns, and the combined block B12 and the 2N rows x By performing multiplication with a matrix C consisting of kN columns, an image block R1 of 1 row x kN columns is generated, and at this time, each element of the matrix C of 2N rows x kN columns is
It is represented by a linear combination of each element of an N×N inverse transformation matrix that expresses a one-dimensional discrete inverse cosine transformation for 1 row by N columns, and the linear combination is expressed in the first block B1 of the one-dimensional frequency domain. a second block A2 of the original image corresponding to all elements of the corresponding first block A1 of the original image and a second block B2 of the one-dimensional frequency domain;
The first block A1 of the original image is expanded k times and linearly interpolated using the first element of the 1 row x kN column block. An image interpolation method characterized in that the linear combination is determined so as to become each element, and one-dimensional expansion/interpolation processing is performed. (2) Divide the original image into blocks of N rows x N columns, perform two-dimensional discrete cosine transformation on each block to transform the original image into a two-dimensional frequency domain, and convert the original image into a two-dimensional frequency domain. An image interpolation method characterized in that two-dimensional enlargement/interpolation of an image is performed by performing the procedure according to claim 1 in the row direction and column direction or in the column direction and row direction. (3) Divide the original image into blocks of 1 row x kN columns, and perform one-dimensional discrete cosine transformation by multiplying each block by a kN x kN transformation matrix to transform the original image into 1
By converting into a 1-dimensional frequency domain and selecting only N elements from the low frequency components of each 1-row x kN-column block in the one-dimensional frequency domain to form a 1-row x N-column block,
A processing procedure for obtaining a reduced one-dimensional frequency domain by reducing the one-dimensional frequency domain by a factor of k in the row direction, and restoring an image that approximates the original image from the reduced one-dimensional frequency domain is as follows: A reduced image in which the original image is reduced to 1/k in the row direction is obtained by performing a one-dimensional discrete inverse cosine transformation of N x N size on each of the blocks of 1 row x N columns, and the reduced image is A procedure is used to restore an interpolated image that approximates the original image by enlarging and linearly interpolating the image by k times in the row direction. 1 block E1 and a second block E2 of 1 row x N columns adjacent to the first block E1 in the row direction are combined to form a combined block E1 of 1 row x 2N columns.
2, an image block T1 of 1 row x kN columns is generated by performing a matrix operation on the combined block E12 and the matrix C consisting of 2N rows x kN columns, and at this time, the image block T1 of 1 row x kN columns is generated.
Each element of the column matrix C is represented by a linear combination of the elements of an N×N inverse transformation matrix expressing a one-dimensional discrete inverse cosine transformation for 1 row × N columns, and for this linear combination, the reduction 1 All elements of the first block D1 of the reduced image corresponding to the first block E1 of the reduced one-dimensional frequency domain and of the second block D2 of the reduced image corresponding to the second block E2 of the reduced one-dimensional frequency domain. Each element of the block of 1 row x kN columns obtained by enlarging the first block D1 of the reduced image by k times and linearly interpolating the first block D1 of the reduced image using the first element is
The linear combination is determined to be each element of the image block T1 of 1 row x kN columns, and an interpolated image that approximates the original image is obtained from the low frequency components of the one-dimensional frequency domain. Image interpolation method. (4) Divide the original image into blocks of kN rows x kN columns, perform two-dimensional discrete cosine transformation on each block to transform the original image into a two-dimensional frequency domain, and convert the original image into a two-dimensional frequency domain. By selecting only N elements in both the row and column directions from the low frequency components of each kN row by kN column block to form an N row by N column block, the two-dimensional frequency domain can be divided into rows and columns. A reduced two-dimensional frequency region is obtained that is reduced to 1/k in both directions, and the reduced two-dimensional frequency region is arranged in the row direction and column direction or in the column direction and row direction in the order of claim 3.
4. The image interpolation method according to claim 3, wherein an interpolated image that approximates the original image is restored from the low frequency components of the two-dimensional frequency domain by performing the procedure described in claim 3. (5) In image encoding, (a) Divide the original image into blocks of 1 row x N columns, and perform one-dimensional discrete cosine transform on each block. A one-dimensional DC component region is obtained by reducing the one-dimensional frequency region to 1/N in the row direction by selecting only the DC component elements of a 1-row×N-column block, and each element of the one-dimensional DC component region is is multiplied by a 1×1 inverse transformation matrix, that is, a constant, to perform one-dimensional discrete inverse cosine transformation to obtain a reduced image in which the original image is reduced to 1/N in the row direction, and the reduced image is The first process is a process of restoring an interpolated image that approximates the original image by enlarging N times in the row direction and performing linear interpolation. element F1 and a second element F2 adjacent to the first element F1 in the row direction are combined to form a combined block F12 of 1 row x 2 columns, and the combined block F12 and a matrix C of 2 rows x N columns are combined. An image block W1 of 1 row x N columns is generated by performing a matrix operation with
It is represented by a linear combination of constants expressing a one-dimensional discrete inverse cosine transformation for one column, and for this linear combination, the first element G1 of the reduced image corresponding to the first element F1 of the one-dimensional DC component region and the first element G1 of the reduced image is enlarged by N times using the second element G2 of the reduced image corresponding to the second element F2 of the one-dimensional DC component region and linearly interpolated, 1 line×N Each element of the column block is
(b) determining the linear combination so that each element of the image block W1 of 1 row x N columns is obtained, and realizing the first processing using the matrix C' determined by the linear combination; Of the two-dimensional frequency domain obtained by dividing the image into blocks of N rows and N columns and performing two-dimensional discrete cosine transformation on each block, only the DC component elements of each block of N rows and N columns are obtained. is selected to obtain a two-dimensional DC component region in which the two-dimensional frequency region is reduced to 1/N in both the row and column directions. A second process is a process of restoring an interpolated image that approximates the original image from the two-dimensional DC component region by sequentially performing the first process, and (c)
The difference component between the average value of the elements in the N rows × N columns block of the interpolated image obtained in the second process and the average value of the elements in the N rows × N columns block of the original image is calculated for each block. seek,
For each block of the interpolated image obtained in the second process, when an image obtained by subtracting the difference component from the pixels in the block is used as an average value corrected interpolated image, the average value corrected interpolated image is obtained from the two-dimensional DC component region. The process of obtaining
(d) When encoding the original image, A DC component in a two-dimensional frequency domain obtained by performing two-dimensional discrete cosine transform on a block basis of N rows by N columns is encoded as code data of the first stage, and from the DC component, the second The interpolated image is generated using the process described above, a difference component between the interpolated image and the original image is obtained, and a two-dimensional discrete cosine transform is performed on the difference component in units of blocks of N rows and N columns. Of the two-dimensional frequency domain obtained, only the AC component is encoded as second-stage encoded data, and during decoding, the DC component obtained by decoding the first-stage encoded data is used as first-stage decoding. Based on the second process, an interpolated image that approximates the original image is restored, and as a second stage of decoding, an average value corrected interpolated image is obtained from the DC component by the third process, and then the second stage The original image is restored by adding data obtained by adding a DC component 0 to the AC component obtained by decoding the encoded data and performing two-dimensional discrete inverse cosine transformation to the average value corrected interpolated image. Featured image encoding method.