JP6864557B2 - Encoding device, decoding device, and program - Google Patents

Description

The present invention relates to a coding device, a decoding device, and a program thereof.

Conventionally, there is known a technique in which an input image is reduced, then encoded and decoded, and the decoded reduced image is subjected to super-resolution processing to be restored. For example, Patent Document 1 discloses an apparatus for increasing the resolution of a video encoded / decoded after reduction by using an inter-frame motion vector.

Further, Patent Document 2 discloses a device that provides a method of expressing a texture that does not make an unnatural feeling while compressing data. Specifically, the input signal is separated into subbands in two frequency domains, the low frequency components are encoded by a conventional coding device, and the high frequency components are analyzed and calculated to calculate typical texture parameters. The texture parameter is sent to the decoding side. Then, the decoding side decodes the low frequency component and replaces the high frequency component with a texture that is not unnatural.

Japanese Patent No. 4876048 International Publication No. 2008/066025

However, in the apparatus described in Patent Document 1, it is difficult to increase the resolution in a non-moving part, and it is possible to increase the resolution by using the inter-frame motion vector of the decoded video, but the image quality is increased. There is a problem that it is necessary for the coding side to obtain the inter-frame motion vector of the input video and transmit it to the decoding side.

Further, the apparatus described in Patent Document 2 has a problem that it is necessary to transmit the texture parameter of the high frequency component and it is necessary to prepare a database of the high frequency component based on the texture parameter on the decoding side. ..

An object of the present invention made in view of such circumstances is to solve the above problems, reduce the input image to improve the coding efficiency, and superimpose the high frequency component lost by the reduction process and the coding process in high image quality. It is an object of the present invention to provide a video coding device, a video decoding device, and a program capable of resolving and restoring.

In order to solve the above problems, the coding device according to the present invention is a coding device that encodes after reducing the input image, and is a reduced image that performs frequency decomposition processing on the input image to generate a reduced image. A generation unit, a compression unit that performs quantization processing on the reduced image to generate a quantization coefficient, and a compression unit that performs decompression processing on the quantization coefficient to restore the reduced image and output it as a restored reduced image. The stretched portion and the restored / reduced image are subjected to frequency decomposition processing to generate a high-frequency component image, and the frequency band in which the power value is equal to or higher than the first threshold value is determined as the used band in the high-frequency component image. It is characterized by including a determination unit and a coding unit that generates a bit stream in which the quantization coefficient and the usage band information indicating the usage band are encoded.

Further, in the coding apparatus according to the present invention, the reduced image generation unit performs n-th order wavelet decomposition processing on the input image to generate a low frequency component image, and generates each pixel value of the low frequency component image. The reduced image is converted into an integer, and the band used determination unit is characterized in that the restored and reduced image is subjected to m-th order wavelet decomposition processing to generate the high frequency component image.

Further, in the coding apparatus according to the present invention, when the value of n is equal to or higher than a predetermined value, the power value of the high frequency component image is equal to or higher than the first threshold value. It is characterized in that a frequency band that is present and whose frequency is equal to or lower than the second threshold value is determined as the band to be used.

Further, in the coding apparatus according to the present invention, the band used determining unit is characterized in that the larger the value of n, the smaller the second threshold value.

Further, in order to solve the above problems, the decoding device according to the present invention is a decoding device that performs super-resolution processing after restoring a reduced reduced image, and has a quantization coefficient of the reduced image and super-resolution processing. An acquisition unit that acquires used band information indicating the used band used in the above, a reduced image restoration unit that restores the reduced image by dequantizing the quantization coefficient and outputs it as a restored reduced image, and the restoration. A super-resolution restoration unit that generates a super-resolution image by performing frequency decomposition processing on a reduced image to generate a high-frequency component image of the used band and allocating the high-frequency component image to the high-frequency component of the restored reduced image. It is characterized by having.

Further, in order to solve the above problems, the decoding device according to the present invention is a decoding device that performs super-resolution processing after restoring a reduced reduced image, and includes an acquisition unit that acquires a quantization coefficient of the reduced image. A reduced image restoration unit that reverse-quantizes the quantization coefficient to restore the reduced image and outputs it as a restored reduced image, and a frequency decomposition process on the restored reduced image to generate a high-frequency component image. Among the high-frequency component images, a frequency band in which the power value is equal to or higher than the first threshold value is determined as the used band, and the restored and reduced image is frequency-resolved to obtain a high-frequency component image of the used band. It is characterized by including a super-resolution restoration unit that generates a super-resolution image by generating and allocating the high-frequency component image to the high-frequency component of the reduced image.

Further, in the decoding device according to the present invention, the super-resolution restoration unit includes a frequency decomposition unit that performs wavelet decomposition processing on the restoration / reduction image to generate a frequency decomposition image including the band used, and the restoration / reduction unit. An alignment unit that generates alignment information indicating the positional relationship of similar blocks between the image and the low-frequency component image of the frequency-resolved image, and as a high-frequency component of the restored and reduced image according to the alignment information. The super-resolution high-frequency component image that allocates the high-frequency component image of the frequency-resolved image to generate the super-resolution high-frequency component image, the restored and reduced image as the low-frequency component, and the super-resolution high-frequency component image as the high-frequency component It is characterized by including a frequency reconstruction unit that performs frequency reconstruction processing to generate the super-resolution image.

Further, in order to solve the above problems, the program according to the present invention is characterized in that the computer functions as the coding device.

Further, in order to solve the above problems, the program according to the present invention is characterized in that the computer functions as the decoding device.

According to the present invention, since the input image is reduced on the coding side, the coding efficiency can be improved. Further, on the decoding side, the high frequency component lost by the reduction processing and the coding processing on the coding side can be super-resolution restored with high image quality.

It is a block diagram which shows the structural example of the coding apparatus which concerns on 1st Embodiment of this invention. It is a figure which shows the processing example of the reduced image generation part in the coding apparatus which concerns on 1st Embodiment of this invention. It is a block diagram which shows the structural example of the decoding apparatus which concerns on 2nd Embodiment of this invention. It is a block diagram which shows the structural example of the super-resolution restoration part in the decoding apparatus which concerns on 2nd Embodiment of this invention. It is a figure which shows the outline of the processing of the alignment part in the decoding apparatus which concerns on 2nd Embodiment of this invention. It is a figure which shows the outline of the processing of the super-resolution high-frequency division attachment part in the decoding apparatus which concerns on 2nd Embodiment of this invention. It is a figure which shows the outline of the process of the frequency reconstruction part in the decoding apparatus which concerns on 2nd Embodiment of this invention. It is a block diagram which shows the structural example of the modification of the coding apparatus which concerns on 1st Embodiment of this invention. It is a block diagram which shows the structural example of the modification of the decoding apparatus which concerns on 2nd Embodiment of this invention.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

(Encoding device)
FIG. 1 is a block diagram showing a configuration example of the coding device 1 according to the first embodiment of the present invention. The coding device 1 reduces the input image, encodes it, generates a bit stream, and outputs the bit stream to the outside. In the example shown in FIG. 1, the coding device 1 includes a reduced image generation unit 10, a compression unit 20, a decompression unit 30, a band usage determination unit 40, and an entropy coding unit 50.

The reduced image generation unit 10 performs frequency decomposition (multi-resolution decomposition) processing on the input image to convert low frequency components into integers by rounding or the like, generates them as reduced images, and outputs them to the compression unit 20. The reduction ratio can be appropriately determined according to the transmission capacity. The reduced image generation unit 10 can use wavelet decomposition as the frequency decomposition. When the nth-order wavelet decomposition is performed, a low-frequency component image LL ⁿ of the input image is generated. Here, the subscript n means the decomposition rank.

FIG. 2 is a diagram showing a processing example of the reduced image generation unit 10. FIG. 2 shows the first-order wavelet decomposition processing for the input image. Each pixel value of the low-frequency component image LL ¹ is converted into an integer by rounding or the like to obtain a reduced image. Since the high-frequency component images LH ¹ , HL ¹ , and HH ¹ are not used, they are not output from the reduced image generation unit 10. In the example of FIG. 2, since the decomposition rank n = 1, a reduced image having a reduction ratio of 1/2 in both the horizontal direction and the vertical direction is generated.

The compression unit 20 performs a quantization process on the reduced image generated by the reduced image generation unit 10 to generate a quantization coefficient, and outputs the quantization coefficient to the decompression unit 30 and the entropy coding unit 50. The processing of the compression unit 20 is performed by a known method such as JPEG2000, AVC / H.264, HEVC / H.265 and the like.

As shown in FIG. 1, for example, the compression unit 20 includes a block division unit 21, a subtraction unit 22, a conversion unit 23, a quantization unit 24, and a prediction unit 25.

The block division unit 21 divides the reduced image generated by the reduced image generation unit 10 into a plurality of blocks, and outputs the block image to the subtraction unit 22 and the prediction unit 25. The size of the block may be variable, for example 32 × 32 pixels, 16 × 16 pixels, 8 × 8 pixels, or 4 × 4 pixels.

The subtraction unit 22 generates a residual block image by subtracting each pixel value of the prediction image input from the prediction unit 25, which will be described later, from each pixel value of the block image generated by the block division unit 21 to generate a conversion unit. Output to 23.

The conversion unit 23 performs conversion processing such as orthogonal transformation on the residual block image input from the subtraction unit 22, calculates the conversion coefficient subjected to the two-dimensional conversion processing, and calculates the conversion coefficient for each block in the quantization unit 24. Output to.

The quantization unit 24 generates a quantization coefficient by dividing the conversion coefficient for each block input from the conversion unit 23 in the quantization step and quantizes it, and outputs the quantization coefficient to the expansion unit 30 and the entropy coding unit 50. ..

The prediction unit 25 performs intra-prediction or inter-prediction with reference to the restored and reduced image stored in the storage unit 34, which will be described later, generates a prediction image, and outputs the prediction image to the subtraction unit 22. Further, when the intra prediction is performed, the prediction information indicating the selected intra prediction mode is output to the entropy coding unit 50, and when the inter prediction is performed, the prediction information indicating the motion vector is output to the entropy coding unit 50. Output.

The decompression unit 30 performs decompression processing on the quantization coefficient generated by the compression unit 20 to restore the reduced image, and outputs the restored and reduced image to the used band determination unit 40. The stretching portion 30 is processed by a known method such as JPEG2000, AVC / H.264, HEVC / H.265 and the like.

As shown in FIG. 1, for example, the stretching unit 30 includes an inverse quantization unit 31, an inverse conversion unit 32, an addition unit 33, and a storage unit 34.

The inverse quantization unit 31 restores the conversion coefficient for each block by multiplying the quantization coefficient input from the quantization unit 24 by a quantization step, and outputs the conversion coefficient to the inverse conversion unit 32.

The inverse transformation unit 32 performs the inverse transformation of the transformation performed by the conversion unit 23 with respect to the conversion coefficient input from the inverse quantization unit 31, restores the residual block image, and outputs the residual block image to the addition unit 33. For example, when the conversion unit 23 performs the discrete cosine transform, the inverse transform unit 32 performs the inverse discrete cosine transform.

The addition unit 33 adds the residual block image input from the inverse conversion unit 32 and each pixel value of the prediction image input from the prediction unit 25 to generate a restored reduced image in block units, and stores the storage unit 34. Output to.

The band used determination unit 40 performs frequency decomposition processing on the restored and reduced image to generate a high frequency component image. The used band determination unit 40 can use wavelet decomposition as the frequency decomposition. In the present embodiment, the m-th order wavelet decomposition processing is performed ^{to generate high-frequency component images LH k} , HL ^k , and HH ^k (k = 1 to m). Then, the used band determining unit 40 determines the used band α, which is the band used in the super-resolution processing in the decoding apparatus described later, from ^{the high-frequency component images LH k} , HL ^k , and HH ^{k, and indicates the used band α.} The used band information is output to the entropy coding unit 50.

The method of determining the used band by the used band determining unit 40 will be described below. In the first method for determining the used band, the frequency band in which the power value is equal to or higher than the threshold value P is determined as the used band α among ^{the high frequency component images LH k} , HL ^k , and HH ^k. The power value is, for example, an RMS (Root Mean Square) value. If any of the power values of the high-frequency component images LH ^k , HL ^k , and HH ^k is equal to or higher than the threshold value P, ^{all the frequency bands of the high-frequency component images LH k} , HL ^k , and HH ^k may be used as the band α. For example, if the decomposition rank m = 5 and ^{the power values of the high-frequency component images LH 3} and HL ⁴ are equal to or higher than the threshold value, the frequency bands of the high-frequency component images LH ³ , HL ³ , HH ³ , LH ⁴ , HL ⁴ , and HH ⁴ are used. Is determined as the used band α. In this case, since the used band α can be indicated by the decomposition rank, α = 3 or 4 may be set.

Next, in the second used band determination method, when the decomposition order n of the wavelet decomposition in the reduced image generation unit 10 is a predetermined value x (for example, x = 2) or more, the power value is the threshold value P or more. The frequency band in which the frequency is equal to or lower than the threshold value F is determined as the used band α. The reason for this is that when the decomposition rank n of the wavelet decomposition in the reduced image generation unit 10 becomes large, the reduced image is distorted by folding back. In this case, if the high frequency component is used as the band when the restored reduced image is decomposed into the nth-order wavelet again and the super-resolution processing is performed on the decoding side, the image may be deteriorated.

Further, in the third used band determining method, in the second used band determining method, the larger the decomposition rank n, the smaller the threshold value F. The larger the decomposition order, the lower the frequency band of the generated image. Therefore, specifically, the band to be used may be determined from the high frequency component image of the (n + 1) order or higher. ^{For example, when the decomposition order m = 5 and the RMS values of LH 3} and HL ⁴ are equal to or higher than the threshold value P as illustrated in the second method for determining the band to be used, and n = 3, the high frequency component image LH ^{The frequency bands of 4} , HL ⁴ , and HH ⁴ are determined as the used band α.

The entropy coding unit 50 performs entropy coding on the quantization coefficient input from the quantization unit 24, the prediction information input from the prediction unit 25, and the usage band information input from the usage band determination unit 40. , Data compression is performed to generate a bit stream, and the data is output to the outside of the encoding device 1.

Here, a computer can be preferably used to function as the coding device 1 described above, and such a computer stores a program describing processing contents for realizing each function of the coding device 1 in the computer. It can be realized by storing it in a unit and reading and executing this program by the CPU of the computer. This program can be recorded on a computer-readable recording medium.

As described above, the coding apparatus and its program according to the present invention generate a reduced image obtained by reducing the input image, analyze the restored reduced image obtained by restoring the reduced image, and perform super-resolution processing on the decoding side. Determine the band to use. As a result, it is not necessary to separately transmit extra information (texture parameters of high frequency components, etc.) to the decoding side, and it is possible to improve the coding efficiency.

(Decoding device)
Next, the decoding device will be described. FIG. 3 is a block diagram showing a configuration example of the decoding device 3 according to the second embodiment of the present invention.

The decoding device 3 acquires a bit stream from the coding device 1, decodes the reduced reduced image, and then performs super-resolution processing to generate a super-resolution image having a resolution higher than that of the reduced image. In the example shown in FIG. 3, the decoding device 3 includes an entropy decoding unit (acquisition unit) 60, a reduced image restoration unit 70, and a super-resolution restoration unit 80.

The entropy decoding unit 60 pairs with the entropy coding unit 50 of the coding device 1, decodes the bit stream encoded by the entropy coding unit 50, and obtains the quantization coefficient of the reduced image and information necessary for prediction processing. The prediction information, which is, and the used band information, which is the information necessary for the super-resolution processing, are acquired. Then, the quantization coefficient is output to the inverse quantization unit 71, the prediction information is output to the prediction unit 75, and the used band information is output to the super-resolution restoration unit 80.

The reduced image restoration unit 70 inputs the quantization coefficient of the reduced image decoded by the entropy decoding unit 60, performs dequantization processing to restore the reduced image, and outputs the restored reduced image to the super-resolution restoration unit 80. To do. In the example shown in FIG. 3, the reduced image restoration unit 70 includes an inverse quantization unit 71, an inverse conversion unit 72, an addition unit 73, a storage unit 74, and a prediction unit 75.

The inverse quantization unit 71 multiplies the quantization coefficient input from the entropy decoding unit 60 by the quantization step to restore the conversion coefficient for each block, and outputs the conversion coefficient to the inverse conversion unit 72. The quantization step may be acquired from the coding device 1, or may have a common quantization table with the coding device 1 in advance.

The inverse transformation unit 72 performs inverse transformation on the conversion coefficient input from the inverse quantization unit 71 to restore the residual block image, and outputs the residual block image to the addition unit 73. The inverse transformation is the same process as the inverse transformation unit 32 of the coding apparatus 1.

The addition unit 73 adds the residual block image input from the inverse conversion unit 72 and each pixel value of the prediction image input from the prediction unit 75, generates a restored reduced image in block units, and stores the storage unit 74. , And output to the super-resolution restoration unit 80.

The storage unit 74 stores the decoding block image input from the addition unit 73.

The prediction unit 75 refers to the restored and reduced image stored in the storage unit 74, performs prediction processing according to the prediction information input from the entropy decoding unit 60, generates a prediction image, and outputs the prediction image to the addition unit 73.

The super-resolution restoration unit 80 performs frequency decomposition processing on the reduced image (restored and reduced image) restored by the reduced image restoration unit 70. At that time, a high-frequency component image of the used band α indicated by the used band information decoded by the entropy decoding unit 60 is generated, and the high-frequency component image is assigned to the high-frequency component of the restored reduced image to generate a super-resolution image. , Output to the outside of the decoding device 3.

FIG. 4 is a block diagram showing a configuration example of the super-resolution restoration unit 80. In the example shown in FIG. 4, the super-resolution restoration unit 80 includes a frequency decomposition unit 81, an alignment unit 82, a super-resolution high-frequency generation / division attachment unit 83, and a frequency reconstruction unit 84.

The frequency decomposition unit 81 acquires the used band α from the entropy decoding unit 60. Then, the restored reduced image is subjected to frequency decomposition (multi-resolution decomposition) processing to generate a frequency-resolved image including the used band α, and is output to the alignment unit 82. The frequency decomposition unit 81 can perform a wavelet decomposition process as the frequency decomposition process. In the following description, it is assumed that α indicates the decomposition rank. The frequency-resolved image includes a low-frequency component image LL ^α of the reduced image, a horizontal high-frequency component image LH ^α which is a high-frequency component image of the reduced image, a vertical high-frequency component image HL ^α , and a diagonal high-frequency component image HH ^α .

The alignment unit 82 generates alignment information (registration information) indicating the corresponding positional relationship between the restored reduced image and the low-frequency component image LL ^α, and outputs the alignment information (registration information) to the super-resolution high-frequency segmentation unit 83.

FIG. 5 is a diagram showing an outline of the alignment process in the alignment unit 82. The alignment unit 82 performs block matching between both frames, for example, using the restored and reduced image as a reference frame and the low-frequency component image LL ^α as a reference frame, and the block having the highest degree of similarity (correlation) within the search range. Generates alignment information that indicates the positional relationship. Although α = 1 in FIG. 5, α may be plural. Block matching is performed by a known method using evaluation functions such as sum of absolute error (SAD) and sum of squared error (SSD). Further, the block matching is performed with a decimal pixel accuracy by, for example, interpolation processing using the parabolic fitting function shown in the equation (1). If the evaluation function value of SAD or SSD exceeds the threshold value, it may not be adopted as the alignment information.

Here, when the pixel position at the search position is x, the SSD (x) represents the SSD value at the pixel position, and more specifically, the SSD (0) is the center position (the position where the SSD value is minimized). SSD (-1) is the SSD value at the position of the adjacent pixel in the −x direction or −y direction from the center position, and SSD (1) is the SSD value at the position of the adjacent pixel in the + x direction or + y direction from the center position. Represents. From the equation (1), the pixel positions (decimal pixel positions) with the decimal pixel accuracy in the horizontal or vertical direction can be calculated respectively.

^{The super-resolution high-frequency segmentation section 83 uses the high-frequency component images LH α} and HL ^α generated by the frequency-resolving section 81 as high-frequency components of the super-resolution high-frequency component image according to the alignment information generated by the alignment section 82. , HH ^α is assigned to generate a super-resolution high-frequency component image, which is output to the frequency reconstruction unit 84.

FIG. 6 is a diagram showing an outline of processing of the super-resolution high-frequency segmentation section 83. The super-resolution high-frequency segmentation section 83 converts the high-frequency component images LH ^α , HL ^α , and HH ^α into super-resolution horizontal component images, super-resolution vertical component images, and super-resolution diagonal component images according to the alignment information. Allocate to. Here, when allocating the high-frequency component images LH ^α , HL ^α , and HH ^α , it is assumed that the alignment information of the same phase position in the ^{low-frequency component image LL α is followed.} This means that if block A in the restored reduced image ^{is similar to block B in the low frequency component image LL α} , then an unknown super-resolution horizontal component image, super-resolution vertical component image, and super-resolution diagonal component. This is because the block having the same phase position as the block A in the image is likely to be similar to the block having the same phase position as the block B in the ^{high frequency components LH α} , HL ^α , and HH ^α.

When there are multiple candidates to be assigned as horizontal, vertical, and diagonal super-resolution components, the super-resolution high-frequency segmentation section 83 averages those values or re-maximum a posteriori (MAP). Configure and estimate unknown values. For more information on MAP reconstruction, see, for example, E. Levitan and G. Herman: “A maximum a posteriori probability expectation maximization algorithm for image reconstruction in emission tomography”, IEEE Transactions on Medical Imaging, vol. 6, no. 3, pp. . 185-192, Sep. 1987. Further, as another method, the super-resolution high-frequency component image may be estimated by the ML method or the weighting according to the distance of the allocated pixels.

The frequency reconstruction unit 84 uses the restored and reduced image generated by the reduced image restoration unit 70 as a low frequency component, and the super-resolution high frequency component image allocated by the super-resolution high-frequency segmentation unit 83 as a high-frequency component, and frequency-reconstructs the frequency. The composition process is performed to generate a super-resolution image, which is output to the outside. When the frequency decomposition unit 81 performs the wavelet decomposition processing as the frequency decomposition processing, the frequency reconstruction unit 84 performs the wavelet reconstruction processing using the same wavelet.

FIG. 7 is a diagram showing an outline of processing of the frequency reconstruction unit 84. When the first-order wavelet decomposition processing is performed by the encoding device, the frequency reconstruction unit 84 performs the first-order wavelet reconstruction processing as shown in FIG. 7 to perform the first-order wavelet reconstruction processing in the horizontal direction with respect to the reduced image. The number of pixels is doubled and the number of pixels in the vertical direction is doubled to generate a super-resolution image having the same size (resolution) as the input image input to the encoding device 1.

Here, a computer can be preferably used to function as the decoding device 3 described above, and such a computer stores a program describing processing contents for realizing each function of the decoding device 3 in the storage unit of the computer. This can be realized by storing the program and reading and executing this program by the CPU of the computer. This program can be recorded on a computer-readable recording medium.

As described above, the decoding device and its program according to the present invention acquire the used band information used in the super-resolution processing from the coding side, perform frequency decomposition processing on the restored reduced image, and perform high frequency based on the used band information. A super-resolution image is generated by generating a component image and allocating the high-frequency component image to the high-frequency component of the restored and reduced image. As a result, it is possible to perform super-resolution restoration of high-frequency components lost by the reduction processing and coding processing on the coding side with high image quality without preparing a database of high-frequency components on the decoding side. Further, by setting the reduction ratio on the coding side and the enlargement ratio on the decoding side as reciprocals, it is possible to restore a super-resolution image to which a high-frequency component is added without changing the resolution of the input image on the coding side.

(Modification example)
Next, a modification of the above-mentioned coding device 1 and decoding device 3 will be described. In the above-described embodiment, the used band α is determined on the coding side, but in the modified example, the used band α is determined on the decoding side.

FIG. 8 is a block diagram showing a configuration example of the coding device 2 which is a modification of the coding device 1. FIG. 9 is a block diagram showing a configuration example of the decoding device 4, which is a modification of the decoding device 3. Instead of the coding device 2 not including the used band determining unit 40, the decoding device 4 includes the used band determining unit 40.

The decoding device 4 acquires a bit stream from the coding device 2, decodes the reduced reduced image, and then performs super-resolution processing to generate a super-resolution image having a resolution higher than that of the reduced image.

Similar to the decoding device 3, the reduced image restoration unit 70 of the decoding device 4 inputs a bit stream of the reduced image generated by the coding device 2 and decodes the reduced image to restore the reduced image. Then, the restored reduced image is output to the used band determination unit 40 and the super-resolution restoration unit 80.

The band used determination unit 40 of the decoding device 4 performs frequency decomposition processing on the restored and reduced image restored by the reduced image restoration unit 70, and calculates the power value (for example, RMS value) of the high frequency component image. Then, the band α used is determined based on the power value, and the band α is output to the super-resolution restoration unit 80. Since the method of determining the used band α is the same as that of the used band determining unit 40 in the coding apparatus 1 described above, the description thereof will be omitted.

The super-resolution restoration unit 80 of the decoding device 4 uses the restored and reduced image restored by the reduced image restoration unit 70 and the used band α determined by the used band determining unit 40 in the same manner as the decoding device 3. Generate a super-resolution image.

Here, a computer can be preferably used to function as the coding device 2 or the decoding device 4 described above, and such a computer can perform processing contents for realizing each function of the coding device 2 or the decoding device 4. It can be realized by storing the described program in the storage unit of the computer and reading and executing this program by the CPU of the computer. This program can be recorded on a computer-readable recording medium.

In the modified example, even if the encoding side does not output the used bandwidth information, the decoding side analyzes the restored and reduced image to determine the used bandwidth used for the super-resolution processing, thereby realizing highly accurate super-resolution restoration. can do.

Although the above embodiments have been described as typical examples, it will be apparent to those skilled in the art that many modifications and substitutions can be made within the spirit and scope of the present invention. Therefore, the present invention should not be construed as being limited by the above-described embodiments, and various modifications and modifications can be made without departing from the scope of claims. For example, it is possible to combine a plurality of constituent blocks described in the configuration diagram of the embodiment into one, or to divide one constituent block.

The program may also be recorded on a computer-readable medium. It can be installed on a computer using a computer-readable medium. Here, the computer-readable medium on which the program is recorded may be a non-transient recording medium. The non-transient recording medium is not particularly limited, but may be, for example, a recording medium such as a CD-ROM or a DVD-ROM.

1, 2, Encoding device 3, 4 Decoding device 10 Reduced image generation unit 20 Compression unit 21 Block division unit 22 Subtraction unit 23 Conversion unit 24 Quantization unit 25 Prediction unit 30 Decompression unit 31 Inverse quantization unit 32 Inverse conversion unit 33 Addition Unit 34 Storage unit 40 Bandwidth determination unit 50 Entropy coding unit 60 Entropy decoding unit (acquisition unit)
70 Reduced Photo Recovery Unit 71 Inverse Quantization Unit 72 Inverse Transformation Unit 73 Addition Unit 74 Storage Unit 75 Prediction Unit 80 Super-Resolution Restoration Unit 81 Frequency Decomposition Unit 82 Alignment Unit 83 Super-Resolution High Frequency Generation Division 84 Frequency Reconstruction Unit

Claims (9)

A coding device that encodes an input image after it has been reduced.
A reduced image generator that generates a reduced image by performing frequency decomposition processing on the input image,
A compression unit that performs quantization processing on the reduced image to generate a quantization coefficient,
A decompression unit that restores a reduced image by performing decompression processing on the quantization coefficient and outputs it as a restored reduced image.
A frequency band determining unit that generates a high frequency component image by performing frequency decomposition processing on the restored reduced image and determines a frequency band in which the power value is equal to or higher than the first threshold value as a used band among the high frequency component images.
A coding unit that generates a bit stream that encodes the quantization coefficient and the used band information indicating the used band, and
A coding device comprising. The reduced image generation unit generates a low-frequency component image by performing n-th order wavelet decomposition processing on the input image, and digitizes each pixel value of the low-frequency component image to obtain the reduced image.
The coding apparatus according to claim 1, wherein the band used determining unit performs m-th order wavelet decomposition processing on the restored and reduced image to generate the high-frequency component image. When the value of n is equal to or greater than a predetermined value, the band used determining unit determines that the power value is equal to or greater than the first threshold value and the frequency is equal to or less than the second threshold value in the high frequency component image. The coding apparatus according to claim 2, wherein the frequency band is determined as the band to be used. The coding apparatus according to claim 3, wherein the used band determining unit reduces the second threshold value as the value of n increases. A decoding device that performs super-resolution processing after restoring a reduced image.
An acquisition unit that acquires the quantization coefficient of the reduced image and the used band information indicating the used band used in the super-resolution processing.
A reduced image restoration unit that reverse-quantizes the quantization coefficient to restore the reduced image and outputs it as a restored reduced image.
The restored reduced image is subjected to frequency decomposition processing to generate a high frequency component image of the band used, and the high frequency component image is assigned to the high frequency component of the restored reduced image to generate a super resolution image. Restoration part and
A decoding device comprising. A decoding device that performs super-resolution processing after restoring a reduced image.
An acquisition unit that acquires the quantization coefficient of the reduced image, and
A reduced image restoration unit that reverse-quantizes the quantization coefficient to restore the reduced image and outputs it as a restored reduced image.
A frequency band determining unit that generates a high frequency component image by performing frequency decomposition processing on the restored reduced image and determines a frequency band in which the power value is equal to or higher than the first threshold value as a used band among the high frequency component images.
A super-resolution restoration unit that generates a super-resolution image by frequency-resolving the restored and reduced image to generate a high-frequency component image of the band used, and allocating the high-frequency component image to the high-frequency component of the reduced image. ,
A decoding device characterized by the above. The super-resolution restoration unit
A frequency decomposition unit that performs wavelet decomposition processing on the restored reduced image to generate a frequency decomposition image including the band used.
An alignment unit that generates alignment information indicating the positional relationship of similar blocks between the restored reduced image and the low-frequency component image of the frequency-resolved image.
According to the alignment information, as the high-frequency component of the restored / reduced image, the high-frequency component image of the frequency-resolved image is assigned to generate a super-resolution high-frequency component image.
A frequency reconstruction unit that generates the super-resolution image by frequency-reconstructing the restored and reduced image as a low-frequency component and the super-resolution high-frequency component image as a high-frequency component.
The decoding apparatus according to claim 5 or 6, wherein the decoding apparatus is provided with. A program for causing a computer to function as the coding device according to any one of claims 1 to 4. A program for causing a computer to function as the decoding device according to any one of claims 5 to 7.

Images