JP5965760B2 - Image encoding device, image decoding device, and program thereof - Google Patents

Description

The present invention relates to an image coding apparatus, an image decoding apparatus Contact and those programs.

  Conventionally, on the encoding side, an original image is reduced and then encoded to generate encoded data. On the decoding side, the encoded data is decoded and then super-resolution is performed. Yes (see Non-Patent Document 1). Here, in the super-resolution used, first, a horizontal high-frequency component H, a vertical high-frequency component V, and a diagonal high-frequency component D obtained by wavelet first-order decomposition of an input image (here, an image obtained by decoding encoded data). Are regarded as low-frequency components, and wavelet reconstruction is performed separately to generate a horizontal high-frequency component H ′, a vertical high-frequency component V ′, and a diagonal high-frequency component D ′. Then, the input image is regarded as a low-frequency component, and the wavelet is obtained using the input image, the horizontal high-frequency component H ′ obtained by the previous wavelet reconstruction, the vertical high-frequency component V ′, and the diagonal high-frequency component D ′. A super-resolution image is obtained by reconstruction.

Y. Matsuo, T .; Misu, S .; Sakaida, Y. et al. Shikikui, “Video Coding with Wavelet Image Size Reduction and Wavelet Super Resolution Reconstruction”, Proceedings of 18th IEEE Ice International IC. PG. 6, p. 1181-1184, 2011

  However, in the super-resolution in Non-Patent Document 1 described above, there is a problem that when an image with a large amount of noise components is super-resolution, an image with good image quality may not be obtained.

  The present invention has been made in view of such circumstances, and an object of the present invention is to provide an image encoding device, an image decoding device, and an image decoding device that can obtain an image with good image quality when an image having a large noise component is processed. It is to provide a super-resolution device and a program thereof.

(1) The present invention has been made to solve the above-described problems, and one aspect of the present invention is an image reduction unit that reduces an input image and generates a reduced image, and a code that encodes the reduced image. A decoding unit that decodes the encoded reduced image, each pixel value of the decoded reduced image is separated into a noise component and a signal component, and a first image composed of the signal component; A noise / signal separation unit that generates a second image composed of the noise component; a signal component super-resolution unit that generates a first super-resolution image by super-resolution of the first image; A noise component super-resolution unit that generates a second super-resolution image by super-resolution of the second image by a method different from the signal component super-resolution unit; and the first super-resolution An adder that adds an image and the second super-resolution image to obtain a super-resolution image of the decoded reduced image, and the decoding A super-resolution for comparing a super-resolution image of a reduced image with the input image and determining a super-resolution parameter for at least one of the signal component super-resolution unit and the noise component super-resolution unit. An image encoding apparatus comprising: an image parameter determination unit; an encoded data generation unit configured to generate encoded data including the encoded reduced image and the determined super-resolution parameter. .

(2) According to another aspect of the present invention, there is provided the image encoding device according to (1), wherein at least one of the signal component super resolving unit and the noise component super resolving unit is the super encoder. When generating a resolution, filtering by a bilateral filter is performed, and the super-resolution parameter includes a dispersion value of the bilateral filter.

(3) According to another aspect of the present invention, there is provided the image encoding device according to (1), wherein the noise / signal separation unit performs first-time decomposition of the decoded reduced image on a space-time wavelet. A spatial wavelet first-order decomposition unit and a spatio-temporal wavelet first-order decomposition result, a spatio-temporal high-frequency component is referred to, a pixel including noise is determined, and a noise component is determined from the spatio-temporal wavelet first-order decomposition result. A noise component separation unit for separating, a noise component reconstruction unit for generating the second image by reconstructing a first wavelet of the separated noise component, and the noise component separation unit separating the noise component And a signal component reconstructing unit configured to reconstruct wavelet first-order the remaining signal components and generate the first image.

(4) According to another aspect of the present invention, there is provided the image encoding device according to (1), wherein the noise component super-resolution unit has twice as many pixels as the reduced image in a vertical direction and a horizontal direction. The second image is arranged in a low-frequency component region, a horizontal high-frequency component region, a vertical high-frequency component region, and a diagonal high-frequency component region when a wavelet first-order wavelet decomposition is performed. And a wavelet first-order reconstruction unit that reconstructs a wavelet first-order from the arrangement result of the noise component arrangement unit and generates the second super-resolution image. To do.

(5) According to another aspect of the present invention, a data separation unit that extracts an encoded image and super-resolution parameters from encoded data, and a decoding that decodes the encoded image and generates a reduced image. And a noise / signal for separating each pixel value of the reduced image into a noise component and a signal component, and generating a first image composed of the signal component and a second image composed of the noise component What is a separation unit, a signal component super-resolution unit that super-resolutions the first image to generate a first super-resolution image, and the second image, the signal component super-resolution unit Adding a noise component super-resolution unit that generates a second super-resolution image by super-resolution using a different method, the first super-resolution image, and the second super-resolution image An addition unit for obtaining a super-resolution image of the reduced image, and at least of the signal component super-resolution unit and the noise component super-resolution unit One is an image decoding apparatus, characterized in that the super-resolution by using the super-resolution parameter.

(6) According to another aspect of the present invention, there is provided the image decoding device according to (5), wherein at least one of the signal component super-resolution unit and the noise component super-resolution unit is the super-resolution. When generating an image, filtering by a bilateral filter is performed, and the super-resolution parameter includes a dispersion value of the bilateral filter.

(7) According to another aspect of the present invention, there is provided the image decoding apparatus according to (5), in which the noise / signal separation unit performs space-time wavelet first-order decomposition on the decoded reduced image. Among the results of the wavelet first-order decomposition unit and the spatio-temporal wavelet first-order decomposition, reference is made to spatio-temporal high-frequency components to determine pixels that contain noise, and the noise components are separated from the results of the spatio-temporal wavelet first-order decomposition. A noise component separation unit that performs wavelet first-order reconstruction on the separated noise component, and generates a second image, and a remaining noise component separation unit that separates the noise component. And a signal component reconstructing unit that reconstructs wavelet first-order signals to generate the first image.

(8) According to another aspect of the present invention, in the image decoding device according to (5), the noise component super resolving unit has a number of pixels twice as large as the reduced image in the vertical direction and the horizontal direction. The second image is arranged in a low-frequency component region, a horizontal high-frequency component region, a vertical high-frequency component region, and a diagonal high-frequency component region when a wavelet is first-order decomposed. A noise component placement unit; and a wavelet first-order reconstruction unit that reconstructs a wavelet first-order result of the placement by the noise component placement unit to generate the second super-resolution image. .

(9) Moreover, the other aspect of this invention is a program for functioning a computer as an image coding apparatus in any one of (1) to (4).

(10) Another aspect of the present invention is a program for causing a computer to function as the image decoding device according to any one of (5) to (8).

(A1) According to another aspect of the present invention, each pixel value of an input image is separated into a noise component and a signal component, and a first image composed of the signal component and a second composed of the noise component. A noise / signal separation unit that generates the first image, a signal component super-resolution unit that generates the first super-resolution image by super-resolution of the first image, and the second image, A noise component super-resolution unit that generates a second super-resolution image by super-resolution using a method different from the signal component super-resolution unit, the first super-resolution image, and the second super-resolution image A super-resolution apparatus comprising: an adding unit that adds a super-resolution image to obtain a super-resolution image of the input image.

(A2) According to another aspect of the present invention, there is provided the super-resolution device according to (a1), wherein the noise / signal separation unit is a space-time wavelet that decomposes the input image into a space-time wavelet first-order decomposition. A first-order decomposition unit and a spatio-temporal wavelet first-order decomposition result, a spatio-temporal high-frequency component is referred to determine a pixel including noise, and a noise component is separated from the spatio-temporal wavelet first-order decomposition result A noise component separation unit, a noise component reconstruction unit that reconstructs the separated noise component into a wavelet first order and generates the second image, and a remaining noise component separated by the noise component separation unit. And a signal component reconstructing unit configured to reconstruct a wavelet first-order signal component and generate the first image.

(A3) According to another aspect of the present invention, there is provided the super-resolution device according to (a1), wherein the noise component super-resolution unit has twice as many pixels as the input image in a vertical direction and a horizontal direction. The second image is arranged in a low-frequency component region, a horizontal high-frequency component region, a vertical high-frequency component region, and a diagonal high-frequency component region when a wavelet first-order wavelet decomposition is performed. And a wavelet first-order reconstruction unit that reconstructs a wavelet first-order from the arrangement result of the arrangement unit and generates the second super-resolution image.

(A4) According to another aspect of the present invention, there is provided the super-resolution apparatus according to (a3), wherein the noise component super-resolution unit is obtained by the reconfiguration unit reconfiguring the wavelet first order. And a pixel value normalizing unit that normalizes the pixel value of the image with the noise level of the input image to obtain the second super-resolution image.

(A5) According to another aspect of the present invention, there is provided the super-resolution apparatus according to (a3), wherein the noise component super-resolution unit is obtained by the reconfiguration unit reconfiguring the wavelet first order. A white noise applying unit that replaces a pixel value of a pixel having a pixel value of 0 with a random value having a magnitude corresponding to the noise level as the second super-resolution image. It is characterized by comprising.

(B1) Further, according to another aspect of the present invention, a low-frequency component region and a horizontal high-frequency component region obtained when wavelet first-order decomposition is performed on an image having twice as many pixels as the input image in the vertical and horizontal directions. And an arrangement unit that arranges the input image in each of the vertical high-frequency component region and the diagonal high-frequency component region, and the arrangement result by the arrangement unit is reconstructed by wavelet first floor, A super-resolution apparatus comprising a wavelet first-order reconstruction unit that generates a resolution image.

(B2) According to another aspect of the present invention, there is provided a super-resolution device as described above, wherein a noise level extraction unit that extracts a noise level of the input image, and a pixel value of the super-resolution image are converted into the noise. And a pixel value normalization unit that normalizes by level.

(B3) According to another aspect of the present invention, there is provided the super-resolution apparatus described above, wherein the input image is subjected to space-time wavelet first-order decomposition to calculate a space-time high-frequency component. The noise level extraction unit extracts an isolated point in the spatio-temporal high-frequency component, and calculates the noise level using an element value of the isolated point.

(B4) According to another aspect of the present invention, there is provided any one of the above-described super-resolution devices, the pixels constituting the super-resolution image generated by the wavelet first-order reconstruction unit, the pixel value And a white noise applying unit that sets a pixel value of a pixel having a value of 0 as a random value having a magnitude corresponding to the noise level.

(B5) Another aspect of the present invention is a program for causing a computer to function as the super-resolution device according to any one of (a1) to (a5) and (b1) to (b4).

  According to the present invention, it is possible to obtain a super-resolution image with good image quality when super-resolution of an image with many noise components.

1 is a schematic block diagram illustrating a configuration of an image transmission system 10 according to a first embodiment of the present invention. It is a schematic block diagram which shows the structure of the image coding apparatus 100 in the embodiment. It is a schematic block diagram which shows the structure of the image reduction part 101 in the embodiment. It is a figure explaining operation | movement of the low frequency component extraction part 111 in the embodiment. It is a schematic block diagram which shows the structure of the noise and signal separation part 105 in the embodiment. It is a figure explaining the calculation result by the spatio-temporal wavelet first-order decomposition | disassembly part 150 in the embodiment. It is a figure explaining the surrounding element referred in the noise level extraction part 151 in the embodiment. It is a flowchart explaining operation | movement of the noise component separation part 153 in the same embodiment. It is a schematic block diagram which shows the structure of the signal component super-resolution part 106 in the embodiment. It is a conceptual diagram explaining the process of the extended LH component production | generation part 161a in the embodiment. It is a figure which shows arrangement | positioning of each component in the wavelet 1st floor reconstruction part 163 in the embodiment. It is a schematic block diagram which shows the structure of the noise component super-resolution part 107 in the embodiment. It is a figure explaining arrangement | positioning of the noise component image by the noise component rearrangement part 170 in the same embodiment. It is a schematic block diagram which shows the structure of the image decoding apparatus 300 in the embodiment. It is a schematic block diagram which shows the structure of the super-resolution apparatus 500 by 2nd Embodiment of this invention. It is a schematic block diagram which shows the structure of the super-resolution apparatus 600 by 3rd Embodiment of this invention.

[First Embodiment]
Hereinafter, a first embodiment of the present invention will be described with reference to the drawings. FIG. 1 is a schematic block diagram showing the configuration of an image transmission system 10 according to the first embodiment of the present invention. As shown in FIG. 1, the image transmission system 10 includes an image encoding device 100, a network 200, an image decoding device 300, and an image display device 400. The image encoding device 100 encodes an input moving image and generates encoded data. The network 200 is a network such as a packet switching network that connects the image encoding device 100 and the image decoding device 300 so that they can communicate with each other. The network 200 may be a network using broadcast waves such as terrestrial digital broadcasting. The network 200 transmits the encoded data generated by the image encoding device 100 to the image decoding device 300. The image decoding device 300 decodes the encoded data transmitted by the network 200 to generate a decoded moving image. The image display device 400 displays the decoded moving image generated by the image decoding device 300.

  FIG. 2 is a schematic block diagram illustrating a configuration of the image encoding device 100. The image encoding apparatus 100 includes an image reduction unit 101, an encoding unit 102, an encoded data generation unit 103, a decoding unit 104, a noise / signal separation unit 105, a signal component super resolving unit 106, and a noise component super resolving unit 107. , And an addition unit 108 and a super-resolution parameter determination unit 109. The image reduction unit 101 reduces each frame of the input moving image to generate a reduced moving image. The reduction rate of the reduction is, for example, 1/2 in both the horizontal direction and the vertical direction. That is, the number of pixels in the horizontal direction and the vertical direction in each frame of the reduced moving image is half that of the input moving image.

  The encoding unit 102 encodes the reduced moving image generated by the image reducing unit 101 to generate encoded image data. For encoding in the encoding unit 102, MPEG (Moving Picture Experts Group) -4 AVC (Advanced Video Coding) / H. H.264, HEVC (High Efficiency Video Coding) and other existing and standardized moving picture coding schemes can be used.

The encoded data generation unit 103 includes the encoded image data generated by the encoding unit 102, the super-resolution parameter determined by the super-resolution parameter determination unit 109, and the noise level L _N generated by the noise / signal separation unit 105. To generate encoded data. The encoded data generation unit 103 generates encoded data by storing the noise level L _N and the super-resolution parameter in, for example, an unused area in the encoded image data format. The super-resolution parameter in the present embodiment is a parameter related to super-resolution in each of the signal component super-resolution unit 106 and the noise component super-resolution unit 107, and details thereof will be described later. Note that the super-resolution parameter in the present embodiment is a parameter related to super-resolution in each of the signal component super-resolution unit 106 and the noise component super-resolution unit 107. Only the parameter relating to the super-resolution of one of the resolution units 107 may be used (when the other is a fixed value).

  The decoding unit 104 decodes the encoded image data generated by the encoding unit 102 and generates a decoded reduced moving image. Decoding in the decoding unit 104 is performed by a method corresponding to the moving image encoding method in the encoding unit 102. Note that the decoded reduced moving image is obtained by encoding and decoding the reduced moving image generated by the image reducing unit 101. Therefore, the number of pixels in the horizontal direction and the vertical direction in each frame is the same as that of the reduced moving image.

The noise / signal separation unit 105 separates each pixel value of the decoded reduced moving image into a signal component and a noise component, and a signal component image (first image) including the signal component and a noise component image including the noise component. (Second image). Note that the noise / signal separation unit 105 detects a noise level L _N indicating the magnitude of noise included in the decoded reduced moving image when performing this separation process, and performs the separation process using the noise level. . Also, the noise / signal separation unit 105 inputs this noise level _LN to the encoded data generation unit 103.

  The signal component super-resolution unit 106 super-resolutions the signal component image to generate a signal component super-resolution image (first super-resolution image). The signal component super-resolution unit 106 performs this super-resolution using various values of super-resolution parameters, and generates a signal component super-resolution image corresponding to each value. In the present embodiment, the signal component super resolving unit 106 uses a super resolving method using self-similarity on the premise that the image has self-similarity.

The noise component super-resolution unit 107 super-resolutions the noise component image by a method different from that of the signal component super-resolution unit 106 to generate a noise component super-resolution image. The noise component super resolving unit 107 uses a method different from the signal component super resolving unit 106 because the noise component image has low self-similarity, and therefore the super resolving method of the signal component super resolving unit 106 is applied. This is because good results cannot be obtained. The noise component super resolving unit 107 uses the noise level L _N detected by the noise / signal separating unit 105 during the super resolving. The noise component super-resolution unit 107 also performs this super-resolution using various values of super-resolution parameters, and generates a noise component super-resolution image corresponding to each value. The super-resolution parameter will be described later.

  The adder 108 adds the signal component super-resolution image generated by the signal component super-resolution unit 106 and the noise component super-resolution image generated by the noise component super-resolution unit 107. Here, the addition of an image and an image indicates generation of an image obtained by adding pixel values of pixels at the same position in these images. Note that the adding unit 108 generates signal component super-resolution images corresponding to various values of super-resolution parameters generated by the signal component super-resolution unit 106 and various components generated by the noise component super-resolution unit 107. The addition of the noise component super-resolution image corresponding to the value super-resolution parameter is performed for all combinations. For example, when there are 27 signal component super-resolution images and 3 noise component super-resolution images, there are 27 × 3 = 81 combinations of these, so the adding unit 108 performs 81 additions. Do.

  The super-resolution parameter determination unit 109 (super-resolution parameter determination unit) compares each image generated by the addition unit 108 with the corresponding frame of the input moving image, and the image closest to the corresponding frame of the input moving image. Select. Then, the super-resolution parameter determination unit 109 notifies the encoded data generation unit 103 of the super-resolution parameter used to generate the selected image. For example, the selected image is generated by the signal component super-resolution unit 106 using the super-resolution parameter Sg1 and the noise component super-resolution unit 107 using the super-resolution parameter Ns1. If the addition unit 108 adds the noise component super-resolution image that has been added, the super-resolution parameter determination unit 109 outputs the super-resolution parameter Sg1 and the super-resolution parameter Ns1 as an encoded data generation unit. 103 is notified.

  The super-resolution parameter determination unit 109 uses, for example, SAD (Sum of Absolute Difference) when selecting an image closest to the corresponding frame of the input moving image. Specifically, the super-resolution parameter determination unit 109 calculates the SAD with the corresponding frame of the input moving image for each of the images generated by the addition unit 108 and adds the image with the smallest SAD to the frame. Select as the closest image. Note that SAD is the total value of the absolute values of the differences between corresponding pixel values calculated over the entire image, and is calculated by the following equation (1). In equation (1), vi (x, y) is the pixel value of the coordinates (x, y) of the corresponding frame of the input moving image, and sr (x, y) is the image generated by the adding unit 108. Is the pixel value of the coordinates (x, y).

  FIG. 3 is a schematic block diagram illustrating the configuration of the image reduction unit 101. The image reduction unit 101 includes a wavelet first-order decomposition unit 110 and a low-frequency component extraction unit 111. The wavelet first-order decomposition unit 110 performs discrete wavelet first-order decomposition on each frame of the input moving image. By this discrete wavelet first-order decomposition, a low frequency component LL, a horizontal high frequency component LH, a vertical high frequency component HL, and a diagonal high frequency component HH of each frame are obtained. Any base such as Haar or Daubechies may be used as the base used for the discrete wavelet first-order decomposition.

  The low frequency component extraction unit 111 generates a low frequency component from the low frequency component LL, horizontal high frequency component LH, vertical high frequency component HL, and diagonal high frequency component HH of each frame obtained by the discrete wavelet first order decomposition of the wavelet first order decomposition unit 110. The frequency component LL is extracted. The low-frequency component extraction unit 111 outputs a low-frequency component LL corresponding to each frame of the input moving image in the order of frames, thereby reducing the moving image. As described above, since each frame of the reduced moving image is the low frequency component LL of each frame of the input moving image, the number of pixels in the horizontal direction and the vertical direction in each frame of the reduced moving image is input as described above. It is half that of a moving image.

  FIG. 4 is a diagram for explaining the operation of the low-frequency component extraction unit 111. As shown in FIG. 4, each frame of the input moving image is decomposed by the wavelet first-order decomposition unit 110 into a low frequency component LL, a horizontal high frequency component LH, a vertical high frequency component HL, and a diagonal high frequency component HH. Of these, the low frequency component extraction unit 111 extracts the low frequency component LL shaded in FIG.

  FIG. 5 is a schematic block diagram showing the configuration of the noise / signal separation unit 105. As shown in FIG. 5, the noise / signal separation unit 105 includes a spatio-temporal wavelet first-order decomposition unit 150, a noise level extraction unit 151, a mask image generation unit 152, a noise component separation unit 153, a noise component reconstruction unit 154, a signal component. A reconfiguration unit 155 is included. The spatio-temporal wavelet first-order decomposition unit 150 decomposes the decoded reduced image into the first-order discrete space-time wavelet for each predetermined number of frames. The spatio-temporal wavelet first-order decomposition unit 150 performs low-frequency component LLL, temporal high-frequency component LLH, horizontal high-frequency component LHL, vertical high-frequency component HLL, temporal horizontal high-frequency component LHH, and time-vertical high-frequency component by this discrete spatio-temporal wavelet first-order decomposition. HLH, horizontal vertical high-frequency component HHL, and diagonal high-frequency component HHH (space-time high-frequency component) are generated.

  At this time, the space-time wavelet first-order decomposition unit 150 performs space-time wavelet first-order decomposition without decimation. Therefore, each component such as the diagonal high-frequency component HHH is composed of a number of elements obtained by multiplying the number of pixels of one frame of the decoded reduced image by a predetermined number of frames when performing the discrete space-time wavelet first-order decomposition. Any base such as Haar or Daubechies may be used as the base used for the first-order decomposition of the discrete space-time wavelet.

  FIG. 6 is a diagram for explaining a calculation result by the spatiotemporal wavelet first-order decomposition unit 150. In FIG. 6, the horizontal axis is the x-axis and the vertical axis is the y-axis. The x axis is the positive direction on the right and the y axis is the positive direction on the bottom. An axis orthogonal to the x axis and the y axis is a time axis, and the upper right direction in FIG. 6 is a positive direction. Low-frequency component LLL, temporal high-frequency component LLH, horizontal high-frequency component LHL, vertical high-frequency component HLL, temporal horizontal high-frequency component LHH, temporal vertical high-frequency component HLH, horizontal vertical high-frequency component, which are calculation results by the spatio-temporal wavelet first-order decomposition unit 150 HHL and diagonal high frequency component HHH have a relationship as shown in the figure. That is, a horizontal high frequency component LHL is present on the right side of the low frequency component LLL, a vertical high frequency component HLL is present on the lower side of the low frequency component LLL, and a horizontal vertical high frequency component HHL is located on the lower right side of the low frequency component LLL. . Further, there is a time horizontal high frequency component LHH in the positive time direction of the horizontal high frequency component LHL, a time high frequency component LLH in the positive time direction of the low frequency component LLL, and a positive time direction of the horizontal vertical high frequency component HHL. Has a diagonal high-frequency component HHH. Although there is a time vertical high frequency component HLH in the positive time direction of the vertical high frequency component HLL, it is not shown in FIG.

Returning to FIG. 5, the noise level extraction unit 151 extracts an isolated point in the diagonal high frequency component HHH obtained by the spatio-temporal wavelet first-order decomposition unit 150, and calculates the median absolute value of the element values of the extracted isolated point. The noise level is _LN . Here, the isolated point is an element in which the element is non- “0” and surrounding elements are “0”. For example, surrounding elements are elements adjacent in the x-axis direction (left and right), elements adjacent in the y-axis direction (up and down), and elements adjacent in the time axis direction, all of which are “0”. When the element has a value other than 0, the element is set as an isolated point. In addition, in the definition of “periphery”, other elements located in the vicinity of the element, such as elements of the front and rear frames adjacent in the x-axis direction (left and right), and elements at the upper right, lower right, upper left, and lower left of the point May be included.

  FIG. 7 is a diagram for explaining surrounding elements referred to in the noise level extraction unit 151. In FIG. 7, each circle represents an element of the diagonal high frequency component HHH. The horizontal axis is the x axis, and the vertical axis is the y axis. The x axis is the positive direction on the right and the y axis is the positive direction on the bottom. An axis perpendicular to the x-axis and the y-axis is a time axis, and the upper right direction in FIG. 7 is a positive direction. The surrounding elements referred to when determining whether or not the element C is an isolated point are the elements shaded in FIG. That is, the element R adjacent to the right of the element C, the element L adjacent to the left, the element U adjacent to the upper side, the element D adjacent to the lower side, the element B one frame before, and the element B one frame after Element N.

Returning to FIG. 5, a mask image generation unit 152, the absolute value of each element of the diagonal frequency components HHH, compared to the noise level L _N the noise level extraction unit 151 detects, the element than the noise level L _N When the absolute value of the element value is larger, the noise mask image M is generated with the element at the corresponding position set to “1”, and otherwise, the element at the corresponding position is set to “0”. The arrangement of elements in the noise mask image M is the same as that of the diagonal high frequency component HHH. Therefore, in the noise mask image M, the number of elements in the x-axis direction and the y-axis direction is the same as the number of pixels in the x-axis direction and the y-axis direction of the decoded reduced image, respectively. Also, the number of elements in the time axis direction is the same as the predetermined number of frames when the temporal and temporal wavelet first-order decomposition unit 150 performs discrete temporal and spatial wavelet first-order decomposition. Since it is generated in this manner, the noise component separation unit 153 using the noise mask image M separates the pixel corresponding to the position of the element when the value of the element of the noise mask image M is “1”. The noise component is separated on the assumption that the noise component to be included is included.

The noise component separation unit 153 separates the noise component for each component other than the low frequency component LLL among the components obtained by the spatio-temporal wavelet first-order decomposition unit 150. A case where noise components of the diagonal high frequency component HHH are separated will be described as an example. The noise component separation unit 153 compares the absolute value of the element value of the diagonal high frequency component HHH corresponding to the element having a value of “1” in the noise mask image M with the noise level _LN . When the absolute value of the element value is smaller than the noise level _LN , the element value of the element of the diagonal high frequency component HHH is considered to be composed of all noise components. On the other hand, when the absolute value of the element value is larger than the noise level _LN , it is considered that the element value is a noise component up to the noise level _LN , and the rest is a signal component. In addition, an element having a value “0” in the noise mask image M is regarded as not including a noise component. In this way, the noise component is separated from each element of the diagonal high-frequency component HHH, and the noise component of the diagonal high-frequency component HHH and the signal component of the diagonal high-frequency component HHH that is the remainder obtained by subtracting the noise component are obtained. Generate. The noise component separation unit 153 separates noise components in the same manner for components other than the diagonal high-frequency component HHH.

  FIG. 8 is a flowchart for explaining the operation of the noise component separation unit 153. The noise component separation unit 153 performs the processing shown in steps S2 to S10 for each component other than the low frequency component LLL (S1). In steps S2 to S10, the description of the component indicates a component that is a processing target. The noise component separation unit 153 performs the processing shown in steps S3 to S9 for each element of the component (S2). In steps S3 to S9, the description of the element indicates an element that is a processing target.

  In step S3, the noise component separation unit 153 acquires the value of the element of the noise mask image M corresponding to the element. Next, the noise component separation unit 153 determines whether or not the value acquired in step S3 is “1” (S4). When it is not “1” (S4-No), the noise component separation unit 153 sets the noise component of the element to “0”, sets the signal component to the element value of the element (S5), and returns to step S2 to be processed. Change the element.

On the other hand, when it determines with it being "1" in step S4 (S4-Yes), the noise component isolation | separation part 153 acquires the element value of the said element (S6). Next, the noise component separation unit 153 determines whether or not the absolute value of the element value acquired in step S6 is larger than the noise level _LN (S7). When the noise level is larger than the noise level L _N (S7-Yes), the noise component separation unit 153 calculates the absolute value of the signal component of the element as the absolute value of the element value of the element acquired in Step 6−the noise level L _N. And the sign (positive or negative) of the signal component is the same as the element value of the element. The absolute value of the noise component of the element is the noise level L _N and the sign (positive / negative) of the noise component is the same as the element value of the element. (S8), returning to step S2, the element to be processed is changed. Also, when it is determined in step S7 that it is not large (S7-No), the noise component separation unit 153 uses the noise component of the element as the element value of the element acquired in step 6, and sets the signal component as “ 0 ”(S9), the process returns to step S2 to change the element to be processed.

  Returning to FIG. 5, the noise component reconstruction unit 154 reconstructs a discrete space-time wavelet without decimation using the noise components of each component such as the diagonal high frequency component HHH generated by the noise component separation unit 153. Then, a noise component image is generated. The noise component separation unit 153 generates noise components for components other than the low frequency component LLL, and the noise component reconstruction unit 154 sets all elements to “0” for the low frequency component portion. This discrete space-time wavelet reconstruction without decimation is performed. The components other than the low frequency component LLL are the time high frequency component LLH, the horizontal high frequency component LHL, the vertical high frequency component HLL, the time horizontal high frequency component LHH, the time vertical high frequency component HLH, the horizontal vertical high frequency component HHL, and the diagonal high frequency component HHH. It is. Note that the same base as that for the spatio-temporal wavelet first-order decomposition unit 150 is used for the first-order reconstruction of the discrete spatio-temporal wavelet without decimation.

  The signal component reconstruction unit 155 generates a signal component image by performing discrete space-time wavelet reconstruction without decimation using the signal components of each component such as the diagonal high frequency component HHH generated by the noise component separation unit 153. To do. Since the noise component separation unit 153 generates signal components for components other than the low frequency component LLL, the signal component reconstruction unit 155 performs the spatio-temporal wavelet first-order decomposition unit 150 for the low frequency component portion. Is used as it is. Note that the same base as that for the spatio-temporal wavelet first-order decomposition unit 150 is used for the first-order reconstruction of the discrete spatio-temporal wavelet without decimation.

  FIG. 9 is a schematic block diagram illustrating a configuration of the signal component super resolving unit 106. As shown in FIG. 9, the signal component super-resolution unit 106 includes a wavelet first-order decomposition unit 160, an extended LH component generation unit 161a, an extended HL component generation unit 161b, an extended HH component generation unit 161c, a variance value setting unit 162, A wavelet first floor reconstruction unit 163 is included.

  The wavelet first-order decomposition unit 160 performs the discrete wavelet first-order decomposition for each frame of the signal component image generated by the noise / signal separation unit 105. Thereby, the wavelet first-order decomposition unit 160 decomposes each frame of the signal component image into a low frequency component LL, a horizontal high frequency component LH, a vertical high frequency component HL, and a diagonal high frequency component HH. Any base such as Haar or Daubechies may be used as the base used for the discrete wavelet first-order decomposition.

  The extended LH component generation unit 161a expands the horizontal high-frequency component LH generated by the wavelet first-order decomposition unit 160 so that the number of elements is doubled in the horizontal direction and the vertical direction. LH is generated. For example, if the number of elements of the horizontal high-frequency component LH is 640 × 480, the extended horizontal high-frequency component Ex. The number of elements of LH is 1280 × 960. Specifically, the extended LH component generation unit 161a places the horizontal high-frequency component LH in the low-frequency component region, sets the other components to 0, performs wavelet first-order reconstruction, and then applies a bilateral filter. The extended horizontal high frequency component Ex. Let LH. Thereby, the extended horizontal high-frequency component Ex. The number of elements of LH matches the number of pixels of each frame such as a signal component image, a decoded reduced moving image, and a reduced moving image. It is desirable to use the same base as that used in the wavelet first-order decomposition unit 160 as the base used for the discrete wavelet first-order reconstruction.

Also, the value σ _LH designated by the variance value setting unit 162 is used as the variance value of the bilateral filter. When the pixel value of the coordinate (x, y) in the input image is represented by f (x, y) and the pixel value of the coordinate (x, y) in the output image is represented by g (x, y), bilateral The filter is expressed by Expression (2).

In Equation (2), w is the kernel size, σ ₁ is the variance value related to the distance, and σ ₂ is the variance value related to the pixel value difference. In the extended LH component generation unit 161 a, w uses a preset value, and σ ₁ and σ ₂ use σ _LH designated by the variance value setting unit 162.

The extended HL component generation unit 161b extends the vertical high-frequency component HL generated by the wavelet first-order decomposition unit 160 so as to have twice the number of elements in the horizontal direction and the vertical direction in the same manner as the extended LH component generation unit 161a. Extended vertical high frequency component Ex. HL is generated. The extended HL component generation unit 161b uses the value σ _HL specified by the dispersion value setting unit 162 as the dispersion values σ ₁ and σ ₂ of the bilateral filter.

The extended HH component generator 161c, like the extended LH component generator 161a, causes the diagonal high-frequency component HH generated by the wavelet first-order decomposition unit 160 to have twice the number of elements in the horizontal direction and the vertical direction. Extended extended diagonal high frequency component Ex. HH is generated. Note that the expanded HH component generation unit 161c uses the value σ _HH specified by the dispersion value setting unit 162 as the dispersion values σ ₁ and σ ₂ of the bilateral filter.

The variance value setting unit 162 inputs the corresponding variance values σ _LH , σ _HL , and σ _HH to the extended LH component generation unit 161a, the extended HL component generation unit 161b, and the extended HH component generation unit 161c, respectively. For example, the variance value setting unit 162 stores a plurality of predetermined values as variance values, and for one frame of the signal component image, the expanded horizontal high-frequency component Ex. LH, extended vertical high frequency component Ex. HL, extended diagonal high frequency component Ex. Dispersion values σ _LH , σ _HL , and σ _HH are input so that HH is generated. Specifically, when there are three predetermined values as dispersion values C1 to C3, the dispersion values σ _LH , σ _HL , and σ _HH each take C1 to C3, so that the extended horizontal high-frequency component Ex . LH, extended vertical high frequency component Ex. HL, extended diagonal high frequency component Ex. The dispersion value setting unit 162 inputs dispersion values σ _LH , σ _HL , and σ _HH so that three HHs can be _generated . Further, the dispersion value setting unit 162 also inputs the dispersion values σ _LH , σ _HL , and σ _HH that are the super-resolution parameters related to the signal component to the super-resolution parameter determination unit 109.

  The wavelet first floor reconstruction unit 163 includes a signal component image, an extended horizontal high-frequency component Ex. LH and extended vertical high frequency component Ex. HL and extended diagonal high-frequency component Ex. HH and wavelet first-order reconstruction are performed to generate a reconstructed signal component image. By doing so, the reconstructed signal component image becomes an image having the same structure as the structure appearing in the signal component image in the high frequency component. Specifically, the wavelet first-order reconstruction unit 163 arranges one frame of the signal component image in the low-frequency component region, and expands the horizontal horizontal high-frequency component Ex. LH and extended vertical high frequency component Ex. HL and extended diagonal high-frequency component Ex. HHs are arranged in a horizontal high-frequency component region, a vertical high-frequency component region, and a diagonal high-frequency component region, respectively, and a wavelet first-order reconstruction is performed to generate a reconstructed signal component image. It is desirable to use the same base as that used in the wavelet first-order decomposition unit 160 as the base used for the discrete wavelet first-order reconstruction.

  For one frame of the signal component image, the extended horizontal high-frequency component Ex. LH and extended vertical high frequency component Ex. HL and extended diagonal high-frequency component Ex. Since a plurality of HHs having different dispersion values used in the bilateral filter are generated, the wavelet first-order reconstruction unit 163 generates reconstructed signal component images for all of these combinations. For example, the extended horizontal high frequency component Ex. LH and extended vertical high frequency component Ex. HL and extended diagonal high-frequency component Ex. When three HHs are respectively generated, reconstructed signal component images are generated for all of these combinations, that is, 3 × 3 × 3 = 27.

  FIG. 10 is a conceptual diagram illustrating the processing of the extended LH component generation unit 161a. As indicated by reference numeral SB in FIG. 10, each frame of the signal component image is decomposed by the wavelet first-order decomposition unit 160 into a low frequency component LL, a horizontal high frequency component LH, a vertical high frequency component HL, and a diagonal high frequency component HH. . The extended LH component generation unit 161a arranges the horizontal high-frequency component LH of these in the low-frequency component region, as indicated by the symbol LHB in FIG. 10, and includes the horizontal high-frequency component, the vertical high-frequency component, and the diagonal high-frequency component. “0” is placed in all the elements of the region. Then, the extended LH component generation unit 161a performs wavelet first-order reconfiguration on this arrangement result, and then applies a bilateral filter to generate an extended horizontal high-frequency component Ex. LH is generated. The extended HL component generation unit 161b also applies the extended vertical high frequency component Ex. HL is generated. The extended HH component generation unit 161c also applies the extended high-frequency component Ex. HH is generated.

  FIG. 11 is a diagram illustrating the arrangement of each component in the wavelet first-floor reconstruction unit 163. In FIG. 11, symbol S indicates a frame of the signal component image, and symbol Ex. LH indicates an extended horizontal high-frequency component, and the symbol Ex. HL indicates an extended vertical high-frequency component, and the symbol Ex. HH indicates an extended diagonal high-frequency component. As shown in FIG. 11, the frame S of the signal component image is arranged in the low frequency component region. Extended horizontal high frequency component Ex. LH is disposed in the horizontal high-frequency component region. Extended vertical high frequency component Ex. The HL is disposed in the vertical high frequency component region. Extended diagonal high frequency component Ex. HH is arranged in a region of diagonal high frequency components. The wavelet first floor reconstruction unit 163 performs wavelet first floor reconstruction after arranging each component as shown in FIG. The reconstructed signal component image obtained by this reconstruction has the same number of pixels as each frame of the input moving image.

  FIG. 12 is a schematic block diagram illustrating a configuration of the noise component super resolving unit 107. As shown in FIG. 12, the noise component super resolving unit 107 includes a noise component rearrangement unit 170, a wavelet first-order reconstruction unit 171, a pixel value normalization unit 172, a white noise application unit 173, a bilateral filter unit 174, A variance value setting unit 175 is included.

  The noise component rearrangement unit 170 includes a low-frequency component region, a horizontal high-frequency component region, and a vertical high-frequency component when an image having twice the number of pixels of the noise component image in the vertical direction and the horizontal direction is subjected to wavelet first-order decomposition. The processing target frames of the noise component image are arranged in the component region and the diagonal high frequency component region, respectively. That is, the same frame is arranged in these four areas. The wavelet first-order reconstruction unit 171 performs wavelet first-order reconstruction on the arrangement result of the noise component rearrangement unit 170. Any base such as Haar or Daubechies may be used as the base used for the discrete wavelet first-order reconstruction.

  As a result, an image having twice as many pixels as the noise component image in the vertical and horizontal directions is generated. In the image generated in this manner, pixels corresponding to elements having pixel values in the noise component image have pixel values. That is, super-resolution suitable for white Gaussian noise with low correlation in the spatio-temporal direction can be achieved. In addition, since the noise of the image imaged with image sensors, such as a CMOS (Complementary Metal Oxide Semiconductor) sensor, is mainly white Gaussian noise, such a super-resolution method is suitable.

The pixel value normalization unit 172 normalizes the pixel value of the image reconstructed by the wavelet first-order reconstruction unit 171 with the noise level _LN . For example, the pixel value normalization unit 172 calculates the average value Na of the pixel values excluding the pixels whose pixel value is “0” in the reconstructed image. Further, the pixel value normalization unit 172 multiplies the pixel value of each pixel of the reconstructed image by a ratio L _N / Na obtained by dividing the noise level L _N by the average value Na. At this time, the median may be used instead of the average value Na.

The white noise applying unit 173 applies white noise to the image whose pixel values are normalized by the pixel value normalizing unit 172. Specifically, the white noise applying unit 173 sets the pixel value of the pixel whose pixel value is “0” in the image whose pixel value is normalized to (−1, 1] with an average value of 0 and a variance value of 1 A value obtained by adding 1 to a normal random number (a number greater than −1 and equal to or less than 1) and then multiplying by a noise level L _N. That is, the pixel value is a normal random number whose average and variance are noise levels L _N and The bilateral filter unit 174 reproduces the whiteness of the noise using the above-described equation (2) for the image to which the white noise is applied by the white noise applying unit 173. As in the bilateral filter in the extended LH component generation unit 161a and the like, w in Equation (2) uses a preset value, and σ ₁ , σ ₂ Is specified by the variance value setting unit 175 Using a sigma _N.

The variance value setting unit 175 inputs the variance value σ _N to the bilateral filter unit 174. For example, the variance value setting unit 175 stores a plurality of predetermined values as variance values, and generates a reconstructed noise component image corresponding to each of these values for one frame of the noise component image. The variance value σ _N is input as shown. Further, the dispersion value setting unit 175 also inputs the dispersion value σ _N that is a super-resolution parameter regarding the noise component to the super-resolution parameter determination unit 109.

  FIG. 13 is a diagram for explaining the arrangement of noise component images by the noise component rearrangement unit 170. In FIG. 13, the symbol N indicates a frame of a noise component image. As described above, the noise component rearrangement unit 170 has a noise component image in each of the four regions of the low frequency component region, the horizontal high frequency component region, the vertical high frequency component region, and the diagonal high frequency component region. Arrange the frames.

FIG. 14 is a schematic block diagram illustrating a configuration of the image decoding device 300. As shown in FIG. 14, the image decoding apparatus 300 includes a parameter extraction unit 301, a decoding unit 302, a noise / signal separation unit 303, a signal component super resolving unit 304, a noise component super resolving unit 305, an adding unit 306, and an output. The unit 307 is included. The parameter extraction unit 301 separates the encoded data received via the network 200 into a super-resolution parameter, a noise level _LN , and encoded image data. The parameter extraction unit 301 inputs super-resolution parameters related to signal components (dispersion values σ _LH , σ _HL , σ _HH ) to the signal component super-resolution unit 304, and relates to noise components (dispersion value σ _N ) Is input to the noise component super resolving unit 305. The parameter extraction unit 301 also inputs the noise level LN to the noise / signal separation unit 303.

The decoding unit 302 is the same as the decoding unit 104 of the image encoding device 100, and decodes encoded image data to generate a decoded reduced moving image. The noise / signal separation unit 303 has the same configuration as the noise / signal separation unit 105 of the image encoding device 100, except that the noise / signal separation unit 151 is not provided. The noise / signal separation unit 303 separates the decoded reduced moving image into a signal component image and a noise component image using the noise level _LN input by the parameter extraction unit 301.

The signal component super resolving unit 304 has the same configuration as that of the signal component super resolving unit 106 of the image encoding device 100, except that the variance value setting unit 162 is not provided. Since the signal component super resolving unit 304 does not have the dispersion value setting unit 162, the super resolving parameters (dispersion values σ _LH , σ _HL , σ _HH ) related to the signal component input by the parameter extraction unit 301 are used, A reconstructed signal component image is generated in the same manner as the signal component super resolving unit 106.

The noise component super resolving unit 305 has the same configuration as that of the noise component super resolving unit 107 of the image encoding device 100, except that the variance value setting unit 175 is not provided. Since the noise component super resolving unit 305 does not have the variance value setting unit 175, the noise component super resolving unit is used by using the super resolving parameter (variance value σ _N ) related to the noise component input by the parameter extracting unit 301. In the same manner as in 107, a reconstructed noise component image is generated.

  The adding unit 306 adds the reconstructed signal component image generated by the signal component super resolving unit 304 and the reconstructed noise component image generated by the noise component super resolving unit 305. Thereby, the input moving image is restored. The output unit 307 outputs a moving image signal (decoded moving image) in which each image is the addition result of the adding unit 306.

  In the noise level extraction unit 151 described above, an element in which the element is non- “0” and the surrounding elements are “0” is set as an isolated point in the diagonal high-frequency component HHH, but is not limited thereto. For example, the median absolute value of elements that are not “0” in the diagonal high-frequency component HHH is used as a threshold value, and among the elements of the diagonal high-frequency component HHH, the element value of the element and the element values of surrounding elements A point where the difference from each exceeds the threshold may be used as an isolated point. Alternatively, among the elements of the diagonal high frequency component HHH, an element in which the element is non- “0” and the surrounding elements are “0” or a minimum value may be set as an isolated point.

Also, the variance value setting unit 162 sets the variances σ ₁ and σ ₂ in the bilateral filter to the same value in each of the extended LH component generation unit 161a, the extended HL component generation unit 161b, and the extended HH component generation unit 161c. However, σ ₁ and σ ₂ may be set individually so that they can be set to different values. Similarly, the variance value setting unit 175 may individually set the variances σ ₁ and σ ₂ in the bilateral filter unit 174 so that they can be set to different values.

As described above, the image encoding device 100 according to the present embodiment superimposes the encoded data obtained by encoding the image obtained by reducing the input moving image and the noise component and the signal component of the reduced image. Therefore, it is possible to obtain a good image quality when the encoded data is decoded while suppressing the data amount of the encoded data.
Further, since the noise component and the signal component are super-resolved by a method corresponding to each of them, it is possible to obtain a good image quality when decoding the encoded data.

[Second Embodiment]
The second embodiment of the present invention will be described below with reference to the drawings. The super-resolution device 500 in the present embodiment super-resolutions the input moving image that has been input, and generates and outputs an image having twice the number of pixels in the vertical and horizontal directions. FIG. 15 is a schematic block diagram showing the configuration of the super-resolution apparatus 500.

  The super-resolution apparatus 500 includes an input unit 501, a spatio-temporal wavelet first-order decomposition unit 150, a noise level extraction unit 151, an arrangement unit 502, a wavelet first-order reconstruction unit 171, a pixel value normalization unit 172, and a white noise application unit 173. , A bilateral filter unit 174, a variance value setting unit 503, and an output unit 504. 15, parts corresponding to those in FIGS. 5 and 12 are given the same reference numerals (150, 151, 171 to 174), and description thereof is omitted.

The input unit 501 accepts an input moving image from the outside. The input unit 501 inputs the received input moving image to the spatio-temporal wavelet first-order decomposition unit 150 and the arrangement unit 502. The placement unit 502, in the same way as the noise component rearrangement unit 170 in FIG. 12, sets the low-frequency component region, the horizontal high-frequency component region, and the vertical high-frequency component region as a pair. It is arranged in the area of angular high frequency components. The variance value setting unit 503 sets a predetermined specific variance value σ _N in the bilateral filter unit 174. That is, in the present embodiment, as in the first embodiment, for each frame, various values are used as super-resolution parameters, and only specific values are used instead of selecting from them. . The output unit 504 outputs an image of the processing result of the bilateral filter unit 174.

  As described above, the super resolving apparatus 500 according to the present embodiment super-resolutions the input input moving image in the same manner as the noise component super resolving unit 107 in FIG. Therefore, it is possible to perform super-resolution suitable for images with a lot of white Gaussian noise, such as images taken with an ultra-sensitive camera.

[Third Embodiment]
The third embodiment of the present invention will be described below with reference to the drawings. The super-resolution apparatus 600 in the present embodiment super-resolutions the input moving image that has been input, and generates and outputs an image having twice the number of pixels in the vertical and horizontal directions. FIG. 16 is a schematic block diagram showing the configuration of the super-resolution apparatus 600. In FIG. 16, the noise / signal separating unit 105 is the same as the noise / signal separating unit 105 in FIG.

The super-resolution device 600 includes an input unit 601, a noise / signal separation unit 105, a signal component super-resolution unit 602, a noise component super-resolution unit 603, an addition unit 604, and an output unit 605. The input unit 601 receives an input moving image from the outside. The signal component super resolving unit 602 super-resolutions the signal component image generated by the noise / signal separating unit 105 in the same manner as the signal component super resolving unit 106 in FIG. However, the signal component super-resolution unit 602 performs super-resolution using a predetermined specific set as super-resolution parameters (dispersion values σ _LH , σ _HL , σ _HH ) related to the signal components.

The noise component super-resolution unit 603 super-resolutions the noise component image generated by the noise / signal separation unit 105 in the same manner as the noise component super-resolution unit 107 in FIG. However, the noise component super-resolution unit 603 performs super-resolution using a predetermined specific value as a super-resolution parameter (dispersion value σ _N ) regarding the noise component. The adding unit 604 adds the image super-resolved by the signal component super-resolution unit 602 and the image super-resolved by the noise component super-resolution unit 603. The output unit 605 outputs an image as a result of the addition performed by the adding unit 604.

  As described above, the super-resolution apparatus 600 according to the present embodiment separates an input moving image into a signal component and a noise component, and inputs the input moving image in the same manner as the noise / signal separation unit 105 in FIG. Is super-resolution. Therefore, it is possible to perform super-resolution suitable for images with a lot of white Gaussian noise, such as images taken with an ultra-sensitive camera.

Note that the super-resolution device 500 in the second embodiment and the super-resolution device 600 in the third embodiment may be included in an image display device such as a television receiver, a television camera, or the like. You may make it include in this imaging device.
Further, in each of the above-described embodiments, the case where each pixel has one pixel value (for example, a luminance value) has been described as an example in order to simplify the description. When there are multiple pixel values, such as when the pixel value has a luminance value, a red color difference, and a blue color difference, or when there are red, green, and blue luminance values, for each pixel value Thus, the same processing as in this embodiment may be performed.

  Further, the computer reads the program for realizing the functions of the image encoding device 100, the image decoding device 300 in FIG. 1, the super-resolution device 500 in FIG. 15, and the super-resolution device 600 in FIG. Each device may be realized by recording on a possible recording medium, causing the computer system to read and execute the program recorded on the recording medium. Or you may make it implement | achieve each apparatus by the hardware for exclusive use for implement | achieving the above-mentioned function. Here, the “computer system” includes an OS and hardware such as peripheral devices.

  The “computer-readable recording medium” refers to a storage device such as a flexible medium, a magneto-optical disk, a portable medium such as a ROM and a CD-ROM, and a hard disk incorporated in a computer system. Furthermore, the “computer-readable recording medium” dynamically holds a program for a short time like a communication line when transmitting a program via a network such as the Internet or a communication line such as a telephone line. In this case, a volatile memory in a computer system serving as a server or a client in that case, and a program that holds a program for a certain period of time are also included. The program may be a program for realizing a part of the functions described above, and may be a program capable of realizing the functions described above in combination with a program already recorded in a computer system.

  The embodiment of the present invention has been described in detail with reference to the drawings. However, the specific configuration is not limited to this embodiment, and includes design changes and the like within a scope not departing from the gist of the present invention.

DESCRIPTION OF SYMBOLS 10 ... Image transmission system 100 ... Image coding apparatus 101 ... Image reduction part 102 ... Encoding part 103 ... Encoded data generation part 104 ... Decoding part 105 ... Noise / signal separation part 106 ... Signal component super-resolution part 107 ... Noise Component super resolving unit 108 ... Adding unit 109 ... Super resolving parameter determining unit 110 ... Wavelet first order resolving unit 111 ... Low frequency component extracting unit 150 ... Time-space wavelet first order resolving unit 151 ... Noise level extracting unit 152 ... Mask image Generation unit 153 ... Noise component separation unit 154 ... Noise component reconstruction unit 155 ... Signal component reconstruction unit 160 ... Wavelet first-order decomposition unit 161a ... Extended LH component generation unit 161b ... Extended HL component generation unit 161c ... Extended HH component generation unit 162 ... dispersion value setting unit 163 ... wavelet first-order reconstruction unit 170 ... noise component rearrangement unit 171 ... Wavelet first-order reconstruction unit 172 ... pixel value normalization unit 173 ... white noise application unit 174 ... bilateral filter unit 175 ... dispersion value setting unit 200 ... network 300 ... image decoding device 301 ... parameter extraction unit 302 ... decoding unit 303 ... Noise / Signal Separation Unit 304 ... Signal Component Super Resolution Unit 305 ... Noise Component Super Resolution Unit 306 ... Addition Unit 307 ... Output Unit 400 ... Image Display Device 500 ... Super Resolution Device 501 ... Input Unit 502 ... Arrangement Unit 503 ... Dispersion value setting unit 504 ... output unit 600 ... super-resolution device 601 ... input unit 602 ... signal component super-resolution unit 603 ... noise component super-resolution unit 604 ... addition unit 605 ... output unit

Claims (6)

An image reduction unit that reduces an input image and generates a reduced image;
An encoding unit for encoding the reduced image;
A decoding unit for decoding the encoded reduced image;
Noise / signal separation for separating each pixel value of the decoded reduced image into a noise component and a signal component to generate a first image composed of the signal component and a second image composed of the noise component And
A signal component super-resolution unit that super-resolutions the first image to generate a first super-resolution image;
A noise component super-resolution unit that generates a second super-resolution image by super-resolution of the second image by a method different from the signal component super-resolution unit;
An adder that adds the first super-resolution image and the second super-resolution image to obtain a super-resolution image of the decoded reduced image;
A super-resolution parameter of at least one of the signal component super-resolution unit and the noise component super-resolution unit is determined by comparing the decoded reduced image super-resolution image and the input image. A super-resolution parameter determination unit that
An image encoding apparatus comprising: an encoded data generation unit that generates encoded data including the encoded reduced image and the determined super-resolution parameter. At least one of the signal component super-resolution unit and the noise component super-resolution unit performs filtering by a bilateral filter when generating the super-resolution,
The image encoding apparatus according to claim 1, wherein the super-resolution parameter includes a variance value of the bilateral filter.   A program for causing a computer to function as the image encoding device according to claim 1 or 2. A data separator that extracts an encoded image and super-resolution parameters from the encoded data;
A decoding unit that decodes the encoded image and generates a reduced image;
A noise / signal separation unit that separates each pixel value of the reduced image into a noise component and a signal component, and generates a first image composed of the signal component and a second image composed of the noise component; ,
A signal component super-resolution unit that super-resolutions the first image to generate a first super-resolution image;
A noise component super-resolution unit that generates a second super-resolution image by super-resolution of the second image by a method different from the signal component super-resolution unit;
An adder that adds the first super-resolution image and the second super-resolution image to obtain a super-resolution image of the reduced image; and the signal component super-resolution unit and the At least one of the noise component super-resolution units performs super-resolution using the super-resolution parameter. At least one of the signal component super-resolution unit and the noise component super-resolution unit performs filtering by a bilateral filter when generating the super-resolution,
The image decoding apparatus according to claim 4, wherein the super-resolution parameter includes a variance value of the bilateral filter.   A program for causing a computer to function as the image decoding device according to claim 4 or 5.

Images