JP2011004345A - Spatial downsampling processor, spatial upsampling processor, encoding apparatus, decoding apparatus, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a spatial downsampling processor, a spatial upsampling processor, an encoding apparatus, a decoding apparatus, and a program, wherein the influence of aliasing distortion can be reduced and a high spatial frequency component can be highly accurately inferred.SOLUTION: The spatial downsampling processor includes: a two-dimensional one-stage discrete wavelet decomposition-processing part 21 for subjecting a processing target image to one-stage discrete wavelet decomposition and calculating an average value of wavelet coefficients in each decomposed frequency component area; a band limitation-processing part 22 for applying previously regulated band limitation to each of high spatial frequency components; a two-dimensional one-stage discrete wavelet reconfiguration part 23 for performing one-stage discrete wavelet reconfiguration by using the low spatial frequency components and the high spatial frequency component subjected to the band limitation; and a horizontal and vertical 1:2 pixel-thinning-processing part 24 for executing spatial downsampling by applying pixel thinning to the reconfigured image data in horizontal and vertical directions respectively at a rate of 1:2 to generate a contracted image signal.

Description

  The present invention relates to a spatial frequency conversion technique using a wavelet transform technique in image encoding, and more particularly to a spatial downsampling processing device, a spatial upsampling processing device, an encoding device, a decoding device, and a program.

  For transmission of moving picture coding, after down-sampling an image on a spatial frequency (hereinafter referred to as “spatial down-sampling”), In some cases, encoding is performed based on an encoding method such as H.264, MPEG-2, etc., and then transmitted, then decoded and up-sampled (hereinafter referred to as “spatial up-sampling”) on the spatial frequency.

  2. Description of the Related Art Conventionally, a spatial frequency conversion technique using a wavelet transform technique in moving picture coding is known (see, for example, Patent Document 1). A conventional spatial downsampling method uses a method of performing a thinning process with a linear low-pass filter, and a conventional spatial upsampling method includes a method of performing linear filtering or nonlinear processing.

  For example, in spatial downsampling, sample points are usually thinned out after applying a low-pass filter in order to prevent unwanted high-frequency components from aliasing.

  Spatial upsampling is a technique for estimating a high spatial frequency component lost by spatial downsampling or encoding by a predetermined method. In recent years, this field has been researched and developed in the genre of super-resolution. As a classic method, there is an up-sampling technique of a spatial sampling frequency by interpolation using a linear filter. As a more recent method, there is a total variation method as a method using nonlinear processing. However, since the Total Variation method needs to use repetitive operations, it has not yet been made into embedded hardware.

Special table 2007-504523

  A problem at the time of conventional spatial downsampling is that it is usually necessary to perform sampling point thinning after applying a low-pass filter adapted to spatial downsampling. That is, since it is difficult to create a filter with ideal characteristics as a low-pass filter, the problem of aliasing distortion cannot be avoided, and this aliasing distortion component appears as moire (interference fringes) or ringing. In addition, the original image component may be slightly damaged.

  Further, a problem in the conventional spatial upsampling is that it is necessary to estimate the high spatial frequency component with high accuracy when estimating the lost high spatial frequency component. If a high spatial frequency component is estimated with low accuracy, it will appear as image quality degradation in the form of jaggy (jagged edges on a staircase or a curved line), moire, or ringing.

  Therefore, an object of the present invention is to provide a spatial downsampling processing device, a spatial upsampling processing device, an encoding device, a decoding device, and a program that can reduce the influence of aliasing distortion and can estimate a high spatial frequency component with high accuracy. It is to provide.

  In the present invention, spatial downsampling of the high spatial frequency component of the original image is performed, encoded and transmitted or stored. When an image is received or read out, spatial upsampling of a known high spatial frequency component of the original image is performed after decoding. That is, since the high spatial frequency component of the original image with respect to the spatial upsampling is known before the spatial downsampling, the spatial downsampling is applied in advance so as not to cause a large image quality degradation when performing the spatial upsampling.

  That is, the spatial downsampling processing device of the present invention is a spatial downsampling processing device that spatially downsamples a spatial frequency component obtained by wavelet decomposition of a moving image, and for a two-dimensional image to be processed in the moving image, The first-order discrete wavelet decomposition is performed in the horizontal direction and the vertical direction, and the average value of each wavelet coefficient for each region of each frequency component in the horizontal direction and the vertical direction subjected to the first-order discrete wavelet decomposition is determined as a high frequency at the time of spatial upsampling. A two-dimensional first-order discrete Wavelet decomposition processing unit that is calculated as multiple information used for component estimation and a high spatial frequency component in the horizontal and vertical spatial frequency components of the two-dimensional image are each subjected to predetermined band limitation. Supplied from the bandwidth limitation processing unit and the bandwidth limitation processing unit A two-dimensional first-order discrete wavelet reconstruction unit that performs first-order discrete wavelet reconstruction using a low spatial frequency component that is generated and a high spatial frequency component that is subjected to the band limitation, and horizontal and vertical directions of the reconstructed image data And a pixel decimation processing unit that performs spatial downsampling by performing pixel decimation and generates a reduced image signal.

  Further, in the spatial downsampling processing apparatus of the present invention, the band limiting processing unit performs a one-dimensional one-dimensional one in the horizontal direction of the horizontal high frequency / vertical low frequency component when the band limitation of the horizontal high frequency / vertical low frequency component is performed. When the discrete wavelet decomposition is performed and the high frequency component side is converted to 0, and the band limitation of the vertical low frequency / horizontal high frequency component is applied, the one-dimensional first order discrete wavelet in the vertical direction of the vertical low frequency / horizontal high frequency component When the decomposition is performed and the high frequency component side is converted to 0 and the band limitation of the horizontal high frequency / vertical high frequency component is performed, all of the horizontal high frequency / vertical high frequency components are converted to 0.

  Further, in the spatial downsampling processing device of the present invention, the spatial downsampling processing device of the present invention and the reduced image signal output from the spatial downsampling processing device are compressed and encoded by a predetermined encoding method to generate encoded data. And an encoder to be generated.

  The spatial upsampling processing apparatus of the present invention is a spatial upsampling processing apparatus that performs spatial upsampling on image data composed of reduced image signals generated by the spatial downsampling processing apparatus of the present invention. A two-dimensional second-order discrete Wavelet decomposition processing unit that performs two-dimensional second-order wavelet transform, and an interpolation process defined in advance for each frequency component from each frequency component supplied from the two-dimensional second-order discrete Wavelet decomposition processing unit A high frequency component estimation processing unit that performs upsampling and multiplies the multiple information to estimate and reproduce each high frequency component; a low spatial frequency component supplied from the high frequency component estimation processing unit; and each reproduced high frequency The first-order discrete wavelet reconstruction using the components, and A two-dimensional first floor discrete Wavelet reconstruction unit for restoring an image with physical object, characterized in that it comprises a.

  In the spatial upsampling processing device of the present invention, the high-frequency component estimation processing unit may perform second-level horizontal in the case of estimation of horizontal high-frequency / vertical low-frequency components as an interpolation process defined in advance for each frequency component. Apply spatial upsampling to each of the high-frequency / vertical low-frequency components and the first-order horizontal high-frequency / vertical low-frequency components, shift the phases of the frequency components that have undergone spatial upsampling, and multiply the corresponding frequency components. In the case of the vertical high frequency / horizontal low frequency component estimation, spatial upsampling is performed on each of the second floor vertical high frequency / horizontal low frequency component and the first floor vertical high frequency / horizontal low frequency component, Multiplying each phase of the frequency component subjected to the spatial upsampling while shifting the phase, and multiplying the corresponding frequency component multiple information, In the case of estimation of direct high-frequency / horizontal high-frequency components, spatial upsampling is performed on each of the second-floor vertical high-frequency / horizontal high-frequency components and the first-floor vertical high-frequency / horizontal high-frequency components, Each phase is shifted and multiplied, and multiple information of the corresponding frequency component is multiplied.

  The decoding device of the present invention includes the spatial upsampling processing device of the present invention and a decoder for decoding the encoded data encoded by the encoding device of the present invention.

  Further, the present invention provides a computer configured as a spatial downsampling processing device that spatially downsamples a spatial frequency component obtained by wavelet decomposition of a moving image, with respect to a two-dimensional image to be processed in the moving image, in a horizontal direction and a vertical direction. In addition to performing first-order discrete wavelet decomposition in the direction, the average value of each wavelet coefficient for each frequency component region in the horizontal and vertical directions subjected to the first-order discrete wavelet decomposition is used to estimate high-frequency components during spatial upsampling. A step of calculating as multiple information to be used; a step of applying a predetermined band limitation to each of the high spatial frequency components in the horizontal and vertical spatial frequency components of the two-dimensional image; and a low spatial frequency supplied from the step High spatial frequency with component and band limitation Performing a first-order discrete wavelet reconstruction using a minute, performing a spatial downsampling by performing pixel decimation in the horizontal direction and the vertical direction of the reconstructed image data, and generating a reduced image signal, It is also characterized as a program for executing.

  The present invention also provides a computer configured as a spatial upsampling processing device that performs spatial upsampling on image data composed of reduced image signals generated by the spatial downsampling processing device of the present invention, and the image data is two-dimensionally two-dimensional. For each frequency component supplied from the step, upsampling is performed using interpolation processing defined in advance for each frequency component, and each high frequency component is estimated by multiplying the multiple information. Reconstructing, performing a first-order discrete wavelet reconstruction using the low spatial frequency component supplied from the step and each reproduced high frequency component, and reconstructing the two-dimensional image as the processing target; It is also characterized as a program for running

  According to the present invention, it is possible to perform spatial downsampling / upsampling with higher image quality than when using the conventional spatial downsampling method and spatial upsampling method.

It is a figure which shows the encoding apparatus of one Example by this invention. It is a figure which shows the decoding apparatus of one Example by this invention. It is a flowchart which shows operation | movement of the encoding apparatus of one Example by this invention. It is a flowchart which shows operation | movement of the decoding apparatus of one Example by this invention. (A) is a process of a low-pass filter applied for downsampling the horizontal sampling frequency in the conventional spatial downsampling method to 1/2 (similarly, the vertical sampling frequency to 1/2). It is a figure which shows an example, (b) is a figure which shows the example of a process which thins out a sample point after applying the low-pass filter in the conventional spatial downsampling method. It is a figure which shows a mode in the case of not applying a low-pass filter by the conventional space downsampling method. (A) is a figure which shows a mode that a zone | band limitation is applied to a horizontal high frequency / vertical low frequency component in the spatial downsampling processor in the encoding apparatus of one Example by this invention, (b) One implementation by this invention It is a figure which shows the band filter function of the horizontal direction in the space downsampling processor in the encoding apparatus of an example. (A) is a figure which shows a mode that a band limitation is carried out to a vertical high frequency and a horizontal low frequency component in the spatial downsampling processor in the encoding apparatus of one Example by this invention, (b) One implementation by this invention It is a figure which shows the band-pass filter function of the vertical direction in the spatial downsampling processor in the encoding apparatus of an example. (A) is a figure which shows the image SpDw spatially downsampled in the encoding apparatus of one Example by this invention, Each of (b) and (c) is an encoding apparatus of one Example by this invention. FIG. 6 is a diagram illustrating a signal gain state of an image signal of a spatial downsampled image SpDw that includes a folding component at ¼ or more of the sampling frequency in each of the horizontal direction and the vertical direction. It is a figure which shows a mode that spatial 2nd-order wavelet transformation is performed with respect to the image SpDw spatially downsampled in the spatial upsampling processor in the decoding apparatus of one Example by this invention. It is a figure which shows a mode that the high spatial frequency component of a horizontal direction is estimated with respect to the frequency component of the image SpDw spatially downsampled by the spatial upsampling processor in the decoding apparatus of one Example by this invention. It is a figure which shows a mode that the high spatial frequency component of a perpendicular direction is estimated with respect to the frequency component of the image SpDw spatially downsampled by the spatial upsampling processor in the decoding apparatus of one Example by this invention. It is a figure which shows a mode that a 1st-order discrete wavelet reconstruction is performed in the spatial upsampling processor in the decoding apparatus of one Example by this invention.

  First, an encoding device and a decoding device according to an embodiment of the present invention will be described.

[Device configuration]
FIG. 1 is a diagram illustrating an encoding apparatus according to an embodiment of the present invention. The encoding device 1 of this embodiment includes a spatial downsampling processor 2 and an encoder 3. The spatial downsampling processor 2 includes a two-dimensional first-order discrete Wavelet decomposition processing unit 21, a band limitation processing unit 22, a two-dimensional first-order discrete Wavelet reconstruction unit 23, and a horizontal / vertical 1: 2 pixel thinning processing unit. 24. Data used for each process of the encoding device 1 according to an embodiment of the present invention can be appropriately stored in a storage unit (not shown) included in the encoding device 1.

  The two-dimensional first-order discrete Wavelet decomposition processing unit 21 performs horizontal and vertical directions for each frame image F (t) at time t of the moving image to be processed by a discrete wavelet decomposition method as an octave decomposition method of spatial frequency. ½ is subjected to first-order discrete wavelet decomposition, and the decomposed two-dimensional frequency component is sent to the band limitation processing unit 22 and the average value of all wavelet coefficients for each region of the decomposed frequency component (for example, square) The average value (the square root of the average value) is calculated, and each average value (MultiLL, MultiHL, MultiLH, MultiHH, which will be described later) is stored and held in the storage unit as “multiplier information” after decoding.

  The band limitation processing unit 22 applies a predetermined band limitation to the high spatial frequency component including the high frequency component in the horizontal or vertical direction, and two-dimensionally converts the low spatial frequency component and the high spatial frequency component subjected to the band limitation. Send to the floor discrete Wavelet reconstruction unit 23. The band restriction defined in advance for each of the high spatial frequency components will be described later.

  The two-dimensional first-order discrete wavelet reconstruction unit 23 performs first-order discrete wavelet reconstruction using the low spatial frequency component supplied from the band limitation processing unit 22 and the high spatial frequency component subjected to band limitation, and performs reconstruction. The processed image data is sent to the horizontal / vertical 1: 2 pixel thinning processing unit 24.

  The horizontal / vertical 1: 2 pixel decimation processing unit 24 performs pixel decimation on the reconstructed image data supplied from the two-dimensional first-order discrete wavelet reconstruction unit 23 in the horizontal and vertical directions at a ratio of 1: 2. The image data (reduced image signal) subjected to downsampling is sent to the encoder 3.

  The encoder 3 compresses and encodes the image data subjected to the spatial downsampling processing by the horizontal / vertical 1: 2 pixel thinning processing unit 24 using a predetermined encoding method (for example, JPEG, MPEG, H.264). Generate encoded data. This encoded data can be stored in the storage unit with the above-described multiple information as auxiliary information, or transmitted outside the apparatus.

  FIG. 2 is a diagram illustrating a decoding apparatus according to an embodiment of the present invention. The decoding device 5 of this embodiment includes a decoder 6 and a spatial upsampling processor 7. The spatial upsampling processor 7 includes a two-dimensional second-order discrete Wavelet decomposition processing unit 71, a high-frequency component estimation processing unit 72, and a two-dimensional first-order discrete Wavelet reconstruction unit 73. Data used for each process of the decoding device 5 according to an embodiment of the present invention can be appropriately stored in a storage unit (not shown) included in the decoding device 5.

  The decoder 6 reads or receives the encoded data encoded by the encoding apparatus 1 of the present embodiment, decodes it according to the corresponding compression encoding method, and decodes the decoded image data (space downsampled image). Is sent to the two-dimensional second-order discrete Wavelet decomposition processing unit 71. Note that the decoding device 5 of the present embodiment reads or receives multiple information as attached information and stores it in the storage unit.

  The two-dimensional second-order discrete Wavelet decomposition processing unit 71 performs two-dimensional second-order wavelet transform on the spatial downsampled image supplied from the decoder 6 and sends each frequency component to the high frequency component estimation processing unit 72.

  The high frequency component estimation processing unit 72 performs upsampling on each frequency component supplied from the two-dimensional second-order discrete wavelet decomposition processing unit 71 using an interpolation process defined in advance for each frequency component, Each high frequency component is estimated and reproduced by multiplying the multiple information, and each reproduced high frequency component is sent to the two-dimensional first-order discrete wavelet reconstruction unit 73. The interpolation process defined in advance for each frequency component will be described later.

  The two-dimensional first-order discrete wavelet reconstruction unit 73 performs first-order discrete wavelet reconstruction using the low spatial frequency component supplied from the high-frequency component estimation processing unit 72 and each reproduced high-frequency component, and the frame image F (t ) To restore the frame image F ′ (t) corresponding to.

  Next, an encoding device and a decoding device according to an embodiment of the present invention will be described.

[Device operation]
FIG. 3 is a flowchart showing the operation of the encoding apparatus according to an embodiment of the present invention. FIG. 4 is a flowchart showing the operation of the decoding apparatus according to an embodiment of the present invention.

  In step S <b> 1, the two-dimensional first-order discrete Wavelet decomposition processing unit 21 of the encoding device 1 according to the present embodiment reads out the 1-frame image F (t) at the time t of the moving image to be processed from the storage unit.

  In step S2, the two-dimensional first-order discrete Wavelet decomposition processing unit 21 of the encoding device 1 according to the present embodiment applies 1 to the read 1 frame image F (t) in the horizontal direction and the vertical direction by the discrete wavelet decomposition method. / 2 is subjected to first-order discrete wavelet decomposition, and an average value (for example, a square root of a square average value) of each decomposed frequency component region is calculated, and each average value (MultiLL, MultiHL, which will be described later) is calculated. MultiLH, MultiHH) is calculated.

  In step S <b> 3, the band limitation processing unit 22 of the encoding device 1 according to the present embodiment applies a predetermined band limitation to each of the high spatial frequency components including high frequency components in the horizontal or vertical direction.

  In step S4, the two-dimensional first-order discrete wavelet reconstruction unit 23 of the encoding device 1 of the present embodiment reconstructs the first-order discrete wavelet using the low spatial frequency component and the high spatial frequency component subjected to band limitation. Perform configuration.

  In step S5, the horizontal / vertical 1: 2 pixel decimation processing unit 24 of the encoding device 1 of the present embodiment performs pixel decimation by 1: 2 in the horizontal and vertical directions on the reconstructed image data, respectively. Image data (reduced image signal) subjected to downsampling is obtained.

  In step S6, the encoder 3 of the encoding apparatus 1 according to the present embodiment compresses and encodes the image data subjected to the spatial downsampling processing using a predetermined encoding method (for example, JPEG, MPEG, H.264). Encoded data is generated and stored or transmitted.

  Here, the difference between the conventional spatial downsampling method and the spatial downsampling process of the present embodiment will be described in more detail.

  FIG. 5A shows a low-pass filter applied for downsampling the horizontal sampling frequency in the conventional spatial downsampling method to 1/2 (similarly, the vertical sampling frequency to 1/2). An example of processing is shown. FIG. 5B shows a processing example in which sample point thinning is performed after applying a low-pass filter in the conventional spatial downsampling method. The reason why the low-pass filter is applied in the spatial downsampling method is that aliasing distortion as shown in FIG. 6 occurs when the low-pass filter is not applied.

  On the other hand, the spatial downsampling processor 2 in the encoding device 1 of the present embodiment performs wavelet decomposition using the discrete wavelet decomposition method as the octave decomposition method of the spatial frequency, and predefines each of the high spatial frequency components. Spatial downsampling is performed by band limiting.

That is, as shown in FIG. 7A, first, the two-dimensional first-order discrete Wavelet decomposition processing unit 21 converts the original image corresponding to the frame image F (t) into a two-dimensional image in half in the horizontal and vertical directions. Perform first-order discrete wavelet decomposition. Thus, each frequency component ^{OrgLL 1, OrgLH 1, OrgHL 1} , OrgHH 1 is obtained.

Next, the two-dimensional first-order discrete Wavelet decomposition processing unit 21 averages representative (preferably all) elements (wavelet coefficients) for each region of each frequency component OrgLL ¹ , OrgLH ¹ , OrgHL ¹ , OrgHH ^1. A value (for example, the square root of the root mean square) is calculated and stored in the storage unit as “multiple information” MultiLL, MultiHL, MultiLH, and MultiHH after decoding.

Next, the band limitation processing unit 22 performs band limitation of OrgHL ¹ which is a horizontal high frequency / vertical low frequency component. This region of OrgHL ¹ includes high frequency components that are ½ or more of the horizontal sampling frequency in the horizontal direction. Therefore, as a band limitation, OrgHL ¹ performs one-dimensional first-order discrete wavelet decomposition in the horizontal direction and converts the high frequency component side (Hs * 3/4 or more) to 0. This state is shown in FIG. 7 (a). As shown in FIG. 7 (b), this is a high-frequency component that is ½ or more of the horizontal sampling frequency in the horizontal direction and less than Hs * 3/4. This produces the same effect as a band-pass filter that performs the band-pass.

Next, the band limitation processing unit 22 performs band limitation of OrgLH ¹ which is a vertical low frequency / horizontal high frequency component. This region of OrgLH ¹ includes a high frequency component that is 1/2 or more of the vertical sampling frequency in the vertical direction. Therefore, as a band limitation, OrgHL ¹ performs one-dimensional first-order discrete wavelet decomposition in the vertical direction and converts the high frequency component side (Vs * 3/4 or more) to 0. This state is shown in FIG. 8 (a), and as shown in FIG. 8 (b), this is a high-frequency component of ½ or more of the vertical sampling frequency in the vertical direction and less than Vs * 3/4. This produces the same effect as a band-pass filter that performs the band-pass.

Next, the band limitation processing unit 22 limits the band of OrgHH ¹ which is a horizontal high frequency / vertical high frequency component. This region of OrgLH ¹ includes a high frequency component that is 1/2 or more of the sampling frequency in both the horizontal and vertical directions. Therefore, as a band limitation, this OrgHH ¹ has a spatial anisotropy characteristic of vision (the spatial frequency-contrast sensitivity in the direction of 45 degrees of visual angle is compared with the spatial frequency-contrast sensitivity in the direction of horizontal 0 degree and vertical 90 degrees. In consideration of (the sensitivity is about ½), all the horizontal high-frequency and vertical high-frequency components are converted to zero.

As described above, the band limitation processing unit 22 of the encoding device 1 according to the present embodiment performs band limitation that is defined in advance for each of the high spatial frequency components including high frequency components in the horizontal or vertical direction. Subsequently, the two-dimensional first-order discrete wavelet reconstruction unit 23 of the encoding device 1 of the present embodiment performs first-order discrete wavelet reconstruction for each of the high spatial frequency components subjected to the band limitation and the low spatial frequency component OrgLL ^1. After the configuration, the horizontal and vertical 1: 2 pixel thinning processing unit 24 performs pixel thinning 1: 2 in the horizontal and vertical directions, thereby obtaining a spatially downsampled image SpDw. FIG. 9A shows a spatial downsampled image SpDw, and FIGS. 9B and 9C include aliasing components at 1/4 or more of the sampling frequency in the horizontal or vertical direction. The state of the signal gain of the image signal of the image SpDw that has been spatially downsampled is shown.

  That is, the image SpDw obtained by the spatial downsampling process of the present embodiment is an image that intentionally includes the aliasing component. However, for example, compared with an image that is simply folded at Hs / 2 and Vs / 2 as in the conventional case, the aliasing component is included in the spatial frequency components of Hs / 4 and Vs / 4 or less that occupy many power components of the image. Absent. This gives rise to the effect of eliminating the high-frequency aliasing component that appears visually in the low frequency range and is conspicuous. In addition, high frequency components can be reliably reproduced at the time of high frequency estimation at the time of spatial upsampling described later, which is advantageous for encoding.

  With reference to FIG. 4, in step S21, the decoder 6 of the decoding device 5 of the present embodiment reads or receives the encoded data encoded by the encoding device 1 of the present embodiment, and responds. Decode according to the compression encoding method.

  In step S22, the two-dimensional second-order discrete Wavelet decomposition processing unit 71 of the decoding device 5 according to the present embodiment performs two-dimensional second-order wavelet transform on the spatial downsampled image.

  In step S23, the high frequency component estimation processing unit 72 of the decoding device 5 of the present embodiment performs upsampling using interpolation processing, and multiplies multiple information as auxiliary information to estimate and reproduce each high frequency component. .

  In step S24, the two-dimensional first-order discrete Wavelet reconstruction unit 73 of the decoding device 5 of the present embodiment executes first-order discrete wavelet reconstruction using the low spatial frequency components and the reproduced high-frequency components, and the frame A frame image F ′ (t) corresponding to the image F (t) is restored.

  As described above, the spatial upsampling processor 7 in the decoding device 5 according to the present embodiment first performs the spatial downsampling on the image SpDw by using the two-dimensional second-order discrete Wavelet decomposition processing unit 71 as shown in FIG. Perform second-order wavelet decomposition.

Next, the high frequency component estimation processing unit 72 estimates a high spatial frequency component in the image SpDw. The image SpDw is an image including a spatial frequency aliasing component as shown in FIG. 1F discrete wavelet decomposition component images SpDw SpDwHL ¹ includes a first floor discrete wavelet decomposition component OrgHL ¹ of Hs / 2~Hs * 3 / aliasing components in the fourth region of the original image Org. In addition, SpDwLH ¹ includes a folding component of the Vs / 2 to Vs * 3/4 region of OrgLH ¹ . SpDwHH ¹ includes both the folded components of OrgHL ¹ and OrgLH ¹ .

Therefore, as shown in FIG. 11, as the horizontal high frequency / vertical flat low frequency component estimation processing of the original image Org (illustrated 111), the second-order horizontal high frequency / vertical low frequency component SpDwHL ² (illustrated 113) is horizontal / vertical. Spatial upsampling is performed by using interpolation processing by a linear filter to quadruple each direction and 1st floor horizontal high frequency / vertical low frequency component SpDwHL ¹ (112 in the figure) to double each in the horizontal and vertical directions. At this time, in the spatial upsampling of each of the frequency components (SpDwHL ² and SpDwHL ¹ ) subjected to the spatial upsampling, the phase is shifted by 1/4. The process of shifting the phase has the effect of preventing the occurrence of moire due to folding. Then, the component obtained by spatially up-sampling SpDwHL ² and the component obtained by spatially up-sampling SpDwHL ¹ are multiplied and further multiplied by a multiple MultiHL (114 in the figure). This result is stored as a spatial upsampling horizontal high frequency candidate UPHL (115 in the figure). This is because the normal frequency spectrum continuously decreases while maintaining the correlation from the low frequency toward the high frequency, and therefore, if there is a high frequency component in the high frequency region to be estimated, the correlation between SpDwHL ¹ and SpDwHL ² is high. Furthermore, since SpDwHL ¹ includes a higher frequency aliasing component as described above, the correlation is higher.

Next, as shown in FIG. 12, as the vertical high frequency / horizontal low frequency component estimation processing of the original image (121), the second-order vertical high frequency / horizontal low frequency component SpDwLH ² (123) is horizontal / vertical direction. In addition, spatial upsampling is performed by using interpolation processing by a linear filter to quadruple each of the first-floor vertical high frequency / horizontal low frequency components SpDwLH ¹ (122 in the figure) in the horizontal and vertical directions. At this time, the phase is shifted by 1/4 in the spatial upsampling of the frequency components (SpDwLH ² and SpDwLH ¹ ) subjected to the spatial upsampling. The process of shifting the phase has the effect of preventing the occurrence of moire due to folding. Then, the component obtained by spatially up-sampling SpDwLH ² and the component obtained by spatially up-sampling SpDwLH ¹ are multiplied and further multiplied by a multiple MultiLH (shown in FIG. 124). This result is stored as a spatial upsampling horizontal high frequency candidate UpLH (125 in the figure). Since the normal frequency spectrum continuously decreases while maintaining the correlation from the low frequency toward the high frequency, the correlation between SpDwLH ¹ and SpDwLH ² is high when a high frequency component exists in the high frequency region to be estimated. Furthermore, since SpDwLH ¹ includes a higher frequency aliasing component as described above, the correlation is higher.

Next, in the case of estimation of the oblique high spatial frequency component, that is, the vertical high frequency / horizontal high frequency component SpDwHH ¹ , the second-order vertical high frequency / horizontal high-frequency component SpDwHH ² is quadrupled in the horizontal / vertical direction and the first-order vertical high frequency The horizontal high-frequency component SpDwHH ¹ is doubled in the horizontal and vertical directions, and spatial upsampling is performed using interpolation processing by a linear filter. At this time, in the spatial upsampling of SpDwHH ² and SpDwHH ¹ , the phase is shifted by 1/4. Then, the component obtained by spatially up-sampling SpDwHH ² and the component obtained by spatially up-sampling SpDwHH ¹ are multiplied, and further multiplied by a multiple MultiHH. This result is stored as a spatial upsampling horizontal high frequency candidate UPHH.

After performing the above-described estimation processing of the high spatial frequency component SpDwHH ¹ in the horizontal, vertical, and diagonal directions, as shown in FIG. 13, the two-dimensional first-order discrete Wavelet reconstruction unit 73 performs the original image Org and the high spatial frequency estimation component. First-order discrete wavelet reconstruction is performed using UpHL, UpLH, UppH.

  According to the encoding device 1 and the decoding device 5 of the present embodiment, processing for performing spatial frequency down-sampling and up-sampling, for example, down-sampling the spatial frequency in advance, image compression encoding and transmission, decoding, and spatial up By performing sampling, it is possible to greatly reduce the transmission capacity with little deterioration of visual characteristics. This is because it is used before / after the compression encoding of a digital cinema image that is compressed and encoded on a frame basis in a moving image system, for example, JPEG2000, which is becoming increasingly high definition today. By using it before / after the compression encoding of the super high-definition image for which compression encoding is being studied in H.264, the transmission capacity can be greatly reduced.

  Note that each of the spatial downsampling processor 2 and the spatial upsampling processor 7 according to the present embodiment can be configured as a computer, and a program for realizing each function can be stored in a storage unit of the computer. The functions of the spatial down-sampling processor 2 and the spatial up-sampling processor 7 can be realized by appropriately reading out and executing the program by the central processing unit (CPU).

  In the above embodiment, one frame image of a moving image has been representatively described, but the present invention can also be applied to still image compression. Further, although the encoding device and the decoding device have been described as including a spatial downsampling processor and a spatial upsampling processor, respectively, the encoder and the spatial downsampling processor are separate devices, and the decoder and the spatial upsampling processor The sampling processor can be configured as a separate device. Furthermore, the spatial upsampling processor can perform spatial upsampling on the reduced image signal output from the spatial downsampling processor. Accordingly, the present invention is not limited to the above-described embodiments, but is limited only by the description of the scope of claims.

  According to the present invention, after performing spatial downsampling that intentionally includes aliasing with less aliasing in low frequency components, compression encoding / transmission / decoding is performed, and spatial upsampling is performed, so significant transmission band compression is achieved. Therefore, it is useful for applications such as compression coding and transmission in digital cinema and Super Hi-Vision that require a large transmission capacity.

DESCRIPTION OF SYMBOLS 1 Encoding apparatus 2 Spatial downsampling processor 3 Encoder 5 Decoding apparatus 6 Decoder 7 Spatial upsampling processor 21 Two-dimensional 1st-order discrete Wavelet decomposition processing unit 22 Band limitation processing unit 23 Two-dimensional first-order discrete Wavelet reconstruction Unit 24 Horizontal / Vertical 1: Two-pixel thinning processing unit 71 Two-dimensional second-order discrete Wavelet decomposition processing unit 72 High-frequency component estimation processing unit 73 Two-dimensional first-order discrete Wavelet reconstruction unit

Claims (8)

A spatial downsampling processing device that spatially downsamples a spatial frequency component obtained by wavelet decomposition of a moving image,
For the two-dimensional image to be processed in the moving image, the first-order discrete wavelet decomposition is performed in the horizontal direction and the vertical direction, and the wavelet coefficients for each frequency component in the horizontal direction and the vertical direction subjected to the first-order discrete wavelet decomposition A two-dimensional first-order discrete wavelet decomposition processing unit that calculates the average value of
A band limitation processing unit that performs a predetermined band limitation on the high spatial frequency components in the horizontal and vertical spatial frequency components of the two-dimensional image;
A two-dimensional first-order discrete Wavelet reconstruction unit that performs first-order discrete wavelet reconstruction using a low spatial frequency component supplied from the band limitation processing unit and a high spatial frequency component subjected to the band limitation;
A pixel decimation processing unit that performs spatial downsampling by performing pixel decimation in the horizontal direction and vertical direction of the reconstructed image data, and generates a reduced image signal,
A spatial downsampling processing apparatus comprising:   The band limiting processing unit performs one-dimensional first-order discrete wavelet decomposition in the horizontal direction of the horizontal high-frequency / vertical low-frequency component when the band limitation of the horizontal high-frequency / vertical low-frequency component is performed, and sets the high-frequency component side to 0 If the vertical low frequency and horizontal high frequency components are band-limited, one-dimensional first-order discrete wavelet decomposition is performed in the vertical direction of the vertical low frequency and horizontal high frequency components, and the high frequency component side is converted to zero. 2. The spatial downsampling processing apparatus according to claim 1, wherein when the band limitation of the horizontal high-frequency / vertical high-frequency components is performed, all of the horizontal high-frequency / vertical high-frequency components are converted to zero. The spatial downsampling processing device according to claim 1 or 2,
An encoder that compresses and encodes a reduced image signal output from the spatial downsampling processing device using a predetermined encoding method to generate encoded data;
An encoding device comprising: A spatial upsampling processing device that performs spatial upsampling on image data composed of reduced image signals generated by the spatial downsampling processing device according to claim 1,
A two-dimensional second-order discrete Wavelet decomposition processing unit that performs two-dimensional second-order wavelet transform on the image data;
Upsampling is performed from each frequency component supplied from the two-dimensional second-order discrete wavelet decomposition processing unit using interpolation processing defined in advance for each frequency component, and each high frequency component is estimated by multiplying the multiple information. A high-frequency component estimation processing unit that reproduces
A two-dimensional first-order discrete wavelet reconstruction is performed in which first-order discrete wavelet reconstruction is performed using the low spatial frequency component supplied from the high-frequency component estimation processing unit and each reproduced high-frequency component, and the image to be processed is restored. A component,
A spatial upsampling processing apparatus comprising: The high-frequency component estimation processing unit is an interpolation process defined in advance for each frequency component,
In the case of estimation of horizontal high-frequency / vertical low-frequency components, spatial upsampling was performed on each of the second-order horizontal high-frequency / vertical low-frequency components and the first-floor horizontal high-frequency / vertical low-frequency components, and the spatial up-sampling was performed. Multiply by shifting the phase of each frequency component and multiplying the corresponding frequency component multiple information,
In the case of estimation of vertical high frequency / horizontal low frequency components, spatial upsampling was performed on each of the second floor vertical high frequency / horizontal low frequency components and the first floor vertical high frequency / horizontal low frequency components, and the spatial upsampling was performed. Multiply by shifting the phase of each frequency component and multiplying the corresponding frequency component multiple information,
In the case of estimation of vertical high frequency / horizontal high frequency components, spatial upsampling is applied to each of the second floor vertical high frequency / horizontal high frequency component and the first floor vertical high frequency / horizontal high frequency component, and the frequency upsampling of the spatial upsampling is performed. 5. The spatial upsampling processing apparatus according to claim 4, wherein multiplication is performed by shifting each phase, and multiplication is performed by multiple information of a corresponding frequency component. The spatial upsampling processing device according to claim 4 or 5,
A decoder for decoding encoded data encoded by the encoding device according to claim 3;
A decoding apparatus comprising: In a computer configured as a spatial downsampling processing device that spatially downsamples a spatial frequency component obtained by wavelet decomposition of a moving image,
The two-dimensional image to be processed in the moving image is subjected to the first-order discrete wavelet decomposition in the horizontal direction and the vertical direction, and each wavelet for each frequency component region in the horizontal direction and the vertical direction subjected to the first-order discrete wavelet decomposition. Calculating an average value of coefficients as multiple information used for estimating a high frequency component at the time of spatial upsampling;
Applying a predetermined band limitation to each of the high spatial frequency components in the horizontal and vertical spatial frequency components of the two-dimensional image;
Performing first-order discrete wavelet reconstruction using the low spatial frequency component supplied from the step and the high spatial frequency component subjected to the band limitation;
Performing spatial downsampling by performing pixel decimation in the horizontal direction and vertical direction of the reconstructed image data, and generating a reduced image signal;
A program for running A computer configured as a spatial upsampling processor that performs spatial upsampling on image data composed of reduced image signals generated by the spatial downsampling processor according to claim 1 or 2,
Performing a two-dimensional second-order wavelet transform on the image data;
For each frequency component supplied from the step, performing upsampling using an interpolation process defined in advance for each frequency component, multiplying the multiple information, estimating and reproducing each high frequency component,
Performing a first-order discrete wavelet reconstruction using the low spatial frequency component supplied from the step and each reproduced high-frequency component, and reconstructing the two-dimensional image to be processed;
A program for running

Images