JP2011254219A - Image coding method, image decoding method, image coding device, image decoding device, and program for the method and device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To realize coding which reduces the amount of codes while retaining a subjective image quality.SOLUTION: The resolution of an image signal of an original signal is converted to a low resolution (S10), and a signal expressed by a resolution lower than that of the original signal (structure component) is subjected to coding processing based on waveform coding (S11). A signal of a texture component obtained by removing a signal of the structure component from the original signal is subjected to coding processing based on texture analysis (S16), and coded data by the waveform coding processing and coded data by the texture analysis are integrated to a coded stream.

Description

  The present invention relates to a high-efficiency image signal encoding method and decoding method.

  H. In the current encoding method represented by H.264 / AVC, a prediction signal is generated by intra-frame prediction, inter-frame prediction, and inter-layer prediction, and a residual signal between the prediction signal and the original signal is to be encoded. Such conventional techniques aim to accurately reproduce the original signal waveform. However, in such a conventional method, since the code amount is controlled by quantization under the waveform reproduction framework, image quality degradation such as block distortion and ringing becomes obvious at a low rate.

  Further, since the prediction process refers to the decoded signal, at a low rate, the deteriorated decoded signal is referred to, and it is impossible to predict a high frequency component above a certain level. For this reason, the prediction performance deteriorates for an image containing many high-frequency components. For example, such low-rate image quality degradation is noticeable in scenes such as water surfaces where a rigid body movement model is not established in the case of inter-frame prediction, and in the case of intra-frame prediction and inter-layer prediction, it is noticeable in the texture area of a fine pattern. Become.

  On the other hand, a technique called texture synthesis is being studied. This is a technique for synthesizing an image based on a predetermined generation rule using given data (referred to as a seed). Texture synthesis does not pursue accurate waveform reproduction of the original signal, but differs from the conventional general coding method in that the feature amount of the texture in the original signal is to be reproduced.

  As a texture synthesis method, for example, there is a method described in Non-Patent Document 1. In this method, a histogram of coefficients obtained as an output of the directional filter bank is used as a texture feature value as an input of the texture synthesis process.

D. Heeger, and J. Bergen, “Pyramid-Based Texture Analysis / Synthesis”, Proc ACM SIGGRAPH 95, 1995.

  In general, the amount of seed data used in texture synthesis can be suppressed to be extremely small compared to the amount of data of the original signal. Therefore, application of texture synthesis to image coding is expected.

  On the other hand, the similarity between the signal generated by texture synthesis and the original signal is not guaranteed from the viewpoint of waveform reproduction. This is because there is no guarantee that the phase information in the image is retained. For this reason, if it is required to obtain a signal with an image quality that is subjectively similar to the original signal on the premise of comparison with the original image, even if the texture synthesis method is simply applied to image coding, the request Can not respond to.

  The present invention has been made in view of such circumstances, and as a low-rate encoding method, a method that replaces the conventional waveform reproduction-based encoding method, based on texture synthesis technology, is subjectively close to the original image quality. It aims at establishing the process which produces | generates the signal of.

  Waveform reproduction using only texture synthesis has its limitations due to its processing structure. This is because the phase information in the image is not retained in texture synthesis. In the present invention, a conventional waveform reproduction-based encoding method and texture synthesis are combined, and an approach is taken in which the weak points of both are supplemented by the advantages of each other. For this purpose, the image signal is divided into components that represent the general shape of the waveform (structural components) and other components (texture components), and waveform reproduction based coding is applied to the structural components. Encoding by texture analysis and texture synthesis is applied.

  That is, according to the present invention, in order to reduce the code amount while maintaining the subjective image quality, the low frequency component of the image signal to be encoded is the structural information, the high frequency component is the texture information, and the image signal is the structural component and the texture component. The structural component is subjected to coding processing based on waveform coding, and the texture component is subjected to coding processing based on texture synthesis. In the case of information that is difficult to visually detect deterioration of high frequency components, a large code amount reduction effect can be expected.

  Specifically, the image encoding apparatus according to the present invention first performs encoding processing based on waveform encoding on a signal expressed at a low resolution by resolution conversion of an original signal to generate encoded data. The encoded data is decoded, and a signal having the same resolution as the original signal is restored by inverse resolution conversion. A difference signal between the restored signal and the original signal is calculated, and texture analysis is performed on the difference signal, so that the signal is expressed by a histogram of pixel values and a histogram group of transform filter coefficients, and is encoded by texture analysis. Generate data. The encoded data by the above waveform encoding process and the encoded data by texture analysis are integrated and output as an encoded stream.

  Alternatively, the image coding apparatus according to the present invention first performs coding processing based on waveform coding on a signal expressed at a low resolution by resolution conversion of an original signal to generate coded data. The encoded data is decoded, and a signal having the same resolution as the original signal is restored by inverse resolution conversion. The difference signal between the restored signal and the original signal is calculated, a frequency band pass filter is applied to the difference signal, and a histogram of pixel values and transform filter coefficients are applied to the difference signal with a limited frequency band by texture analysis. The encoded data is generated by texture analysis. The encoded data by the above waveform encoding process and the encoded data by texture analysis are integrated and output as an encoded stream.

  Note that, in the above-described image encoding, the representation accuracy of the histogram obtained by texture analysis can be controlled according to the amount of restriction of the frequency band for the difference signal.

  In the present invention, an image signal can also be encoded as follows. First, the input image is separated into a low frequency band signal and a high frequency band signal, and the low frequency band signal is subjected to an encoding process based on waveform encoding to generate encoded data. The high frequency band signal is expressed by a histogram of pixel values and a histogram group of transform filter coefficients by texture analysis to generate encoded data by texture analysis. The encoded data by the above waveform encoding process and the encoded data by texture analysis are integrated and output as an encoded stream.

  In addition, the image decoding apparatus of the present invention receives encoded data obtained by waveform encoding processing, generates a decoded signal by decoding processing, and has the same resolution as the original signal by inverse resolution conversion for the decoded signal. Restore as signal. On the other hand, encoded data obtained by texture analysis is input, and residual signals that could not be expressed by waveform encoding processing are decoded by texture synthesis processing. A decoded signal is obtained by integrating these two signals.

  According to the present invention, it is possible to realize encoding that reduces the code amount while maintaining subjective image quality.

It is a figure which shows the structural example of an image coding apparatus. It is a figure which shows the structural example of the analysis filter of Steerable pyramid. It is a figure which shows the other structural example of an image coding apparatus. It is a figure which shows the other structural example of an image coding apparatus. It is a figure which shows the structural example of an image decoding apparatus. It is a figure which shows the structural example of the synthesis filter of Steerable pyramid. It is a flowchart of an encoding process. It is a flowchart of a texture analysis process. It is a flowchart of a decoding process. It is a flowchart of a texture synthetic | combination process.

  Hereinafter, an outline of processing performed in the present embodiment will be described.

  FIG. 1 shows a configuration example of an image coding apparatus according to an embodiment of the present invention. The flow of the encoding process will be described with reference to FIG. First, the image encoding device 1 inputs an image signal (X × Y pixel) to be encoded. The resolution converter 10 converts the input image signal into a reduced image signal of X ′ × Y ′ pixels by resolution conversion processing. Here, X ′ and Y ′ are integers satisfying X ′ ≦ X and Y ′ ≦ Y. As the resolution conversion process, a spatial filter and subsampling are used. Spatial filters used here can be broadly classified into separable filters and non-separable filters.

The separable filter is a filter that processes a 2N + 1 tap filter having coefficients (a _−N , a _{−N + 1} ,..., A _N−1 , a _N ) independently in the horizontal and vertical directions. The non-separable filter applies a (2N + 1) × (2N + 1) tap two-dimensional filter defined as follows to the target signal.

  Sub-sampling is a process of thinning out signals for every r pixels according to the resolution ratio r.

  The reduced image signal obtained as the output of the resolution conversion process is the structural component in this method. Therefore, the waveform encoding processing unit 11 performs waveform encoding processing on the image signal (X ′ × Y ′ pixel) which is a structural component, and outputs an encoded stream. As the waveform encoding process, H.264 has been described. H.264 / AVC-compliant encoding scheme is used. In the present invention, the waveform encoding process is H.264. For example, methods such as JPEG, JPEG2000, and MPEG-2 can be used.

  The decoding processing unit 12 applies H.264 to the encoded stream obtained as the output of the waveform encoding process. A decoding process defined by H.264 / AVC is performed to obtain a decoded signal. As an encoding method, H.264 is used. When a method other than H.264 / AVC (JPEG, JPEG2000, MPEG-2, etc.) is used, a corresponding decoding process is performed.

  The inverse resolution conversion unit 13 performs an inverse resolution conversion process on the decoded signal (X ′ × Y ′ pixel) obtained as an output of the decoding process, and an image signal (X × Y pixel) having the same resolution as the original image. As a result, a decoded signal of the structural component is obtained.

The difference signal generation unit 14 calculates a difference signal between the image signal of the original signal and the decoded signal (X × Y pixel) of the structural component. This difference signal becomes a texture component in this method. The texture analysis processing unit 15 performs texture analysis processing on the texture components, and extracts the following two types of histograms as information necessary for texture synthesis at the time of decoding. One is a difference signal histogram. The other is a histogram for transform coefficients obtained by Steerable pyramid transform (see Reference 1) for differential signals.
[Reference 1]: EP Simoncelli and WT Freeman, “The Steerable Pyramid: A Flexible Architecture for Multi-Scale Derivative Computation”, IEEE Int'l Conf. On Image Processing. Washington DC, October 1995.
Steerable pyramid transform is a kind of directional filter bank, and its output is defined by the number of directions and the decomposition level. When Steerable pyramid transformation with the number of directions M and decomposition level J _max is performed with the original signal as 2 ^J × 2 ^J pixels, M types of direction components are used at the jth decomposition level (0 ≦ j ≦ J _max −1). Each directional component has 2 ^JJ × 2 ^Jj conversion coefficients. At the highest decomposition level j = J _max , there is no division with respect to the direction component, and 2 ^J−Jmax × 2 ^J−Jmax conversion coefficients. In addition to the above, 2 ^J × 2 ^J conversion coefficients are provided as high frequency components.

  Further, since the transform coefficient is expressed by a real value, each direction component for each decomposition level is quantized with a predetermined quantization width K, and a histogram for the quantized transform coefficient is used as texture analysis information.

As an example, FIG. 2 shows a configuration example of an analysis filter for steerable pyramid conversion when M = 2 and J _max = 1. The input of the high-pass filter 21 and the low-pass filter 22 is an image signal (difference signal). This analysis filter performs specified level decomposition and direction decomposition on the input image signal, and calculates filter coefficients as high-frequency components and low-frequency components. The high-pass filter 21 outputs a high-pass filter coefficient, the first-direction component pass filter 231 outputs a low-pass filter + first-direction component pass filter coefficient, and the second-direction component pass filter 232 The coefficient of the low-pass filter + first-direction component pass filter is output, and the low-pass filter + downsampled coefficient is output from the low-pass filter 233 and the downsampling unit 234.

  When increasing the decomposition level, the processing of the portion 23 surrounded by the broken line is recursively repeated for the downsampled signal output from the downsampling unit 234.

  FIG. 3 shows another configuration example of the image encoding device. In the image encoding device 1-1 illustrated in FIG. 3, a high-pass filter processing unit 16 is provided between the difference signal generation unit 14 and the texture analysis processing unit 15 with respect to the image encoding device 1 illustrated in FIG. 1. Is provided. Other components are the same as those of the image encoding device 1 shown in FIG. The high-pass filter processing unit 16 applies a high-pass filter to the difference signal to limit the target of texture analysis processing to the high-frequency component of the difference signal. In this case, the high-pass filter can be set based on the frequency characteristics of the filter used in the resolution conversion process.

Here, the histogram representation methods are organized. When the histogram is expressed as h _B [i] (i = 0, 1,..., B−1), h _B [i] stores the number of samples whose target signal value is i. At this time, B is called the bin number of the histogram, and h _B [i] is called the frequency value of the i-th bin.

  As described above, by limiting the frequency band targeted for texture synthesis processing, the power of the differential signal is also limited according to the bandwidth. As a result, it is possible to reduce the amount of information expressing the histogram compared to before the bit restriction.

  Also for the histogram for the conversion coefficient of Steerable pyramid conversion, the number of bins of the histogram is limited based on the power of the image after resolution conversion. Specifically, the bin number of the histogram is set to be smaller as the power becomes smaller. This is because the number of bins in the histogram of the transform coefficient is always kept constant considering that the bit depth required for the transform coefficient also changes according to the frequency component of the signal before conversion and the frequency characteristics of the filter used for resolution conversion. This is because when the power of the image after the resolution conversion is small, there is a possibility that a redundant expression is given to the conversion coefficient.

A histogram for the transformation coefficient of the m-th (m = 0, 1,..., M−1) direction component at the j-th decomposition level (0 ≦ j ≦ J _max −1) in the Steerable pyramid transformation is expressed as L _x ^{(j, m)} [ i] (i = 0,..., 2 ^{2 (Jj)} −1) [where subscript X is a notation instead of Bδ (δ is a subscript), and so on). As described above, at the highest decomposition level j = J _max , there is no division related to the direction component, and it is limited to the histogram corresponding to m = 0. In addition to the above, since there are 2 ^J × 2 ^J conversion coefficients as high-frequency components, a histogram for storing these high-frequency components is represented as H _J [i] (i = 0,..., 2 ^2J −1). And

A histogram for transform coefficients of m-th (m = 0, 1,..., M−1) direction components at the j-th decomposition level (0 ≦ j ≦ J _max −1) is expressed as L _x ^{(j, m)} [i] (i = 0,..., 2 ^{2 (Jj)} −1) means that the conversion coefficient expressed by a real value is approximated by Bδ types of discrete values. In this discretization, the quantization width to be used is q _{j, m} , and the m-th (m = 0, 1,..., M−1) direction component conversion at the j-th decomposition level (0 ≦ j ≦ J _max −1). Assuming that the coefficient c _{j, m} [k] (k = 0,..., 2 ^{2 (Jj)} −1), the approximation error _Ex ^{(j, m)} accompanying this discretization is expressed as follows: .

In this equation, the value x sandwiched between the “L-shaped symbol” and the “left-right inverted L-shaped symbol” means rounding off the decimal point with respect to the real number x. In addition, the approximation error for high frequency components is expressed as follows for convenience.

Note that m = M used here is not used to identify a direction component but as an index representing a high-frequency component.

  The following equation is defined as the sum of approximation errors for the omnidirectional components at all resolution levels.

As the number of bins increases, the approximate error sum E _total (Bδ, δ) decreases monotonously. Therefore, when setting the number of bins, a threshold T is set for Equation (2), a plurality of values as candidates for the number of bins are prepared, and the maximum value satisfying the following equation among the values included in the candidates is set. Is the number of bins.

E _total (Bδ, δ) ≦ T
FIG. 4 shows still another configuration example of the image encoding device. In the image encoding device 1-2 shown in FIG. 4, the band separation unit 101 separates the input image signal into a low frequency band signal and a high frequency band signal. The waveform encoding processing unit 102 performs an encoding process based on waveform encoding on the low frequency band signal to generate encoded data. On the other hand, the texture analysis processing unit 103 performs texture analysis on the high frequency band signal and generates encoded data of texture analysis information in which the high frequency band signal is expressed by a histogram group of pixel values and a histogram group of conversion filter coefficients. The encoded data by these waveform encoding processes and the encoded data by texture analysis are integrated and output as an encoded stream.

  As described above, the advantages of the configuration of the image encoding device 1-2 in FIG. 4 that uses the method of separating and processing the high-frequency band signal and the low-frequency band signal as resolution conversion are as follows. The point is that processing can be performed in parallel. Therefore, in the encoding process of FIGS. 1 and 3, the texture synthesis process cannot be executed until the waveform encoding process / decoding process is completed, whereas in the configuration of FIG. 4, both can be processed in parallel. In an environment where parallel processing is possible, an improvement in processing speed can be expected.

  Decoding processing will be described with reference to FIG. FIG. 5 shows a configuration example of the image decoding apparatus. The image decoding device 3 includes a texture synthesis processing unit 30, a decoding processing unit 31, an inverse resolution conversion unit 32, and a decoded signal generation unit 33.

  In the decoding process, two types of encoded information are input. First, the decoding processing unit 31 performs a corresponding decoding process on the encoded stream obtained as the output of the waveform encoding process, and generates a decoded signal. The inverse resolution conversion unit 32 performs an inverse resolution conversion process on the decoded signal (X ′ × Y ′ pixel) obtained as an output of the decoding process to obtain a decoded signal (X × Y pixel) of the structural component.

  On the other hand, the texture synthesis processing unit 30 receives the texture analysis information and decodes the texture information based on the following texture synthesis method. In the encoding process, when a band limiting process using a high-pass filter as shown in FIG. 3 is added, the target of the texture synthesis / analysis process is limited to frequency components in a certain band.

  The method of Non-Patent Document 1 is used as the texture synthesis method. In this method, a histogram of coefficients obtained as an output of a directional filter bank called Steerable pyramid is used as an input for texture synthesis processing. The outline of this method is shown below.

  The input of the method includes an appropriate initial image (white noise image in Non-Patent Document 1), a pixel value histogram for a target texture image (hereinafter referred to as a target texture image), and each of the two images. Is a histogram of each subband. Steerable pyramid is used for subband decomposition. An image obtained as an output of texture synthesis is called a synthesized image. Note that, at the start of the following processing, the composite image is initialized with the initial image.

[Texture synthesis method]
(1) A white noise image as an initial image of the processing target image, pixel value histograms for the target texture image, and a histogram of conversion coefficients for each subband (referred to as a subband histogram) for both images are input. To do.
(2) Based on the pixel value histogram of the composite image (initially the initial image) and the pixel value histogram of the target texture image, the pixel value histogram of the composite image is brought close to that of the target texture image.
(3) A composite image is generated using the pixel value histogram of the composite image obtained in (2) above.
(4) Using the composite image obtained in (3) above as an input, the directional filter bank is forward-converted with respect to the image and a conversion coefficient is calculated. A histogram of conversion coefficients is generated for each subband.
(5) Using the subband histograms of both the composite image and the target texture image as input, the histogram of each subband of the composite image is brought close to the subband histogram of the same band of the target texture image.
(6) Using the subband coefficient of the composite image corrected in (5) above as an input, the directional filter bank is inversely transformed and a composite image is output.
(7) The processes (2) to (6) are repeated a certain number of times.

  As a technique for obtaining subband coefficients in texture synthesis, application of a filter bank other than the above, such as wavelet, is also being studied. The texture feature amount prediction mechanism according to the present invention is not limited to the above-described directional filter bank, and can be similarly applied to other filter banks.

As an example, FIG. 6 shows a configuration example of a synthesis filter for the steerable pyramid transformation when M = 2 and J _max = 1. This synthesis filter synthesizes an image signal from the input filter coefficients. The high-frequency synthesis filter 42 receives the filter coefficient output from the high-pass filter 21 shown in FIG. 2 and synthesizes an image signal. The first direction component synthesis filter 401 receives the coefficients of the low-pass filter and the first direction component pass filter output from the first direction component pass filter 231 shown in FIG. The second direction component synthesis filter 402 receives the coefficients of the low-pass filter and the second direction component pass filter output from the second direction component pass filter 232 shown in FIG. The up-sampling unit 403 receives the low-pass filter output from the down-sampling unit 234 shown in FIG. 2 and the down-sampled coefficient and performs up-sampling, and the low-frequency component synthesis filter 404 calculates the image signal from the up-sampled coefficient. Is synthesized. The low frequency synthesis filter 41 synthesizes the outputs of the first direction component synthesis filter 401, the second direction component synthesis filter 402, and the low frequency component synthesis filter 404. From the outputs of the high-frequency synthesis filter 42 and the low-frequency synthesis filter 41, the final differential image signal is synthesized.

  When increasing the decomposition level, the process of the portion 40 enclosed by the broken line is recursively repeated.

  Hereinafter, a specific processing flow of the present embodiment will be described.

[Flowchart (encoding process)]
FIG. 7 is a processing flowchart of the image encoding device 1-1 shown in FIG. Other embodiments can be similarly processed.

  (1) An image signal (X × Y pixels) is input, converted into a reduced image signal of X ′ × Y ′ pixels by resolution conversion processing, and the reduced image is output (step S10).

  (2) The reduced image signal obtained as the output of the resolution conversion process is input, the waveform encoding process is performed according to a predetermined encoding method, and the encoded data is output. Examples of the waveform encoding process include H.264. There is an encoding method compliant with the H.264 / AVC standard (step S11).

  (3) The encoded data generated in (2) is decoded to obtain a decoded signal (X ′ × Y ′ pixels) (step S12).

  (4) The decoded signal (X ′ × Y ′ pixel) is input and reverse resolution conversion processing is performed to obtain a structural component decoded signal as an image signal (X × Y pixel) having the same resolution as the original image (step S13). ).

  (5) The difference between the decoded signal generated in (4) above and the original signal is calculated and output as a difference signal (step S14).

  (6) Using the difference signal as an input, a signal obtained by band-limiting the signal using a high-pass filter is output (step S15).

  (7) Using the band-limited signal generated in (6) above as an input, texture analysis is performed and a pixel value histogram and a filter coefficient histogram for each band of the directional filter bank are output (step S16). . The processing in step S16 is referred to as “texture analysis processing”.

[Flowchart (texture analysis processing)]
FIG. 8 is a detailed process flowchart of the texture analysis process (step S16). Hereinafter, the flow of the texture analysis process will be described.

  (1) The input signal is read and a histogram of pixel values of the input signal is output (step S161).

(2) The parameter direction number M that defines the Steerable pyramid transformation and the decomposition level J _max are read (step S162).

(3) Steerable pyramid conversion determined by the number of directions M and the decomposition level J _max is performed on the input signal, and a conversion coefficient is output (step S163).

(4) A histogram is obtained for 2 ^J × 2 ^J conversion coefficients belonging to the high frequency component, and output as H _J [i] (i = 0,..., 2 ^2J −1) (step S164).

(5) The following processing is repeated for j = 0, 1,..., J _max −1 (steps S165 to S170).

  (6) The following processing is repeated for m = 0, 1,..., M−1 (steps S166 to S168).

(7) A histogram for 2 ^{2 (Jj)} transform coefficients belonging to the mth direction component at the jth decomposition level is obtained, and L _x ^{(j, m)} [i] (i = 0,..., 2 ^{2 (Jj)} -1) (step S166).

(8) A histogram for “2 to the power of 2 (J−J _max )” belonging to the highest decomposition level j = J _max is obtained, and L _x ^Y [i] (i = 0,. 2 (J−J _max ) -1). Here, Y is (J _max , 0). At the highest decomposition level, there is no division related to the direction component, and it is limited to the histogram corresponding to m = 0 (step S171).

[Flowchart (decoding process)]
FIG. 9 is a processing flowchart of the image decoding device 3 shown in FIG.

  (1) The encoded data obtained as an output of the waveform encoding process is input, a decoding process corresponding to the waveform encoding process is performed, and a decoded signal is output (step S20).

  (2) Using the decoded signal (X ′ × Y ′ pixel) obtained as an output of the decoding process as an input, reverse resolution conversion processing is performed to obtain a decoded signal of X × Y pixels (step S21).

  (3) Using the texture analysis information as an input, texture synthesis processing is performed and a decoded texture signal is output (step S22). Details of this processing will be described later with reference to FIG.

  (4) Using the structural component decoded in (2) above and the texture component decoded in (3) as input, add the two signals and output the added signal as a decoded signal (step S23).

[Flowchart (texture synthesis processing)]
FIG. 10 is a detailed process flowchart of the texture synthesis process (step S22 in FIG. 9).

  In the texture synthesis process, the histogram of the Steerable pyramid transform is input, and Steerable pyramid is used for subband decomposition. An image obtained as an output of texture synthesis is called a synthesized image. Note that, at the start of the following processing, the composite image is initialized with the initial image.

  (1) As input, a white noise image as an initial image of a processing target image, each pixel value histogram for a target texture image (hereinafter referred to as a target texture image), each subband for both images Read the histogram. This histogram is called a subband histogram (step S221).

  (2) The pixel value histogram of the composite image and the pixel value histogram of the target texture image are input, the pixel value histogram of the composite image is approximated to that of the target texture image, and the pixel value histogram of the composite image after processing is output (Step S222).

  (3) Using the pixel value histogram of the composite image obtained in (2) above, the pixel value of the processing target image is corrected and a composite image is generated (step S223).

  (4) Using the composite image obtained in (3) above as an input, the directional filter bank is forward-converted with respect to the image and a conversion coefficient is calculated (step S224).

  (5) Using the conversion coefficient of each subband as an input, a histogram of the conversion coefficient is output (step S225).

  (6) Using the subband histograms of both the composite image and the target texture image as input, the histogram of each subband of the composite image is approximated to the subband histogram of the same band of the target texture image, and the processed histogram Is output (step S226).

  (7) Using the subband histogram obtained in (6) above, the value of the transform coefficient is corrected to generate a corrected value (step S227).

  (8) Using the subband coefficient of the composite image corrected in (7) above as an input, the directional filter bank is inversely transformed and a composite image is output (step S228).

  (9) The above process is repeated a predetermined number of times (steps S222 to S229).

  The image encoding and image decoding processes described above can be realized by a computer and a software program, and the program can be recorded on a computer-readable recording medium or provided through a network. .

DESCRIPTION OF SYMBOLS 1 Image encoding apparatus 10 Resolution conversion part 11 Waveform encoding process part 12 Decoding process part 13 Inverse resolution conversion part 14 Difference signal generation part 15 Texture analysis process part 3 Image decoding apparatus 30 Texture synthesis process part 31 Decoding process part 32 Inverse resolution Converter 33 Decoded signal generator

Claims (10)

A process of inputting an image signal to be encoded and converting the resolution to an image signal having a lower resolution than the input image signal;
A waveform encoding process for performing encoding processing based on waveform encoding on the low-resolution image signal and generating encoded data;
Decoding the encoded data;
A process of performing inverse resolution conversion on the decoded decoded signal and generating a decoded signal having the same resolution as the image signal to be encoded;
Calculating a difference signal between the input image signal and the decoded signal after the reverse resolution conversion;
A texture analysis process for generating encoded data of texture analysis information in which the difference signal is expressed by a histogram of pixel values and a histogram group of transform filter coefficients by texture analysis on the difference signal;
An image encoding method comprising: as an encoded stream, a process of integrating the encoded data by the waveform encoding process and the encoded data by the texture analysis process into an encoded stream. The image encoding method according to claim 1,
A step of applying a frequency band pass filter to the difference signal before texture analysis processing;
In the texture analysis process, encoded data is generated by texture analysis of a differential signal with a limited frequency band. The image encoding method according to claim 2, wherein
An image encoding method, comprising: controlling an expression accuracy of a histogram obtained by the texture analysis in accordance with a frequency band restriction amount for the differential signal. A process of inputting an image signal to be encoded and separating the input image signal into a low frequency band signal and a high frequency band signal;
A waveform coding process for performing coding processing based on waveform coding on the low frequency band signal to generate coded data;
A texture analysis process for generating encoded data of texture analysis information in which an image signal of the high frequency band signal is expressed by a histogram group of pixel values and a histogram group of transform filter coefficients by texture analysis on the high frequency band signal;
An image encoding method comprising: as an encoded stream, a process of integrating the encoded data by the waveform encoding process and the encoded data by the texture analysis process into an encoded stream. A process of generating a decoded signal by the decoding process using the encoded data by the waveform encoding process as input;
Restoring the decoded signal as a signal having the same resolution as the original signal before encoding by inverse resolution conversion;
The process of decoding the residual signal that could not be expressed by the waveform encoding process by the texture synthesis process using the encoded data from the texture analysis as input,
An image decoding method comprising: a step of generating a decoded signal by integrating two signals of a signal restored as a signal having the same resolution as the original signal and a signal obtained by decoding the residual signal. A resolution converter that inputs an image signal to be encoded and converts the resolution to an image signal having a lower resolution than the input image signal;
A waveform encoding processing unit that performs encoding processing based on waveform encoding on the low-resolution image signal and generates encoded data;
A decoding processor for decoding the encoded data;
A reverse resolution converter that performs reverse resolution conversion on the decoded decoded signal and generates a decoded signal having the same resolution as the image signal to be encoded;
A differential signal generation unit for calculating a differential signal between the input image signal and the decoded signal after the reverse resolution conversion;
A texture analysis processing unit that generates encoded data of texture analysis information in which the difference signal is represented by a histogram of pixel values and a histogram group of transform filter coefficients by texture analysis of the difference signal;
An image encoding device, wherein the encoded data by the waveform encoding process and the encoded data by the texture analysis process are integrated into an encoded stream as an encoded stream. A band separation unit for inputting an image signal to be encoded and separating the input image signal into a low frequency band signal and a high frequency band signal;
A waveform encoding processing unit that performs encoding processing based on waveform encoding on the low frequency band signal and generates encoded data;
A texture analysis processing unit that generates encoded data of texture analysis information expressing the image signal of the high-frequency band signal by a histogram group of pixel values and a histogram group of transform filter coefficients by texture analysis on the high-frequency band signal;
An image encoding device, wherein the encoded data by the waveform encoding process and the encoded data by the texture analysis process are integrated into an encoded stream as an encoded stream. A decoding processing unit that receives encoded data obtained by waveform encoding processing as input and generates a decoded signal by decoding processing;
A reverse resolution converter that restores the decoded signal as a signal having the same resolution as the original signal before encoding by reverse resolution conversion;
A texture synthesis processing unit that receives encoded data by texture analysis and decodes a residual signal that cannot be expressed by waveform coding processing by texture synthesis processing;
A decoded signal generating unit that generates a decoded signal by integrating two signals of a signal restored as a signal having the same resolution as the original signal and a signal obtained by decoding the residual signal Decoding device.   An image encoding program for causing a computer to execute the image encoding method according to any one of claims 1 to 4.   An image decoding program for causing a computer to execute the image decoding method according to claim 5.

Images